Windows XP only: Today's USB flash drives are huge, but they come formatted with the FAT32 limit of 4GB files—if you want to format them as NTFS under Windows XP you'll need a little trick.
Windows XP does have the ability to format drives with the NTFS file system, but you wouldn't know it by looking at the format dialog—normally the option is disabled. To enable it, open up Device Manager and find your USB drive, go to the Properties -> Policies tab and then choose "Optimize for performance". Once you've done this, you'll see the NTFS option in the format dialog.
Readers should be warned, however, that once you've enabled write caching you will need to use the Safely Remove Hardware dialog to avoid losing data—though once you format the drive as NTFS you can switch the write caching back off.
The choice between NTFS and FAT32 isn't cut-and-dry—while NTFS does allow larger file sizes, encryption, compression, and permissions, there's a lot more overhead to using it—and more importantly it won't really work on non-Windows systems.