Whited00r is obviously not legal, since from what I can gather, they're shipping a heavily modified version of iOS 3.1.3 which brings a number of features from later iOS versions to devices that do not support iOS 5. This includes features like multitasking, reminders, an iCloud alternative based on DropBox, home-screen folders, Newsstand, custom backgrounds (without using WinterBoard), and lots more.
As you can see in the video below, it's pretty fast - although the developers stress that, e.g., Whited00r 5 on the iPhone 3G is not as fast as iOS 5 on the 3GS (which has more powerful hardware, so that makes sense). Still, it's pretty impressive.
It's not all sunshine and celery stalks, though. It doesn't come with the App Store application, so you'll have to install official iOS applications through the web interface on the device. It also comes with Cydia, as well as a Whited00r-specific application store. Carrier unlock and jailbreaking is built-in as well.
You install Whited00r through iTunes, and they have download links to Apple's official ROMs. Whited00r supports the iPod Touch 1G and 2G, as well as the iPhone 2G and 3G. Considering how fast this stuff runs, it makes you wonder just how much Apple is holding out on us just to get us to upgrade, but alas.