Here's my 2 cents worth about handheld retro-gaming/emulation...
I agree, the Caanoo looks cool, and quite promising. I couldn't really, with any conscience, talk you out of buying one. However, if you're only looking at console emulation, a PSP with custom firmware would be nearly as good, I think.
Looking at the OpenHandhelds webpage for the Caanoo, many of the same emulators listed there also have PSP versions. For instance: PicoDrive (MegaDrive and MegaCD), MasterBoy (Sega Master System and Gameboy/GB Color), MAME4all (arcade), gpsp (Game Boy Advance), and DOSbox (vintage PC gaming). Not sure about the Caanoo, but the PSP has an app called "Popstation", which uses the PSP's native ability to play PSone games. I didn't see a PSone emu at OpenHandhelds.
The choice is yours, really. The Caanoo is cool, but PSP consoles are far more common and easy to get hands on. Plus, now with the invention of PROcfw, they are super easy to soft-mod (Even I could do it, and it only took me about a half-hour). The Caanoo being Linux-based is a neat feature, and opens up new possibilities... you just have to decide how important those possibilities are to you, and how often you'd use them.
Best of luck, and let us know if/what you get for yourself, OK?