Get an Xbox hack it and play whatever you like, even better install a large sized HDD and get a PS2 to Xbox pad converter
Xbox games
PC games via Dos Box
Point and Click games via SCUMMVM
Arcade system games
NES games to N64 games
GameBoy to GBA games
Sega System games up to Mega CD/32X
PSX games
And lots more
Component Cables can also be applied for Hi-Def gaming though best to go official but these are kinda hard to find now
I had well still have a GP2X that was a nice little handheld console for developers but the PSP at the time over took that with ease so now it collects dust, the PSP is another option but you are much more limited to what can be run on this, but it is great for mobile gaming and can run a lot of emulators though some are not perfect most games run very well
Please Note all N64 emulators will have problems with certain games, there is no perfect N64 emulator even after tweaking settings
I also like using my Wii for Lightgun based games, that is ideal because of the funky controllers, overall the Wii is also another great console for playing other emulated systems on.
Even better with an external USB HDD and Gamecube Ports for GC controllers or PS2 to GC controller converters, though I do like the GC pads anyway, they do have much the same layout os most of todays pads
Most consoles have emulators for them
The Xbox is one of the easiest to mod and has a good range of working emulators for it (a bonus it can still play Xbox games that there is no usable emulator of)
The PSP is also very easy to mod and has a good range also just not as good as the xbox does, but does play PSP and officially emulated PSone classic games (N64 will be a bit of a let down here though, also the PSP lacks a few buttons compared with an analog pad so you are limited to the controls used for some PSX games)
The Wii that although is not as good as the Xbox for emulators does let you play a lot of great games Wii + GameCube for starters a bunch of officially emulated games (Including N64 games) before we even start with homebrew emulation that is also vast in the systems that can be emulated on the system
All the above system are easy enough to mod, I would say out of those the Xbox is maybe the harder one to do mostly because you may need extra hardware that you may not already have to implement the initial mod, the Wii is very very easy as is the PSP, but you may want large storage for all consoles, Xbox uses IDE HDD's though you can get a SATA to IDE converter working there, the PSP 1K 2K and 3K versions use Magic Gate Pro Duo memcards but I use a dual Micro SD to Pro Duo converter with 2x32GB micro SD's inserted giving me over 50GB of space (It's the stupid way the space is calculated in 1000's instead of the real amount that should be 1024's hence I don't get 64GB from 2x32GB cards and for the Wii it is best to use a USB HDD (USB 2.0 or higher, the Wii USB port is USB 2.0 though)) I use a 2TB USB 2.0 Wall powered WD drive though you don't need a wall powered drive most go for USB powered drives.