To be %100 fair, the GBA probably is the best handheld ever. Its more or less as powerful as a SNES (greater in some ways, inferior in others) which means its basically all 2D games taken to their limit. It can do some polygonal bullshit, but luckily there is very little of it. The GBA library is amazingly massive and diverse, and the Micro and SP have some of the best screens and form factors of all time. The SP is probably the best since it can play games all the way back to b/w GB. Advance Wars 1, Astro Boy, Super Robot Wars J, Initial D, Drill Dozer, Metroid Fusion, Warioware Twisted, damn this system is good. People need to not get so hung up on the stuff like Bratz and Barbie and Army Men. It doesn't matter that shitsoft was made for it, what matters is that great stuff was made for it.
The PSP just doesn't have the diversity in software. Its more of a "big gun" in the sense that I'll buy a PSP game and get SUPER into it for quite a while...and then it just doesn't get used because there isn't enough software. The screen is amazing.
The DS is like this too. I'll get really into a Layton or Rhythm Heaven game and then put it back on the shelf.
The NEC handhelds aren't that great. HuCard-only battery hogs with unreadable screens. Very comfortable to hold, and absolutely cutting edge for the time, but very little use today. The LT isn't a handheld at all unless you wear a f*cking battery belt.
The NGP is probably second place for me. The game library is small, for sure, but they are such charming games. Aside form the Pachislot BS they are ALL amazing. Sure, the screen isn't backlit, which is a pain, but that's part of why you get FIFTY f*ckING HOURS of battery life. Try that on...anything else. Then there is the fact that this thing is SNK all the way and has the best 8 direction stick/pad to ever appear in the history of video games.
Well, I think I finally disagree with Zeta. While I will say there are some excellent games on the GBA, it's a tiny percentage of the overall (GBA) library. Don't get me wrong, I love the thing, and I own 10 of the f*ckers in the various form factors, plus most of the great non-ridiculous $$$ games. But the fact is, I only own about 30 games, some which aren't all that, and there's very little else I want for the system.
The PSP I have about maybe 25 games for, all of which have more depth than anything but the very deepest GBA games (and I count Yoshi's Island there <3), and many of which I've been really sucked into for 10's to hundreds of hours. I'm talking:
Burnout Legends
Ace Combat X
Streetfighter Alpha 3 Max
GTA VCS
GTA LCS
Medal of Honour Heroes
EchoChrome
metal gear Peace Walker
Monster Hunter [whichever you want]
There are more, and I'm sure, careful though I am, I haven't picked out the only PSP games I like. Plus with its video and Internet capabilities, the PSP is the greatest handheld console yet made. I also like the DS, but for specific games like Layton and Ghost Trick.
In its own time, I'd say the Gameboy (original) was probably the greatest handheld of all time, just for library, longievity, and being so forward thinking yet perfectly marketed. It's an icon for a reason.
But, I still love the PC Engine, and a full colour handheld that plays the entire library of PCE HuCards, was MINDBLOWING at the time, even with its disadvantages. There was nothing like it, and the lynx was ass compared.
The NGPC came out way later, and while it was well considered, like a colour version of the original gameboy, with some neat 3rd party games, it was a bust. It came out too late, not every game is great, and even the good ones lack depth. except maybe Metal Slug and Card Fighters (if I can work out how to play it), and the latter was riding on the coat tails of Pokemon.
I like the NGPC, but whether we're talking all-time, or in it's own time, it was never a "great" console. At least it wasn't ass though (lynx).
Here's my Gamefaqs review of the Lynx
http://www.gamefaqs.com/lynx/916379-lynx/reviews/review-147593