I have a 250GB hard drive in my PS2 and I play everything off it, even though I have a modchip. Load times are almost non-existant, and pretty much every game works. At least with the latest HD Loader (not the retail HD Advance rip off discs). In fact, I have never encountered a game that doesn't work. Sometimes you have to patch an ISO before you install it in order for it to work (FF XII, for example), but that's not a problem.
http://www.ps2hd.com/list/Go there for compatibility info.
Oh, IS possible to use a hard drive with some slim PS2 models after some heavy modification (solder an IDE connector to the motherboard - around 40 soldering points, all super tiny). You can only do it if your system has the model number SCPH-7000X. That's v12 and v13 of the machine, sold back in 2004. Newer machines cannot be modified.