I like some games on the N64. I really enjoyed Mario 64, Zelda Ocarina of Time (but I still hate Majora's Mask) Paper Mario, F-Zero X, and I played so very much Mario Kart 64 multi player.
The problem with the N64, Saturn, PS1, Jaguar, and even the 32X in some cases is that early 3D/polygons are not pretty. Just like early 2D graphics on generation 2 consoles haven't held up to the test of time graphically. They were more utilitarian, like a set up game board or maze path. It wasn't until at least the 3rd generation 8-bit era where games started to become pleasant looking. This is analogous to generation 6 consoles which were powerful enough to take the 3D concept and make it so much better with more polygons, colors, and textures. Instead of people looking like they are made out of origami, they started having features resembling human.
2D games on the 5th gen consoles can really look amazing though.
There are, of course exceptions to every rule. And this is my own opinion.