I would vote N64 for sure.
Nintendo has always taken shortcuts when making consoles. Amusingly, the N64 is the one they took the
fewest shortcuts on, followed by the Gamecube. All of them were severely hindered by one or more crippling design decisions...
-The original NES was crippled by bad overall hardware design, giving it a high hardware failure rate in terms of software loading. The re-release fixed it, but by then, the system was already obsolete. This is the only console in the list, IMO, which was crippled by bad physical attributes rather than electronic moronicism. Of course, one might also fault it for the lackluster color encoder, but it was usually only Commodore 64 fans who bitched about this.
-The SNES had an amazing graphics processor for its time, and the same can be said for its sound processor. Its shortcut comes in the form of its pitifully slow CPU and lack of VRAM. The system could have been the slayer machine, but the low VRAM and terrible CPU speed, and not to mention the horribly slow ROM access time, crippled it. "But it had the same VRAM as the other machines of its time!" Yes, it did... and that made it underwhelming for what it could have become.
-The N64 can be only faulted for its lack of system RAM and its adherence to solid state storage in an age where it was being readily replaced with cheaper media. The latter wasn't a crippling factor but the former was.
-The Gamecube was primarily crippled by its inferior graphics processor and foolish media choice.
-The Wii... wow, where to begin here... it's little more than a Gamecube with an additional GPU and some more system RAM... the ultimate shortcut machine. By the time the Wii went to manufacturing, the components had become so cheap that Nintendo started *making* money each console sale rather than losing it... a phenomenon not really seen since the Atari 7800.
Now, these are all relative to other consoles of their respective eras, so keep that in mind.

Most consoles throughout history have suffered from some kind of limiting factor that could have been avoided... the Genesis and its pitiful color palette, the Playstation and its terrible 3D hardware, etc... but there are also consoles which hit the formula perfectly when compared to their direct competition... the Dreamcast (slaughters the Playstation and gives the PS2 a serious run for its money) comes to mind here, as does the console this site is based on... the PC Engine (slaughters the NES).
If only hardware perfection translated into software quality... but alas, it doesn't, and each console's success has always been based not on technical prowess but software viability. Even when a game SUCKS MAJOR ASS, if you convince enough suckers to buy it, it's a success.