kirk's bestof series
Now, I don't rate my opinon as a gamer that highly, but on most of the forums I'm on, everyone once in a while someone will ask "what's the best game for [INSERT SYSTEM HERE]" so I thought I'd finally gather my thoughts and make answers that are canonical (and cut-and-pasteable!) for me, along with some background info on why these games are so important for me...

Games are listed in roughly descending order of coolness.

Mega Man Series
Oh man...this game just resonated for me, one of my first big exposures to Japanese cute robot design. Beating bosses in cool one on one battles and taking their weapons was excellent.
Bionic Commando
Another terrific 2D platformer, swinging with the bionic grip thing was a great dynamic, and would could be better than putting a bazooka in Hitler's ugly mug? Plus had a cool "let me tell you a story" framing device before and after the actual game.
Blaster Master
Created great sci-fi worlds, great visual design. The tank upgrades were great, especially the way the wheels rotated down to hover ala Back to the Future. Hampered by its "big head guy running around" 2D sections, and an incredible spike in difficulty on the final level.
This game is why I got an NES rather than living with my superior-in-many-ways C=64...when I saw it in the black-covered "Player's Guide" I just had to have it. Of course that's the same guide that probably turned me into the walk-through using wuss I am today.
Except for some Gameboy games, the only 2D Mario I ever beat. I really liked the ending, it was just one of those "oh it was just a dream" but it was well done.
Zelda 2
Again, the only 2D Zelda I've beaten, and again a game where that "Player's Guide" was a huge help. The sword combat was really well done.
Just classic 2D co-op action. Up Up Down Down Left Right Left Right B A Start!
Sentimental favorite (one of the first games I got) but with some nice co-operative action.
Solar Jetman
Too difficult but with a nice kinetic feel.
Micro Machines
Cool overhead racer...I think it actually had kooky 4-players-using-2-controllers support.
Honorable Mentions:
Smash TV / Archon / SMB3

Tetris Attack
The best head to head "puzzle" game I've ever played. Lots of games let you send garbage blocks over on the other player, but few make those garbage blocks the key to turning the tables and getting sweet, sweet revenge. Nintendo not bringing its GameCue Puzzle Collection (including a 4-player version of this game) to the USA is a misdemeanor against humanity.
Donkey Kong Country
People who dismiss this game as shallow eye candy probably haven't had the plasure of walking a newbie gamer through all its well-designed levels and boss fights.
Super Metroid
One of the pinnacles of 2D gaming, a great feeling of exploring an alien world.
Fun 4 player 2D racing with amusing board design, taking toy cars around real world environments.
Stunning (for the time) polygon graphics and solid rail gameplay, but I think hampered by the crosspad.
Yoshi's Island
A terrific platformer. The grab Mario before he gets carried off makes for a wonderfully forgiving mechanic, the bosses as magically overgrown regular enemies is nifty, and the monkeys darting around make for a lovely game.
Super Mario Kart
I want to like this game more than I actually StarFox, I really played more of the N64 version first and like it better, but I have to give this game its props.
Super Punch-Out
Great big wonderful cartoony graphics...a worthy followup to everyone trying to beat up Mike Tyson on the NES.
Smash TV
Excellent port of Robotron's spiritual descendent
A decent head to head game, with elements of Tetris and those Artillery Duel games...lacking of the arcade's 3 player support hurts it.
Honorable Mentions:
F-Zero / Super Mario World / Pilotwings / Super Mario All-Stars

Super Smash Brothers
It's like someone nabbed our middle school gaming daydreams..."wouldn't it be cool if you could, like, fight Samus from Metroid vs Mario vs Link vs Kirby, all at the same time, and they had their weapons and powers and stuff?"
Mario Party 1 2 3
They started to run this series into the ground (what are they up to, like 6? But the old ones still maintain their high prices) but the concept is brilliant...bring Classic Style gaming into a modern setting. Only the first one gets the board game thing right (the stadium, as minimal as possible) but the minigames are still maximally fun.
Mario Kart 64
This game was the first showcase for the N64's fantastic 4-player abilities, and is still a boat load of fun.
Diddy Kong Racing
Others disagree, but I thought this stood head and shoulders above Mario Kart 64...the Kart engine wasn't quite as good, but it included hovercraft and planes, and battlemodes that were actually fun to play. Plus it had a one player and cooperative "adventure" mode that was terrific...boss battles in a Kart racer, whodathunkit?
Battle Tanx / Battle Tanx: Global Assault
Sadly, there are no "tank" games better than these on any console since the N64...the PS2's "ThunderTanks" (by the same company) completely dropped the ball, but these games have great 4 player deathmatch and single or co-op adventure modes.
Rogue Squadron
Wow. There is something about flying an X-wing that will always have a direct plug into my inner child. More approachable than the PC's X-wing games, largely ground-based missions, but Oh So Cool.
StarFox 64
This is the game that sold me on the system...the first time system I owned while it was still current since the NES. Just the beauty of the start of the first level, you and your wingmen skimming over the water's surface...
Pokemon Puzzle League
It's just Tetris Attack with fewer animations and more sounds, but Tetris Attack's seesawing-"turn the tables" gameplay makes it the best head to head puzzle game in the history of mankind, so....
Space Station Silicon Valley
Quirky but clever puzzle solving game where you had to work and gain control over a wide variety of robocritters.
Blast Corps
Another puzzle solver, this time controlling a variety of unique robots to clear a path for the nuclear warhead carrying truck.
Honorable Mentions:
Super Mario 64 / Zelda: Ocarina of Time / Mario Tennis / Blitz 2000

Rogue Squadron: Rogue Leader
Wow. As great as Rogue Squadron was for the N64, this game just took it to a whole new level. The battle over Endor was jawdropping.
Super Monkey Ball 1+2
Can't stand the one player mode but one of the finest multiplayer options for the system...the racing, hang-gliding/target, and punch were all nifty in the first game and much improved in the second.
Super Smash Bros Melee
A lot more frenetic than its N64 ancestor...too bad you have so much you have to unlock in one player mode!
Mario Kart Double Dash
Another worthy sequel.
Mario Power Tennis
Another minor improvement over the N64 version, but solid 4-player fun.
Pac-Man Vs
Great fun for dedicated gamers and the non-hardcore alike, using the GBA as a small screen for Pac Man while everyone plays the ghosts.
Luigi's Mansion
A nice short story of a game, with nice show-off-the-system graphics.
WarioWare, Inc
Hee! Short Attention Span Theater for gamers. Could stand more simultaneous minigames, though.
Time Splitters 2
A quality FPS by the same team that brought us Goldeneye on the N64...nice co-op mode.
Metroid Prime
I mostly watched my friend chug through this, but it really captured the mood of the original and made the transition to 3D fairly smoothly.
Honorable Mentions:
StarFox: Armada / Zelda: The Wind Waker / Super Mario Sunshine