Halo 2 did it for me. After absolutely loving Halo: Combat Evolved (being the first FPS I properly played and enjoyed right the way through, apart from the level, "Keyes", UGH), I found Halo 2 really disappointing. The timed run at the end of Halo was exciting, and at first, challenging. I was a bit staggered when I found myself upon a boss battle at the end of Halo 2.
A boss battle?? That's the sort of thing reserved for RPGs and JRPGs and anything OTHER than the Halo series. Plus the guy had a shield and the stupid hammer that wiped out most of the AI help at the beginning, so the odds were stacked against you. I just hated how Bungie thought they had to include it to spice it up a little.
I also didn't like the level design in Halo 2. Sure, they didn't do the ol' "lets go through it again, but backwards," that Halo suffered from, but the levels themselves were pretty samey. Corridor after corridor introduced Flood spore after Flood spore... I found myself not caring if the Master Chief's helmet fell off and the Flood infected his brain.
I even preferred the music in Halo: CE. And Halo 3. Halo 2's soundtrack didn't do it for me at all.