Man, Everyone is picking smell....but here's a thought.
Sight would be by far the worse. I would pick death over blindness...
Touch would almost be just as bad. You might say "But then you'll feel no pain," but take it from someone who has lost a pint of blood without being aware of it that this is not what it is cracked up to be.
Taste might be cool for a moment to lose, but then you will lose the ability to tell if you are eating something you really shouldn't. Eventually, all the puking you would do would make this suck.
Smell is compelling to lose, but it also has its disadvantages. You won't be able to know if you are slowly suffocating in a room of methane for instance. It would also affect taste as well, so see above.
So I picked hearing. Not only would I not have to wake up to screaming children, but I would also use my eyes and nose more. And I know what you are saying "But then a ninja could sneak up and kill you without you ever knowing!" That's what they invented the mirror for.
T