Audio balancing.
If there's one thing that turns me off of a game, it's bad audio balancing. Bioshock Infinite is a prime example of this. Overall a very enjoyable game with some cool stuff going on, but the fact that the audio balance is so fundamentally broken makes playing through it way less fun than it should be.
Right from the very beginning of the game, the "press start to begin" screen opens with a CLICK that sounds more like a rifle shot, and things go downhill from there. Elizabeth's infuriatingly breathy dialogue is barely audible, as is Booker's dialogue which consists, as Yahtzee put it, him looking at increasingly impressive things followed by him saying "huh."
This sound, which necessitates putting my TV up very high in order to hear, is in the same audio balancing track as the enemies screams, shouts, death cries, and general loudness, to say nothing of the voice recordings, radio broadcasts, civilian dialogue, and general noise of Columbia, which is easily three or four times as loud. It's insane.
Fighting a group of enemies is one thing, as is the music (I did manage to turn down the sound effect and music volume to manageable levels, though it did mean that I couldn't hear the record players most of the time), but when hitting four enemies with Murder of Crows results in four grown men screaming into my ears at the same time, somebody hasn't done their job properly.