There have definitely been times when I honestly couldn't have cared less about the overarching plot of a game, or cared more about some tiny segment of it than the rest of it. For example, I honestly would have enjoyed L.A. Noire just as much if each case or desk had been their own self-contained story and they hadn't bothered with an overarching plot.
And it doesn't really bother me when others don't care about story. Plenty of games don't have them (or, at least, don't have one that matters). I have friends who actually skipped nearly all the dialogue in Fallout 3 and New Vegas because, to them, the fun was in the fighting and exploring, not listening to plot.