I'm no expert on the quality of life that pornstars enjoy. I imagine it's similar to regular acting; the most famous stars get treated rather well, and it quickly tails off the further down the pyramid one gets.
It's a life choice that they made, so I wouldn't view them as totally helpless, aside from those victims of sex trafficking or child abuse (who clearly deserve sympathy and justice of course, there's no argument, but this thread isn't about them).
However, even if it is a choice, it's certainly not a job that I would enjoy doing. I do hear that a lot of male pornstars get bullied into doing gay porn if they want to get paid. The women probably expect a glamourous job, or do it out of desperation, and then get treated like shit and socially ostracised. So yeah, it probably sucks for a lot of them. Should you feel bad about watching porn? Like J Tyran posted above me, probably about as bad as you should feel for, say, buying an iPad or something. Unfortunately, our monkeyspheres only extend so far, and it would be difficult to care about every person that might have been tangentially wronged through our consumption habits. Unless what you're viewing is illegal, or clearly distressing for the participants, then yeah. Watch away.