As long as something does not cause harm to real people I'm fine with it. Whether lolicon causes any harm is debatable, but, like with violent videogames, I've yet to see any compelling arguments to support the hypothesis that it is indeed harmful.
I find that many of the things that are depicted or shown in hentai are not suited to my tastes, or are things that I find repulsive. But like any other medium, it is mostly a simple matter of personal taste.
As for the argument that situations that are depicted in H materials that would be extremely immoral and illegal in real life should also be as immoral and illegal, I only have this to say: you need to distinguish between reality and fantasy. Setting philosophical debates about existence aside, reality is very different from drawings, animations, pixels, and vertices. To my knowledge, fictional depictions do not think or feel. They don't, because they can't.