Oh, boy. Here we go.
Pre-agricultural man has absolutely no concept of monogamy or sexual fidelity. None, zero, zilch. Like agriculture, man invented the concept of monogamy as a way of further commodifying our basic needs under the (then) false assumption that resources (even sexual) were limited. Monogamy and marriage is bred of an extremely sexist impetus, where women were expected to barter their sexuality in exchange for food and protection, which are unnecessary concerns in a (natural) egalitarian setting. For that matter, paternal certainty and the import we place on lineage is an invention that also sprung into existence to establish the passing of property, in response to agriculture, which itself is an arguably unnatural occurrence in our species, and was in all likelihood the product not of limited resources but of megalomania, an outlier behavior that spread through greed and violence (which are sociopathic behaviors, and for the purposes of this discussion: "not natural human behavior" but learned, for most), and not through any evolutionary impulse. Behaviorally, yes, the illusion of scarcity makes us hoarding, jealous, violent people. This goes against our very nature, and this entire problem (monogamy IS one of the problems) stems from the invention of agriculture.
I know that these are hard pills to swallow, but bear with me.
We evolved in a setting where reproductive sex was secondary to sex as a vehicle for social bonding and tension release. In no setting without agriculture (and, for that matter, without gender inequality) does monogamous pair-bonding occur naturally. Agricultural man is a relatively new age for man, and comprises just 10,000 years of our 200,000 year history, and that's not even factoring in the many hundreds of thousands of years prior, linking us most notably to the ultra-egalitarian and sexually promiscuous Bonobos, whose very survival (like early man's) was contingent on the impulses to share not only food but sex as well. These same impulses survive in us today, but we are a very very repressed species whose social conditioning puts us at odds with ourselves constantly.
Let me address the hyperbole that is "agricultural man" and the absurdity that I can purport to say what man was like over 10,000 years ago. To this very day, over a 100 seemingly-disparate tribes exist throughout the world who share some very fundamental things in common, the most important among them is their lack of agriculture. Like early man, they exist today in the most natural setting imaginable, where megalomania and hoarding and sexual stinginess were regarded as detrimental behavior to society, and such individuals are outcast or swiftly punished. Most of these tribes have no words for our concept of marriage (and it baffles them greatly), no words for the concept of war or even murder. When everything (people included) is shared, there is absolutely no need for violence or sexual jealousy, impulses that we fight every single day (which should be indicative enough that "we're doing it wrong"). These fiercely egalitarian tribes behave the way they do because it maximizes not only their chances of survival (when everyone truly cares about each other, and children don't even know who their "real" father is), but because a sexually promiscuous lifestyle reaps so many social AND health benefits.
Marriage is not a human universal. It's a logical development when resources seem scarce and we want to ensure that our offspring are the sole owners of our property, but this is far from a natural impulse. Like the bonobo, human females are the only other species who remain sexually receptive throughout all cycles of menstruation. Like the bonobo, instances of bisexuality are not outliers but are, in fact, the norm, and studies testing what individuals SAY they're attracted to and what their genitalia is saying have shown that bisexuality is more of a rule than an exception. If marriage and offspring bearing are naturally occurring in our species, bisexuality would not even exist. And yet, here it is.
So, the idea of monogamy is especially baffling in the face of female orgasm. Let's overlook the fact that group sex is the most sought after porn by far, and instead look at a simple truth: women take far longer to reach orgasm than men. The reason for this is that, again, we evolved in a setting where women had multiple sexual partners simultaneously, and it was (and really, still is) utterly necessary for several men to do their thing for all individuals involved to reap the full benefits of the sexual experience. Did you know that the uhh, "sounds" a female makes while mating are (like our bonobo cousins) meant only to arouse and bring other men into the equation? Did you know that the first and last spurts of male ejaculate contains a spermicide so that we can "compete" in reproduction after ejaculation and certainly not before? Did you know that the simple act of genitalia plunging and thrusting removes something like 80% of ejaculate? Remember what I said about hunter-gatherer tribes not having paternity certainty, and how children were raised in an environment where every male in the tribe cared for them as their own? This is how prehistoric man developed and what our bodies are still fine-tuned to respond to.
We have a genetic predilection for sexual novelty, and we are absolutely unable to stay sexually excited with one partner without a variety of tricks like, say, closing your eyes and imagining another partner, or role-playing, or using all sorts of jump-start techniques to make the act exciting again. Sexual dysfunction is a misdiagnoses, and we wrongly assume that something is wrong with us or our partner. Our bodies are telling us something. They're not saying "you picked the wrong person to mate with" or even "spread the seed" or "make babies." They're saying we need to socially bond with many others and that we were meant to do this in the most intimate settings.
All the different schools of thought and terminology we use to describe differing sexual behaviors aren't helping the matter. We are meant, as a species, to love and care for those around us, and both men and women have a need not only for physical and sexual intimacy from those around them, but a need for novelty as well. Like the Muoso tribe, I call it friendship, and the idea of romantic foreverness is never naturally factored into my attraction and egalitarian impulses.
Blah blah, I could gab about this for days. Read "Sex at Dawn" if you want to continue the discussion with all the science weighed in rather than conjecture of what the people immediately around us are doing.
tl,dr: "It's wrong, it goes against our base animal instincts."