In this day and age, relationships should be exactly as exclusive as they're agreed upon to be. If you can't or won't limit yourself to just one sexual partner, but still want to have a primary relationship, that's cool... but make sure that everyone involved agrees that the relationship is an open one. If you can't or won't bring it up, if your relationship isn't important enough to discuss such things honestly, then I think you have to re-examine why you're with this person to start with.
That said, absolutely. As a guy, I would definitely back off from anyone in a relationship already, unless I was told that the relationship was an open one. Whether that in itself would weird me out too much is something I don't know yet, but you don't interfere in exclusive relationships.