If you think that's a punishment, then you must think that men are currently being punished for not being allowed into women's restrooms and changing areas, right?
Let me throw some hypotheticals at you.
You are out with your baby, and you need to change them, but the only changing facilities are in the women's bathroom. What do you do? Do you change your baby in public? Do you take them to the men's bathroom and put them in the sink? What do you do with the used nappy? Do you put it in a regular bin (and break the law?) Do you wait until you get home and risk leaking baby shit everywhere? Or, do you just use the women's bathroom?
Alternately, you are out with your young daughter. She needs the toilet, but is too young to go by herself. What do you do? Do you force her to hold it in until you get home? Do you take her to the men's toilet and force her to go to the bathroom in front of a bunch of adult men? Do you ask a random stranger to escort your child to the bathroom? Or, do you just take her to women's bathroom and let her do her business in safety and comfort?
The reason conservatives never think of how impractical it would actually be to actually ban men from women's bathrooms is because conservatives struggle to imagine men as active parents. That line of thinking is punitive to men, it prevents men from being equal partners in parenthood and relegates them to the role of providing awkward moral support to the women who actually do things. Even as a cis man, you are perfectly entitled to go into a womens' bathroom if you have a reason to be there.
You need evidence that men attack or creep on women in bathrooms and changing spaces?
Also, can I say how funny it is to see you suddenly pretending to care about sexual violence.
I agree with you though, men are trash. All men. Without exceptions. It's only natural to assume that any man is a violent, degenerate rapist. That's why, instead of targeting innocent trans people, we should be focusing our efforts on restricting the ability of men to move freely in society. As a modest proposal, I would propose some kind of mandatory collar which can deliver electric shocks to men who are giving in to their natural subhuman urges. Long term however, I think the only real solution will be to drastically reduce the male population through sex-selective abortion, so they can no longer pose a risk to normal people.
That way, noone would ever have to worry about using a bathroom. The remaining men would of course have to pee inside transparent boxes where they can be observed for signs of relapse.