I think the other side gets it wrong by treating someone's identity as an argument to shut down. People can know themselves in ways outsiders can't, and calling that sexist feels like a stretch to me. I am not sure how to fully explain it, but that's my gut.