Even if someone called me a “male” constantly it would kind of freak me out. They are theoretically interchangeable, but male or female just comes off less “human” I guess. If they choose to only ever say male or female then it makes you wonder if they literally perceive you as inhuman.
Always is a real stretch, but I get your point. The reason is that being angry is easier than being happy.