To be honest, I wouldn't think there is anything malicious going on. If I saw that there was different outfits for genders, I might refer to the person based on their outfit (depending on the circumstance). While that is problematic in many ways, I don't want to assume anyone's gender. For all you know some people might think you are a transitioning ftm and wouldn't want to insult you by saying she,lol. Maybe I think to deeply about this because of my experience with GID, but I look out for these things. However, I understand what you are saying.
If you are ready to come out, it might be a good idea to talk to your employer. Maybe they will let you wear the scarf instead of the tie? If you are worried about your employer discriminating against you, I would check out their companies policy. They might be accepting of this, though there are always some risks. Also, I would research discrimination laws within your locality. I can't imagine Sweden's anti-discrimination laws wouldn't cover gender identity, but since I live in the U.S, I can't say for certain.