In my experience things like your face and body fat distribution will change enough that they look different than before. Unless you live and work in a place where this is common it's doubtful anybody will think HRT. They may notice and talk though. They do about me but I have grown my hair long and don't really try very hard to conceal anything. Even with that, it has not caused me any problems and the novelty has worn off for most of those that were joking around before.
Sapere Aude