Jenny,
Unless it becomes known you're "home" and people have their eye open looking for you, no-one's going to recognise you, honestly.
And in my experience, the women I know have never had any objection to me using the Ladies, in fact in pubs and bars they've used it as an opportunity to chat with me in peace and quiet while we brush our hair, touch up lipstick ....
I recently had a woman I knew well chat to me at a wedding anniversary (me sitting between my mum & dad, with just five others making up our table) .... she had known me and my folks since my childhood but did not know of my transition. I could see she was really puzzled as to who I was, she even asked people "who was that woman ? ", but got evasive answers as no one wanted to break my confidence and "gossip" !
Don't worry and enjoy being back with family and friends ....
Laura x