Starting afresh somewhere else can be a good move, no history, no need to explain to people you rarely see, and the chance to surround yourself with a network of the sort of friends you need now. It sounds like your old friends have stayed the same while you have grown, so it's probably time to seek new ones.
I was talking to a lesbian couple I know well, this afternoon, and one of them said the same thing to me about my impending move interstate. If you are only associating with the sort of friends you prefer, obviously they will just want to see you happy. End result, little need to worry what the outside world thinks.
Hope your move goes well.
Karen.