Yes, your fears are somewhat normal and reasonably justified.
Your strength will totally go away and you'll end up a total girl. Things I formerly could toss around with one hand, I can't even lift any more. However, if you work out diligently, you can gain strength. You won't, however, develop any muscles with definition. This means you can work out all you want, you won't develop "manly muscles".
Next, you'll have to learn to think like a girl and learn the things that all girls know. You can't go out for a walk after dark, pretty much anywhere, if you're alone. You must always be aware of your surroundings and plan escape routes. You won't be parking in parking garages, especially after dark and you REALLY don't want to use stairs in enclosed stairways. There's a reason you nearly always see women out walking or running in pairs or groups.
Finally, do consider taking self-defense classes. Despite the fact that we can't hit very hard, there are a LOT of places a man can be hit that will cause considerable pain or damage. The best classes go on for quite some time and involve heavy exercise. These classes will allow you to develop the muscle and coordination to hit and kick very hard, the knowledge to know exactly where to hit without thinking and the reflexes to do it all quickly. In addition, you'll learn how to break forward and backward choke holds and to turn from being "attacked" to being "the attacker".
Like most all of the rest of us, I was super paranoid when I first stepped out the door dressed up to go to therapy. As I moved to full time, I became a little more comfortable. As time has gone on, I am no longer self-conscious.
As I progress in my self defense training and awareness of avoiding risky situations, I've become a much more confident female.
It takes time and it can be done. You have to first change your thinking, accept your limitations and develop your defenses, based on being female.