Suzi, I can relate to this. In less than 3 months here, I have gone through a couple of changes in how I identify myself, and I still can't get over the feeling of being an impostor here as well as among women even if I were to transition.
Part of it is probably because like you, I was neither interested in 'traditional' girly stuff since childhood nor did I question my gender so strongly until very recently. Even now the dysphoria (physical, mental, social) is troubling and sometimes depressing but not as severe as it seems to be for so many others.
Another part is because I know I can never be a 'complete' woman (sorry but couldn't think of a better term), i.e. once you strip away all the stereotypes and even all the physical stuff that can be changed, what remains as the essential difference between men and women is the reproductive system, which (at least at present) cannot be fully changed. I know it's inappropriate to say because there are cis men and women who partially or fully lack it and they are no less of the gender they identify with, but still, it feels like a block which can't be overcome in fully accepting myself, let alone acceptance by others.
I think there are also skeletal and brain-level differences which cannot be changed but it seems our brains are anyway wired more to the side of female or wanting to be female as you said.
Yet another part is because I wonder whether I will miss some things or not like some new things if I were to transition. Will I be forced to change jobs, maybe my entire career path? Will I be forced to leave my wife? How will I react to male attention, chivalry, chauvinism, prejudice etc?
I have a certain way of looking at each of these right now but I am not sure if it will remain unchanged through transition, and if it changes, which one of that would really be me?