You don't need to spend a lot of money to get started, especially as there will be quite a bit of trial and error in determining your sizing and what styles and shades suit your femme side.
Basic stuff like panties, bras, hosiery and even shapewear can be bought from your local discount department store quite inexpensively or online if you are not comfortable buying these things in male mode. I would also keep an eye out on the clearance racks as there are often quite a few items in unusual sizings, which may suit your body shape as a male.
Things like breastforms, hip pads and gaffs can be quite easily made at home with cheap and easy to find materials. The internet has some amazing ideas on how to create these things effectively on a budget.
Once you have created your female shape, I would do some basic measurements to find out what size you will be in female clothing. Next stop is your local thrift store to buy some essentials like skirts, blouses etc. You will also find handbags, purses and even women's shoes in your size here.
As for makeup, I would recommend a cheap brand to start with. Yes the quality will not be there and it won't last a full day's wear, however at this stage it is all about finding colours that suit you and practice applying it. Once you know what shades work with your skin tones you can save up for the better quality brands.
As for wigs, I still have the first one I bought online for less than $20 and I still use it regularly. Again, it may be better to start cheap as it will take a few goes to find a shape that suits your face and style. I have found shoulder length hair with curly bangs to be most effective in hiding my high forehead and long shaped face.
As for your eyebrows, trust me, these make a huge difference. You can pluck your own quite easily and I would recommend starting with a few hairs at a time if you are worried about someone noticing. Then slowly work by removing the hairs from the bottom of brows, so as to push the brow further up your face, plus with a little shaping you can have brows that work in both male and female mode.
I understand your reticence about going out in public fully dressed, it is a big step and requires a lot of confidence, but so totally worth it. In the meantime focus on putting together your total femme package at home, which should totally achieve your goal of feeling more feminine.
Best of luck!!
Love GiGi xx