For my legs, HRT did reduce the muscle mass and slightly increased the fat on my upper thighs and rear. A daily exercise and diet program will make the legs stronger and help keep the thigh fat from getting to large.
I do lots of squats and leg raises in my exercise routine. For my diet, I eat lots of vegetables. I do not starve myself. I try to eat enough of the right kind of calories to maintain my desired weight. Also, there is genetics. If all of your relatives are skinny, expect the same.