I'm going to say for primary care it doesn't matter so much.. but as far as endocrinologist- this one does matter.
I can't imagine my transition with a different endo from the one I have. I owe a lot to him, I feel.
When looking for endos, look for ones that have treated many trans patients- preferably a trans specialist. They are rare but they do exist

Basically why it's so important is you don't want to be gatekept on low doses or shoddy forms of administration. Also a lot of endocrinologists don't truthfully care about the health of trans patients and will prescribe you anything.. Which could lead to you being quite the unhappy camper physically and/or mentally.
Hormones are serious biz. Definitely worth doing your research on endocrinologists.