Hi, Stacy.
I feel for your story and empathize more than you can know.
When I was divorced (time one)

my oldest daughters were in low single-digits in ages. As they got older we saw one another less and less. I was in Europe for a few years and their mother moved to the wilds of the Pac Northwest after re-marrying.
It was odd though. I'd send letters about every two months. Never got any back, but did get a few returned as "not longer at this address." I discovered that at one time the addy I'd been given was for a bank on the other side of the country from where they lived. That was somewhat startling.
But as they got into their teens we had verbal contact made at their initiative. "Momma and our Dad told us you were queer and bad. I didn't remember anything bad about you. In fact, I've always imagined you as a sort of gallant prince who could somehow someday come and save me." (Her mother had attempted to strangle her when she was 17. She went to live with the step-dad after that as the marriage had broken about a year before.)
Anyhow, when I began transition I spoke with them both. I was living in TN and they were both still in the Pac NW. It was odd for me, telling them with a lot of worry and then how they responded. One, who now lives w/ her financier-husband and three children in Marin County was enthusiastic about the changes. Altho she did later tell me in response to some pics I'd sent of the "product,"

"wow, you look really good! I was sooo worried about that. I really was hoping you weren't gonna look like a guy and you don't. I am so happy about that."
(O, and all those letters that did arrive? She still ad them and would read them again and again.)
The yoinger one has her evangelical beliefs obtained from a foster-family her mom had placed her with when she was 14. She doesn't think what I did was "right," but she lives with it fairly comfortably.
Both women have made trips to our house and we've seen them and their families a few times. This summer we'll be meeting the oldest and her family for a few days at their Cape Cod "getaway."
I guess my point is that not everything is of necessity going to turn out horribly. Apply understanding and, in my experience, don't subject the children to any bad-mouthing of the other parent. Their lives are not the appropriate battlegrounds for those kinds of wars. I never did and both say now they found that good and that it made them less-inclined to accept as true what they were being told at home.
I hope this all works out well for you and for your son. The pain of those losses and missed time can be excruciating psychically.
Nichole