I'm envious of little kids, not babies, they don't know what's going on. To me I might as well be envious of a male dog than a baby. But where does that get me? No where except further down into the pit of dysphoria.
Change the things that can be changed, and learn to live with what can't be. It's not easy, but there's no other way to actually live your life, not just exist, but live. Dwelling on what could, or should, have been does nothing but cause harm.
There's a quote that I'm always reminded of when seeing young boys, "It's never too late to have a happy childhood." Yeah, not all of my childhood and upbringing was perfect, or right for me, but it's not too late to experience those things. With your own children, having a laugh with friends, or by watching other kids growing up. I had more of a childhood watching two kids briefly over less than 9 months than I did in my own life. That sounds creepy, but seeing the interaction between them and their father filled me with hope for when I was in his position, and the gaps I'd missed in my own life.