I get jealous around cis-males, but it's getting much better now that I am on T and am seeing some changes. When hormonal and top surgery factors are considered, there is not much that is different between us and cis-males. The only thing significant is chromosomes (which isn't really significant except for genetics), reproductive ability (although this varies greatly between people), and the appearance of the genitals. There's not much else really, if you ask me.
I am short compared to my immediate family, and most of my extended family. My mom is 5'10, my dad is 6'1, and my brother is 6'5. I am 5'6. My dad's parents were 5'0 and 5'3 so I am not that out of the ordinary. It's strange how sometimes it skips generations (the rest of my dad's family is relatively tall). My whole mom's family is tall too, so I guess my paternal grandparents and me are the odd ones out.
As for HGH, it's kind of weird. It definitely will not increase your height if you're done growing (when the growth plates fuse, as someone said previously), and there is claims that it can increase "vitality" (energy, tissue repair, energy, etc.) but this hasn't been tested enough. It's also prohibitively expensive (also previously stated), and negative side effects aren't fully known yet. It's just not worth it at this point.