Here's a thought... if you temporarily register with an English GP to get a referral to the English GIC of your choice, you can then continue your care in Wales once that referral comes through because it will still be funded by England as that is where the original funding request was made.
I know of several people who have lived in England temporarily (for Uni, etc.) and have started their GIC pathways there, only to move back to Wales later. Their Welsh GPs have been happy to continue their care, and they're able to continue attending the English GIC of their choice, fully funded by NHS England. It's that initial referral that's the key factor in getting the GIC of your choice.
Whilst you're waiting for your GIC appointment to come through, you can still attend a Welsh GP as a temporary patient if you need medical treatment for any reason.
The Beaumont Society were mistaken, incidentally (and they are based in London, so their advice tends to be London-centric). The Patient Charter is individual to each NHS body, and the right they mentioned is guaranteed by NHS England. To clarify: you have the right to be referred to any service that is commissioned by your particular branch of the NHS, which is NHS Wales in your case. NHS England's Patient Charter may very well guarantee English citizens the right to attend any clinic it commissions, of which there are several throughout England. But of course, NHS Wales only commissions one clinic - Charing Cross - for gender services for all Welsh citizens... so under NHS Wales' Patient Charter, you only have the right to be referred to that one.
So much for a 'National' Health Service, eh?