I´d try to get a clear answer out of him. He says he wants to be alone.
He says you are the only one who can handle him.
Sounds like he needs space for a while but is afraid to let you go. Being with somebody who accepts you for you is quite unique, and specially trans people see how valuable this is. It kind of makes sense for him to be scared of losing it.
Whether you should move on or not is a decision you need to make for yourself. If he wants to be alone and not have a relationship, it´s up to you what you want to do. You could wait it out, but you can´t predict how long it´s going to take.
You can´t predict how he transforms, either. In my experience HRT is very transforming and over time, your needs change. So neither of you might want the same thing in a while from now.