I won't say that it will never return, but it might not return for a very long time. I was 19 when I first discovered the possibility of transitioning, and then gave up after researching it and realizing that it was financially far out of reach. But when I finished school and started working eight years later, the urge came back.
That time I gave up after finding out that gender identity disorder was considered a mental illness (this was in 1989). I got married, had kids, and thought I had put away the gender thing for good.
Then ten years later I got divorced. In 1998/99, I started dressing at home, set up an online persona, started researching therapists and doctors, and then -- I was in court on business, and I saw a young, almost passable transgender who had a case under her male name get mistreated by a judge.
I was in a custody fight over my kid at the time, so I went home, cross dressed one last time, looked in the mirror, said "this just isn't me," and it worked. For a time.
The anxiety and panic attacks started about six years later, in 2005. I managed to stay in denial until late in 2014. Now here I am.
YMMV, but if the dysphoria returns, come see us again, or contact your local LGBT center.