Welcome to Susan's Place. I am not a lawyer so the best advice would come from one. My personal opinion is it would be best if you forget about this and move on with your life. If you still worked with the company and the company had a LGBT policy, the company could do something about the manager. As you no longer work with them, they have little incentive to take any action. If you want to take them to court, you will have large legal expenses and it's very unlikely you will win the case.
Being transgender may have resulted in me losing two job. Like you, I didn't have much in the way of legal options so I had to move on and rebuild my life.