I think coming out at school will probably be easier than you think, I think this upcoming generation is more accepting of LGBT issues than previous generations. Of course, there will be some teasing, but they'll get over it quickly. I acutally did come out as male at school back in 9th grade, and no one seemed to care and I wasn't teased, albeit I didn't use the men's room or their locker rooms, and me be "Vincent" was generally pretty short-lived. I think most kids will be accepting once they get over the shock of it and any gossip, it might suck for awhile but I feel confident that you'll be pleasantly surprised and just relieved once you get it off your chest; I am just mostly concerned about your mom finding out...
But, Problem 1: having a big school isn't necessarily a bad thing, if it's big enough where not everyone knows you, it'll be easier to blend in. Also with a bigger school, there's tons of other drama going on to distract from the "oh X is saying she's a man now" gossip going on.
Problem 2: IF there's a lot of LGB people, they will probably be more likely to stick up for you, even if you're the only out T. There might be other trans kids there, they just might not realize that they're trans yet, or are too afraid to come out as well. IF there's no GSA anymore, but still lots of LGB kids, maybe you could try to find a way to start the group back up? Just having a group will be able to lend you support.
Problem 3: I wouldn't worry about facebook, things on there come and go, and most likely in 10 years you'll probably won't be friends IRL or on-line with most of the people you went to high school with. Though I am not social media savy, I know all kinds of crap goes on there, and it seems completely awful and overwhelming at the time, but it'll be forgotten about quickly.
Problem 4: Your teachers must be super weird, I've never had a teacher call be "ma'am" before... But, I think most teachers won't just ignore you if you tell them to stop calling you "ma'am" or to ask to be called by male pronouns. Many teachers are really worried about being reported for discriminatory behavior, etc. that they'll probably just want to avoid trouble; also, I think many school districts nowadays also have teachers go through some sort of sensitivity training. Though, if you have a school psych or guidance counselor you trust, it might be best to talk to them first since they might know how to best approach the teachers or if there's any sort of policy that might be relevant. I wouldn't be surprised though if somehow they get your mom involved, but maybe if you can explain to the counselor your situation with you mom they might be willing to work with you.
Problem 5: I know we've talked about your mom before, but... I really think you need to find some sort of back-up living situation just in case you ever need to get out of there quickly. She sounds like she has her own problems she needs to deal with, and she def sounds imblanced. You should see if there's any friends who'd be willing to let you crash at there place for awhile, or if there's any sort of youth shelter nearby.