If you haven't come out to him or your sister yet and are just basing how he will react off his general attitude towards LGBT, I wouldn't write him off completely - plus your sister might intervene in some way, no? Unexpected people seem to come around and accept things. The converse of this is, of course, that people you think will be accepting end up not. It's really unknown how people will react until it happens.
Think about this in context. Write out the reasons you want to start T and then compare them to the potential your brother-in-law will try to restrict your access to nieces and nephews. Remember that potential is nowhere near 100% - even if he acts that way at first, people do come around over time (even a lot of time); he will have to make a big fuss about family events and probably have to deal with backlash from any supporting family members; he would have to acknowledge his rejection over the years as time passed, etc.
A good way to approach this may be to think about talking to your sister in confidence first, or your parents, to get a feel for their reactions - that way you will know who will have your back before breaking the news to him.
As for my own experiences, I started to transition aware of the fact there is a good chance none of my extended family members will ever speak to me again. None of them know because it is not worth dealing with. (I'm from a family of texas southern baptists that almost disowned me when I told them I was an atheist. My mother cried and told me I had shamed her forever. I can't imagine what they would say about the trans stuff) I haven't been to a family holiday in years though, so the only real loss is the chance of ever reconnecting with them. I also don't talk to my immediate family.
Additionally - for me, I couldn't see a future for myself at all without transitioning, so the opinions of family members never factored into the equation.
