My brother is an over-the-road truck driver and would tell me when he was passing through my area so we could meet for lunch. Then COVID happened, and we agreed to wait.
Meanwhile, I was still transitioning. When I got the next call, I decided it was time to come out to him before we met for lunch. I told him my appearance had changed since the last time he saw me. I explained that I had been in therapy and was diagnosed as transgender. He and I were always close, so I thought if anyone in my family would be accepting, it would be him.
He went into a rage and said many nasty things. My last words to him were, "I love you, Brother." He hung up, and we have not spoken since.
In your situation, explain that you are going through some things and you are unsure if a visit would be appropriate. If your brother insists, maybe give him a warning as I did. If he sounds positive about it, proceed. If not, there is your reason to cancel (or let him back out).
You know your situation better than I do, so take this with a grain of salt. I hope it is helpful. Good luck.