Your question, I think many if not most of us have had to come to terms with. Not really sure what constitutes half-passable is but you are you identify to be, no matter what others opinions are of you. I know that is hard sometimes ( and hard to get used to also ) but it is both the wonderfully exciting and challenging truth of it.
I don't really own any guy clothes but I do own lots of clothes that are fairly non-girlie for lack of a better term. When I put my ever-lengthening hair up in a bun and don't wear makeup and don't watch my voice, yeah maybe not showing my most feminine self but I still am who I am.
I have not used a male-labeled bathroom for at least three years if not more. Do I worry about offending people? Nope, not really. What a silly thing to be offended about, I think. If our culture insists on non-legally segregating places to do our basic functions, I guess I'll play along

I've never had anyone say anything bad to me, the most is a couple quizzical or stunned looks. If anyone would confront me they better be prepared for a real confrontation.
It's sort of strange that of all the things that historically divided genders (we haven't been able to vote for even 100 years in most places) That bathrooms become the thing of the day. Lockerrooms are a bit of a touchier subject perhaps because of the open nudity that can exist if only briefly. Still, I haven't used a male labeled one of those for at least 2 yrs or more.
So yeah use the bathroom of your identity and don't look back is my advice.