It's clear that different people mean different things by "passing", and place different importance on it (whatever it is to each of them) and see it as important in different ways and for different reasons. So I think it's reasonable to expect people to see it differently from you. I'm against others telling you how important (or not important) it should be to you, but that works both ways - none of us should be legislating on each others' experience or path of transition, though it's okay to ask questions, so long as they don't have condemnation built into them.
Personally, I consider "passing" to be the thing I spent nearly 50 years doing, back when I was trying to convince everyone I was a man - and I'm done with that, I'll just be myself now. But I'm aware that I am privileged to live in a country where being trans has legal protection, and in a part of that country where people for the most part will simply accept me unquestioningly (or so it seems, so far). I look like the kind of woman I've always wanted to look like (although I still suffer from Persistent Penis Syndrome, until later this year - and would of course prefer to be in my 20's and svelte and all the rest of it, but we make do).
So much of it depends on what kind of woman you are (since we're talking about MTF transition here), and what's involved in looking the way you want to look, want to be seen. So many variables.