While I certainly don't concern myself with passing, and I would never have any interest in living stealth, there are a lot of very valid reasons I understand that other people put a higher priority on those things.
1. Some people are happy standing out with all their special uniqueness on display, and some people are happier if that's more private. Nothing wrong with either way.
2. Even in this day and age, there's the question of safety, dependent a bit on where you live and also other factors (like race); it can be downright dangerous to be a non-passing or non-stealth trans person.
3. There's a lot of pressure from the outside to pass, the assumption that you must want to pass, and so we're bombarded with (sometimes well-intentioned) advice on passing from the cis-people in our lives - doctors, friends, etc. Other people in this thread have smartly compared the desire to pass with the desire to be "beautiful," and the pressure definitely works in the same way. The pressure to pass, or to care about passing, or to try to pass, is a lot like the pressure to be thin. You may not actually want to be. It may not even be healthy for you to be. But the culture we live in tells us that that's the ideal and that we must pursue it, and different people react in different ways to that.
4. I say I don't care about passing and would never want to live stealth, but that may have a significant amount to do with the fact that the people around me - family, friends, coworkers - all treat me like the woman I am without me having to worry about that. If that weren't the case, I might feel differently. Doctors used to advise trans women to move to a new location, start a new job and an entirely new life post transition, in part due to the belief that if people knew you were trans, they would never see you as your true self. Some trans people may feel (maybe even accurately depending on where they're at) that this is still the case.
5. Different trans people feel different ways about their own transness. For me, being trans is a part of my identity and always will be, I value it, so to hide that would feel like going back into the closet - or a whole new closet, maybe. For others, it isn't, and so it feels like being openly or obviously trans could detract from how much people are able to see their true gender.
Basically, there's a lot of circumstantial factors that impact how we're going to feel about passing and stealth. For some people in some circumstances, there's good reasons to worry about passing and stealth may be the best option, for others it isn't.