Yeah, I think the forehead is very important, because we look each other in the eyes and though people aren't consciously aware of the general differences of a testosterone-poisoned bone structure, I do think they pick up on it at a subconscious level, at the very least.
If you have any kind of a brow ridge, this can be a problem. Bangs are pretty reliable at hiding it, as are certain kinds of glasses. Facial surgery is what it takes to get rid of it, usually in conjunction with rhinoplasty.
Now, context is also important. If everything else about you is obviously female, you likely won't get clocked. Once people get to know you, if you haven't already been clocked then your gendering is likely to stick. But then there will be people who just can't stop looking, and once they pick up on it they start looking for other cues to confirm their suspicions or resolve their sense of ambiguity. Interestingly, a very good voice can nip this in the bud.
But yeah, it's pretty important if you want consistent, unambiguous female gendering.