My first four years had minimal effects because my hormone levels remained too low during that period. Many sources, as you stated, claim that after three years, what you have is what you get.
However, what is not clearly stated is that it requires three years of sustained hormone levels within the target gender range.
If they are too low, as was the case in my case, the body processes them as waste and eliminates them. Since I was also on testosterone blockers, all of my hormones were too low.
Spiro and finasteride affect the 5α-reductase enzyme, which affects all hormones, so that threw another wrench into the works.
Once I got my levels into the correct range, about two years ago, things then started changing for me.
So don't start counting from the day you took your first pill. The clock doesn't really start until your hormone levels are within the proper range.