I hate to say this but you should probably listen to your doctor, and ignore anyone else, because we are all anatomically slightly different and ultimately that will make a difference. Your doctor is the one who knows what should be possible for you.
My surgery is 31 years old. I don't really dilate these days... perhaps once a month or so, but I must have quite a lot of space in the pelvis and so I can get the orange soul source in almost to the point where I lose the end. Now my surgery is a deal more primitive than yours, so if your doctor reckons you should be achieving the last dot then, while I agree you shouldn't over-strain, I do think you should try.
Practically speaking the last dot on the biggest dilator is about 6 inches which is enough to make sure you should never have size issues with the average man.