I think people are misunderstanding what he is saying. He was told that the person he was going to see was an "endo", only to find out that he was not and wanted Brody to leave his current doctor so that he could be Brody's family doctor and he could get Brody a referral to a "endo". But since Brody's current family doctor knows everything about him, and would gladly do a referral to an "endo" why would he change doctors??