I'm afraid your doctor may be onto something, and you might need another blood test to figure out which doctor is telling you the truth (what's a 'kaiser doctor'? is that your GP?).
There is a condition that can happen when you're on T: your haemotocrit can go too high. Haematocrit is the percentage of red blood cells in your blood. If it goes too high, you'll have a condition called polycythemia which needs medical treatment. It's actually very common in guys who go on T and this is one of the reasons why we need to have regular blood tests.
Symptoms of polycythemia include dizziness, headaches, blurred vision, red skin, tiredness, abdominal discomfort and itchy skin after a bath or shower. See that list? It includes both of the symptoms you mention: fatigue and dizziness! Have a look at the patient information leaflet that comes with your T: you'll see that this is a known risk factor for taking it.
So yes, go back to your doctor and get another blood test done. Ask them to check your haematocrit specifically (not just haemoglobin in general), and ask them the treat you accordingly. Oh, and taking less testosterone is the last solution for this problem: it's more commonly treated with medication or by blood letting.
