If you're still on HRT it shouldn't be testosterone. Your presentation has powerful psychological effects for both you and everyone around you, it's true that the clothes don't make the man, but they do inform people how to act. You should feminize your male mode somehow just to keep you grounded if you can't simply present female. How to do that is up to you, women's jeans, a bra, whatever you think you can get away with, just something to keep one foot in, so to speak. You also might find it helpful to talk to your therapist about it.
I was thinking about this later, but jeans in business casual it's probably a bad plan, go with women's slacks then >.>