Well, better safe than sorry when newly post-op is true, yes. It sounds like things are going OK, though, and lack of condom shouldn't change anything in that respect. Brassard recommended waiting to 6 weeks, but some people definitely do heal faster than others.
(There are some studies suggesting that over time the skin changes to resemble that of the cis vagina, as well.)