Before I didn't have to take it into account *at all,* probably 5 months. (I had a lot of swelling.) I could wear about 80% of what was in my closet either for short periods or in controlled situations by around the 3-month mark, though, and I had a bunch of looser pants ("real" dressy pants, not sweatpants) that were just fine with no restrictions.
I'd say, if you'll be there for only up to 6-7 weeks post-op, I'd definitely pack for swelling and soreness. Worst-case is you don't need it but have comfy clothes, versus the reverse...
Good luck!!!