Dilating is about training the muscles, for the most part; if you do stop using the larger one, you can always regain width later on if you decide you need to. Depth can be lost permanently, but width.
With that said, I've been using only the orange one for years (I'm several years post-op), and I am still quite tight. I don't think it's a significant concern. And, well, TMI, but I was bigger around than that in my pre-op days... so it does happen.