Hello there; I used to swim until a year or so ago, and I picked up a few things that might be of help.
If you have processed hair, it's best to avoid getting chlorinated water in it at all costs. When my hair was longer, I would tie it up and wear a swim cap. It's not pretty, but it'll save your color and the texture of your hair. If you absolutely cannot wear a swim cap, make sure all of the excess color has washed out, and deep condition the night before you swim. It should offer your hair some temporary protection from the harsh chemicals. Best of luck to you!