I could never get a binder to work for me, and since I was very small in the chest and lifted weights a lot, I could get away with wearing a really loose shirt and no bra.
Before I transitioned, from fifth grade to when I started my transition at twenty six, I slouched badly. Even though I had small breasts, I was so ashamed by them that I'd slump and put my shoulders forward. My father used to get behind me and pull my shoulders back and tell me to stand straight. Over the years, the slouching did chance the musculature and spinal angle, and now walking with my shoulders straight really hurts my lower back. There are other reasons as well, but that one predominates.
In high school all the females had to have their spines checked for scoliosis. Mine was just borderline so they didn't put a brace on me, but I kind of wish they had now. If holding your shoulders back hurts your lower back, go to an orthopedist who can tell you if you have a problem with spinal angles or its just due to binding. It may be inconvenient now, but if its something correctable, like changing the way you bind or fixing an orthopedic problem and you don't do it now, you will have back problems that just get worse.
sam1234