Add in the fact that a genetic male's collarbone slopes out to the extreme outside end of the shoulder socket while a genetic female's slopes to the inside edge of the shoulder socket which gives a narrower look to the chest as well. Other problems will also affect your upper torso. Asthmatics will tend to develope a hollow near the end of the breast bone, and other problems may cause the breast bone to stick out.
Too many variables, but at puberty, the bones also start becoming harder and less resilient, and that is responsible for setting in some changes that were more elastic before. This happens to genetic girls earlier than genetic boys, but its pretty much over by 25 for both genders.
Puberty is not the easiest thing in the world, but its the only way to keep getting older.