I can relate a bit. I had a sweater that I absolutely loved, and knew I'd never be able to wear again until I got surgery (which is a lifetime away, at least from where I'm standing). I held onto it for about a year and a half, couldn't throw it out, couldn't even donate it. So I cut it up, and made it into a had. Confession? I cried the entire time I did it, but I'm so glad I did.
If you can find some way to keep it, reuse it, hell even just cut a little swatch out of them and make a scrapbook or something, that might help. I hate throwing away anything that's ever been part of my life, I've still got notes my friends passed to me from 5th grade.