Just don't listen what other people tell you! I mean, you are what you feel like, aren't you? There are boys that dress up very feminine and the other way round as well. There is nobody asking whether they are normal boys or not. You're an individual and you have the right to dress as you want to dress. That does not change who you are. As you already said, clothing does not make the person!
Tell people who are wondering who or 'what' you are... I can understand that people tend to get unsure when they cannot classify you. It is always the same...questions can mostly not being avoided I guess...just answer them, again and again. Stand up for yourself...