Before I even accepted myself and began the coming out process, I was always very masculine. My parents would become upset when people took me for a male, but I liked it. The one thing I didn't like was when people tried to use it as an insult because I figure, "You wouldn't try to insult any average Joe by saying he looks masculine, so either I do or I don't, and if I do then don't play like it's a bad thing". Basically, I always wanted to be seen as a normal dude and a lot of people saw through it.
Unfortunately some of my behaviors came across as just wrong, for example I did get into trouble for things any other guy would have been less, well, hassled about. Plus when I came out to friends they mostly all just said, "yeah, I'm not surprised", or "I expected that". Which actually was a major relief and it pleased me that they'd seen the male me all along.
As long as no one is giving you trouble, I guess it doesn't matter. There are a lot of people who are quirky in some way. Some of us due to having differently gendered mannerisms, and others just for being 'too chatty', 'too quiet', 'artsy', or an infinite number of other things. So long as you're not in any sort of trouble, I would try to relax about it a little.