This is one of those things that's so relative and then exasperated by society's views on the topic. For instance, the proliferation of porn with larger-than-average guys has led many normal guys to think they're totally inadequate.
Then you have gay guys who perpetuate that large is best and some women who just happen to like large better than small.
But all of these all boil down to personal preference - both on the side of the penis-owner and who gets the affection of said penis
Like Sharky mentioned, if you've fooled around already and she knows what you have and she's still there, you probably don't have a lot to worry about.
And there's even individuals who were born male who aren't big enough for full on penetration so they use extra "tools" to help out.
#1 thing is to get cozy with what you have. Except for very expensive, risky surgeries your dick will only get so big, end of story. My life didn't end when I had the big revelation that I had a smaller than average dick.
The #2 thing is whoever you get with sexually just needs to let you know what they like. If you don't have enough naturally to have penetrative sex, maybe something else can be worked out (or you use the "tools"). There's tons of ways to please in that department.
