(Disclaimer that this is mostly what I've picked up over time and from personal experience to some extent)
It sounds to me like your workout program isn't working out for you. That's probably going to be your biggest issue in your case since you said you aren't too bad in the diet department. That can make or break things. If you haven't been getting noob gains (I'm assuming) and you aren't starving yourself or eating pure, proverbial crap, that's where I'd look first.
It isn't just about not pushing yourself enough, though that is a part of it. You weren't specific as to what you meant by that, but in case it's helpful to you or someone else... The point is to go to failure - what that means is your last rep is the last one you can do with good form. So if you have to do 10 reps of dumbell chest press, you pick a weight that you think you can do just 10 reps with good form and then adjust the weight for your next sets if needed (like if you couldn't finish or it was too easy). Poor form is where you're starting to "compensate" to get the rep - so other muscles are starting to take over, you're going lop-sided, you're using momentum, that kinda thing. The exercise loses it's meaning pretty quickly after that. Typically something like 2 seconds up, brief pause, 3 seconds down though you might have some exceptions. But that can really help to take some of the internal effort away because the "rules" are already set.
For the diet part - having a nutrient-dense diet is more important than anything else. Focus on that first. You'll probably see gains from that alone, actually, even if you aren't upping your cals. After that, you can start looking at increasing your calories. If you increase your cals without enough nutrients in your diet, your body isn't going to have much mortar for to build muscle.
What you end up eating doesn't have to be complicated. It's better if it's simple. Start with items that are as close to their natural state as is reasonable, then go nuts with however you like them done up. There are of course some processed foods that are good for you like PB (especially if you can go with the 100% peanuts stuff) and EVOO and whole grain bread. But you get what I mean. To start out, I wouldn't worry too much about specific stuff like macros and micros or whether agave or honey is better for you, unless you're into that kinda thing anyways. Don't skimp on the veggies ofc but you know that. The rest can come over time when you're ready/able.
Sorry that was longer and more lecture-y than was probably necessary...
TL;DR - Basically, start with getting your fundamentals down, then you can start going in a more specific route that suits you like gaining mass etc.
