It’s funny, I have the opposite problem to the OP, my listening ability is miles better than my speaking ability, even at times than my reading ability, but then that could be because I’m not a big reader in my native language. I’ve listened a lot more than I’ve read, and definitely more than I’ve written or spoken in the language, so yeah, just do more of what you’re weakest at I guess.
That said, I do have huge problems understanding natives speaking at a natural speed, I still can’t really do that in most situations as they swallow words and merge them into each other (as do all natives of any language from what I can tell), but again, most of my listening has been to slowed down speech, so I’m not well practiced at listening to natives talking naturally.
Something that Steve said in one of his videos was spot on I think “don’t stay with the beginners material” which can be applied to any level I guess, if you’re not constantly practicing listening to natural speed material then you can’t be expected to understand it.
One thing I can say is it takes a lot of time, and from my small experience so far it kind of creeps up on you, one day you can’t understand something, then a little while later you go back to it and suddenly you’re understanding it, and you can’t really explain what happened. There’s some kind of lag in language learning, what you do in the language today, you might not see the fruits of that labour for a few months or so. It’s one of the strangest things I’ve noticed since I started trying to learn a language, and it appears that it happens to most if not all learners.
In a way it helps me with motivation as I now realise that although I may not be seeing progress on a daily basis, I know it’s in the post, so to speak. I wonder, if all first time learners had that knowledge, maybe the dropout rate wouldn’t be so high, but I think it’s something you have to experience to believe.