Is it really not helpful to start speaking from the beginning?

I can’t see any point in speaking German to someone since I cannot assemble sentences other than basic ones, and I struggle to understand. I do practise speaking a lot on my own, to train the mouth muscles, and learn the music of the language. Obviously my accent is very imperfect (apparently it’s good for my level), these things take time, and I still struggle to hear and reproduce a couple of the vowels. They will hopefully come over time with more listening.

A friend who is bilingual in English and German would say things in French to me, but he never understood the replies so it was pointless. And yes native French speakers do understand me. I don’t speak German to native speakers I know, I need to get to lower B2 for that.

Children when young have the advantage of simple conversations.

It depends for 100% on you.
If you would like to speak as soon as possible, you can try to speak a bit already on the first day of language learning. If you would like to remember more words to understand more, you can save up to 2000 words before speaking.
Choose always what is more comfortable for you!