Is the YouTube import not working?

  1. I go to the YouTube video.
  2. I click on the web plug-in LingQ icon.
  3. I click the “Import” button.
  4. The plug-in says, “Importing…”
  5. The plug-in then says, “Done!”
  6. I click the “Open lesson” button.
  7. LingG shows some gears and “Generating lesson,” “AI features can take some time. Try again later.” I’m a little confused now how it’s both “done” and “generating lesson” at the same time. (But then again, I think a meal’s prep and cooking is “done” when it’s been put on the table, not when it’s still in the oven, starting to brown and the plates and silverware still need set.)
  8. LingQ offers a “Go back to the Library” button. (But I wasn’t prior in the Library, I was in YouTube.)
  9. I click the “Go back to the Library” button and the video that was “done” importing doesn’t appear at all, not even with a “generating lesson…” overlay to indicate it’s still working on a long-running task.

Sure, AI features can take some time. I do “try again later,” a second time.

And a third. As I do, I ponder of a number of good UX practices that give user feedback on the progress of long-running processes.

I then I write this up chronicling the user experience on the fourth attempt.

(After having done much of this cycle fairly often recently. I’ve yet to figure out the correlating circumstances when it works vs when it doesn’t. At this point, I don’t think it has to do with what I do as a user and see a feature usage workaround.)

For me personally, this has been one of the most important LingQ features and it just doesn’t seem to work. The possible lack of error handling and misleading or confusing messages presented seem to make it even more unnecessarily frustrating.

2 Likes

Thanks, we will look into it.

2 Likes

Hi @gmeyer !

Thank you for pointing out this. The LingQ team intends to improve the indication of background processes.

Concerning YT imports - when the video has no original subtitles (auto-generated or absent ones) it needs to be additionally processed by AI to transcribe the audio to text. In this case, until the lesson is processed, it can be found in the Pending tab on the Continue Studying shelf. The notification also should appear when the lesson is ready.

1 Like

Before we get to how to string it together in a better user experience…

What pending tab are you referring to? I click on Lessons, I see Continue Studying, but there aren’t any tabs under that. It’s a carousel of tiles, one for each course.

1 Like

The tab shows only when the lessons are locked because of processing.
Here is how it looks:

1 Like

That’s rather hidden. I’d never seen that. In fact, I don’t even think appearing and disappearing menu choices based upon temporary processing state of data has ever been a thing in decades of UI design.

What are your thoughts on how to make it better?

For instance, rather than informing not-done as “Done!” could instead…?

Or, instead of “Go back to Library,” why not go to the pending list so the user sees something is working / happening?

2 Likes

I’m sorry, my previous responses were a bit rash, let me summarize again.

Audio-Subtitle Synchronization Issue The main problem is the lack of synchronization between audio and subtitles, as most YouTube videos lack subtitles. Although YouTube has automatic subtitle recognition that is accurate to the word’s position (some plugins allow clicking on a word to correspond to the timeline), it does not form complete sentences. Without complete sentences, it’s challenging to learn on LingQ.

LingQ’s Reprocessing LingQ might reprocess users’ videos for better subtitle synchronization. This reprocessing could theoretically solve most issues, given the maturity of many speech recognition models.

Drawbacks However, there are drawbacks:

Import Time: Importing takes time, approximately 5-20 minutes.
Synchronization Issues: Even after recognition, there may still be synchronization issues.
The first issue is understandable, but the second could be optimized.

Narrow Time Range The time range for text-audio synchronization is too narrow. I’ve used Whisper, but the text recognized may not perfectly match the audio timeline, sometimes off by a few hundred milliseconds. Perhaps we could try expanding the range for all sentences by 500ms with a script? LingQ seems unprocessed in this regard.

Model Inefficiency After importing many videos, it’s evident that LingQ’s recognition isn’t very effective, with many videos and sentences being unusable. It’s unclear if LingQ has considered switching to a larger model to address this. The current model seems to be a middle-sized one, and changing it involves server computation costs and other challenges. Perhaps leveraging the user’s computer could be an option?

Other Solutions? YouTube’s recognition is already quite good. If it could automatically integrate scattered sentences, they might be directly usable. Could AI be used to integrate sentences? Or perhaps grammar models could identify verb-noun combinations to construct complete sentences. This method seems feasible and highly versatile.

1 Like

I’ve now seen this process from import >> “Done” >> “Pending” >> imported, but I think I’m now seeing some failures. @LingQ_Support

I have one lesson that has been pending for > 10 hrs. It still shows in pending. Should I believe it is still processing? It is 90 minutes long. Surely if the length was an issue, it would warn me or throw an error, right? (Edit: nvmd on this one, it now says it failed, and I was able to successfully delete this one.)

I also have a previous import that failed and stayed in the pending tab. I tried to delete, which seems to have been successful, but the icon is still there. When I click on it it says
lesson: Invalid pk “29953234” - object does not exist. Is it stuck there forever?
Edit 2: Now that is cleaned up, too. It had been there for about a week.

Anyway, thanks!

1 Like

I am now importing a lesson transcribing from audio and the previous 2 lessons are visible again under pending lessons, despite having been deleted. So, it appears, that the ghosted deleted lessons are still there somehow, though there is no real impact on usability.

Hi @hiptothehop!
Thank you for pointing out these issues.
Currently, there is a known bug with the deleted lessons showing up. It should be fixed soon.

3 Likes

Youtube has word-for-word Json subtitles that are well worth being utilized


All we need to do is to re-specify the subtitles and realign the fragmented sentences into one completed sentence. I think this is very easy to do algorithmically, and probably even easier with AI.

Hopefully lingq will consider trying this approach, which is more effective than re-speech recognition.

After my manual testing, this really seems to work.

1 Like

Fortunately, it seems that we can take care of the Youtube subtitles ourselves and link them to the corresponding video. I will now try to get to work on fixing the Youtube subtitles and hopefully find a good way to do it.

1 Like

I’m going to give up on using Lingq’s import now that I’ve found a very graceful way to do it.
2024-4-17-16-37-35

This plugin is: https://chromewebstore.google.com/detail/netflix-bilingual-subtitl/ociifpecaephgjmjlkepjmiemcjidbgn?pli=1
But since this feature has been removed I’m using an older version.
see post below , an automatic solution to fix sentense

—delete—
I did just ask the developer about it, but haven’t gotten a response yet. Now if you want to use this feature as well, try to chat me privately if I see it.

By the way, I’d like to try to start a topic for German Youtube auto-generated subtitle corrections and share the corrected subtitles for your convenience, just maybe.

1 Like

Thanks for sharing that. I’m going to add this word level timestamp approach to the Youtube tools.

Have got this working. Word level timestamps on the Finnish text of English spoken video

2 Likes

I got in touch with the developer, and the reason the developer eschewed my manually moving subtitles feature above was because he developed the ability to automatically combine sentences, which is what I proposed in the first place.

To summarize, if you don’t want to put up with the errors caused by Lingq’s recognition, try using Youtube’s auto-generated subtitles directly, and this plugin fixes Youtube’s auto-generated subtitles.



Fixing broken sentences
Anyone who has used Youtube’s auto-generated subtitles knows that they have always had a problem with broken sentences. These subtitles have no punctuation marks, and different sentences are linked together and break at certain points in time (as in the main subtitle below). Since Youtube’s subtitle translation is based on these broken sentences line by line, this results in a poor translation quality.
We use machine learning to repair these subtitles and reassemble the broken sentences, which greatly improves the quality of the translation.
Translated with DeepL Translate: The world's most accurate translator (free version)


1 Like

I have a question: if I’m right, the auto-generated one in the video should be in English, and why are you highlighting Finnish?

If you want to highlight the translated text, how do you ensure the accuracy of the word-for-word translation highlighting?


Also, looking at my post in the previous entry, you didn’t format the youtube auto-generated subtitles, which would cause the translation to be way off. And you’re working off of that. This plugin above solves that problem.

1 Like

No formatting done, just a google translation of the auto generated subtitle. The Finnish and English match well in the above video too. Maybe im used to it, but the short subtitle bursts (3-4 words of broken autgen, no punctuation) seem to be easier to read when watching a fast video. If i was reading it as a transcript in LingQ… maybe I would want the punctuation.

I like to watch videos in English while reading the corresponding Finnish subtitle. Doing a lazy method (watching a video) paired with another lazy method of knowing exactly what is being said as you read it.

Its in that Youtube json data

1 Like