In importing audio the AI module is completely free to punctuate and insert paragraphs. Of course a snippet of text is not a sentence and a sentence is not a paragraph, so the AI instructions are to make every snippet into a sentence and every “sentence” into a paragraph. This results in a lot of unnecessary pages and the inability of the translator to take the whole context of the sentence into account. As the AI instructions are clearly contrary to the definition of both a sentence and a paragraph, I have the following suggestions:
Do not insert any additional paragraph (barring the first paragraph after the title).
Create real sentences: words that seem to make up a sentence ending in a point. That eases understanding of a sentence.
I have been solving the paragraph issues with Rooster’s program, but with many lines it is cumbersome and unnecessarily time consuming.
Due to the unreliability of the YouTube import I’ve been using Faster Whisper XXL for quiet a while now. (I hav ebeen mentioning this here and there, so I may sound like an advertiser ). It incorporates a sentence argument that causes it to split each line in the created subtitle file to correspond to one sentence, which has been working pretty good for me. Each sentence still starts at a new line, but at least it is easy to see where one sentence ends and another one starts.
It seems like you are talking about videos with subtitles. I am not. This is about importing audio such that the Lingq AI interprets the text and creates written text. The issue is the way it separates text into units of text called sentences and paragraphs.
I do wonder whether Faster Whisper XXL is capable of interpreting audio and making it into reasonable sentences. However, it probably costs an amount of money on top of the Lingq subscription. Although I am curious to know how it works and how well it works, it is not my intention to use it on top of Lingq. If I would not use Lingq, then I might consider it. Is there a path toward this tool? If so, what are the costs?
Me neither. I was talking about an external tool I use for creating the transcripts of the audio files, which is based on whisper ai. It can relyable detect sentence endings in Korean, but I haven’t tested it for other languages yet. It doesn’t create paragraphs, though, although something like ChatGPT might be of use here. You can download it for free, just google it.
Thank you. I will look up the tool and see if it is useful for Finnish. BTW I am slowly starting to watch Korean learner podcasts as well and if the tool is useful for Korean, I am very happy about that as well. Hopefully it does some good for Finnish as well. Thank you for the tip!
EDIT: I forgot to mention. I actually don’t care about paragraphs because most AI cannot make paragraphs like humans do. Each paragraph is and should be a coherent logical part of a section or chapter. The coherency makes the paragraph better and I feel at the moment that only humans can do this task properly. That might change in future, but for now, I prefer to leave paragraphs out when transcribing and insert paragraphs when I am reading or analyzing the text. Again, thank you for the tip. I found it in github.
Okee, suffering from old age I guess. I downloaded and installed the faster whisper xxl successfully, it works. However, I completely forgot to note the github library I got it from (it was far after midnight), even though I did mark it to be watched for releases, I am having a hard time find it.
Could you please remind me which github repository and library this was?
I am a github member, so once I have the repo, I can re-find it. Sorry for the trouble.
@zoran could you please respond whether this is feasible and for when it would fit the planning? Even though I might find a workaround in a separate module, it is an additional step, above separating the audio, that I can do without. So, a response please?
Your faster whisper xxl tool works like a charm. By this I mean, that the end result is really a set of sentences: very, very good!
The weird thing is that the intermediate srt file contains partial sentences, even though they are way better than the Lingq AI result, they seem to be limited in length (may be due to screen size). But, once imported into Lingq, they become full sentences.
It looks like the Lingq importer merges the subsentences into sentences, which is ok with me. It delivers an endresult I can work with.
There are different settings for how the length of the individual lines are determined, word count beeing the standard, iirc. There is a --sentence setting applicable, causing each line in the srt file to be a full sentence.
I’ve actually just downloaded the executable
Works well enough for me.
That alone makes me wonder how well humans can do. As said, I never actually bothered to check whether ai can handle it. I guess it may also depend on the kind of text.
). It incorporates a sentence argument that causes it to split each line in the created subtitle file to correspond to one sentence, which has been working pretty good for me. Each sentence still starts at a new line, but at least it is easy to see where one sentence ends and another one starts.
As said, I never actually bothered to check whether ai can handle it. I guess it may also depend on the kind of text.