Return of the Rooster - LingQ Video Tools Version 2

image

More Info Support Purchase Sign up to LingQ
Forums Discord SendOwl Rooster Referral

amazon2
dubhints

V2 STATUS: ALPHA

Browsers: Chrome
Latest Version: 0.50
Patch Notes: Discord

Netflix Test Notes

You must disable the LingQ Import Extension (if applicable) while using Rooster Video Tools on Netflix

Disney+ Test Notes

An included TAMPERMONKEY script is required for access to the Audio Files and subsequent functions on Disney+. (Audio Download, LingQ Transcribe, Whisper JAX)

The Audio Downloader is based off this TAMPERMONKEY disney-audio-downloader. It does not work properly in its base state, if you have it installed you will need to uninstall it.

The script is Included in the ZIP of Video Tools V2 → /scripts/test/disney/AudioDownloaderRooster.js

Instructions to Install Tampermonkey

(Instructions for Disney - same for Amazon Prime)
Install TamperMonkey
image

Copy and paste the script located inside AudioDownloaderRooster.js. (In the ZIP folder)
image
Save the Script
image
Enable the script

Overview

Rooster Video Tools is a browser extension developed for Chrome Manifest 3. The main functions are dynamic LingQ lesson creation and providing a sophisticated video overlay for language learning. It supercedes V1 Rooster Video Website Tools and is included with MasterLingQ Premium as an optional install.

Author Notes

Video Tools version 1 was an experiment into the world of website manipulation and user experience. We have learned a lot and as a community have come up with several ideas, fixes and improvements to the software. Unfortunately the overall sentiment is still that while being extremely beneficial it is confusing and increasingly so for new users with each update.

Goals

  1. Improve the New User experience
  2. Improve the GUI
  3. Modify/remove confusing systems
  4. Expand user choices for translations / transliteration / text-to-speech
  5. Improve the No LingQ mode
  6. Integrate all video websites into a single extension

Development Progress

As of 5 November 2024. Expected completion end November 2024.
Status Key: [TBA, Pre-Alpha, Alpha, Beta, Release]
DRM = Digital Rights Management → Audio is protected and can’t be used

Website Language Completion Availability Status Subtitles Audio / Transcribe
Version 2 Features . 98%+ :ballot_box_with_check: Alpha . .
6Play French 80% :ballot_box_with_check: Alpha X :white_check_mark:
Amazon Prime Multi 90% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Aniworld Japanese/German 70% :ballot_box_with_check: Pre-Alpha N/A :white_check_mark:
Animelon Japanese 90% :ballot_box_with_check: Alpha :white_check_mark: TBA
AppleTV Multi 10% X TBA :white_check_mark: X (DRM)
ARD Mediathek German 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
Britbox British English 90% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Canal+ French/English 90% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Crunchyroll Japanese 90% :ballot_box_with_check: Alpha * X (DRM)
Dailymotion Multi 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
Dood * 80% :ballot_box_with_check: Alpha N/A :white_check_mark:
Disney+ Multi 95%+ :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
DRDK Danish 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
Finnish Me Finnish 10% X TBA :white_check_mark: :white_check_mark:
Foxtel English 50% X TBA :white_check_mark: ?
FranceTV French 95%+ :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
HiAnime Japanese/English 80% :ballot_box_with_check: Alpha :white_check_mark: TBA
Hulu English/Spanish 30% X TBA :white_check_mark: X (DRM)
IQ Mandarin/Vietnamese/Thai/Korean 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
Max English/Spanish 30% X TBA :white_check_mark: X (DRM)
MegaTV Greek 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
MegaCloud * 80% :ballot_box_with_check: Alpha :white_check_mark: TBA
Netflix Multi 98%+ :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
NPO Dutch 90% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
NRK TV Norwegian 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
Paramount+ English/Spanish 90% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Rumble Multi 70% :ballot_box_with_check: Pre-Alpha N/A :white_check_mark:
Rutube Russian 95%+ :ballot_box_with_check: Alpha N/A :white_check_mark:
Stan Australian English 95% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Streamtape * 80% :ballot_box_with_check: Alpha N/A :white_check_mark:
SVT Play Swedish 95% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
TikTok Multi 90% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
TugaFlix Portuguese 80% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
TurkSubs24 Turkish 60% :ballot_box_with_check: Pre-Alpha X (HC) :white_check_mark:
Udemy Multi 50% X TBA :white_check_mark: ?
VidMoly * 70% :ballot_box_with_check: Alpha X/? :white_check_mark:
Viki Asian 95% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Vix Spanish 90% :ballot_box_with_check: Alpha :white_check_mark: X (DRM)
Weverse Korean 95% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
Yle Areena Finnish 95% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
YouTube Multi 98%+ :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
YouTube Shorts Multi 95% :ballot_box_with_check: Alpha :white_check_mark: :white_check_mark:
YouTube TV English/Spanish 70% :ballot_box_with_check: Pre-Alpha :white_check_mark: X (DRM)

Version 2 Features List

LingQ Features

Feature Description
LingQ Profile Sync all preferred settings directly from your LingQ Profile
Course and Lesson Creation Automatic Course / Lesson creation based on adjustable import settings. Import options include: Subtitles, Transcribe, Translate to Target and Own Files. Also Import the YouTube comments as a LingQ lesson. (optional)
Lesson Loading Reload existing LingQ lessons into the overlay. Selectable via dropdown or lesson Id input
Word and Phrase Status Create and update LingQ word statuses. Available on mouseover, keybind or by clicking the word in the captions.
Display and Update Stats Automatically record Words Read and Listening Time. (Optional)
LingQ Transcribe Generate subtitles from the video’s audio using LingQ provided Whisper model.
LingQ Transliterate Parse all available Transliteration types. Available as a caption display element or on word click
LingQ Text to Speech Return all voice types and options. Available in multiple places including Audio Dub and on word click.
LingQ Word Translate Utilize Context Word Translations (GPT) and DeepL as optional Clicked Word translation sources
LingQ Sentence Translate Choose between GPT, DeepL and Google for sentence translations. (LingQ lessons only)

Caption Features

Feature Description
Paired Captions The studied subtitles can be paired with a translation line generated from various sources.
Word Elements The words in the studied subtitle are replaced with LingQable elements. (Similar to LingQ reader)
Translation Elements The word translations are pre-generated and displayed above/below the word as it appears. Up to 2 different languages translations can be generated and displayed at a time.
Transliteration Elements Display your preferred transliteration option directly above/below the word in the captions.
Phrase LingQing Highlight phrases in the captions to display a LingQable translation menu.
Spoken / Non Spoken Speed Automatically adjusts the video playback rate depending on if someone is speaking or not.
Word Level Timestamps A word marker is placed underneath the word as it is spoken in real time. Also available as a scrolling popup. The tick rate can be increased for more precise languages.
Shadow Caption Pause the video for a user preferred interval after each line.
Previous / Next Caption Move to the next or previous subtitle. Also adjusts video time accordingly. Available via button click or keybind.
Replay Caption Replay the current caption. Available via button click or keybind.
Time Differential Add a timing delay in either direction for when subtitles are displayed relevant to the speech.
Pause on Mouseover Automatically pause the video at end of caption when mousing over the captions. Video plays again on mouse exit if it was previously playing.
Blur unless Mouseover The captions box can be blurred out unless being hovered by the mouse.

Overlay UI

Video Control Bar 690x86
Mode Control Font Control Display Control Word Color Control
242x500 250x500 189x301 690x456
Element Control Word Side Bar Keybindings Control Sources Control
181x499 308x500 218x500 210x500
Word Status Clicked Word Theme Editor
226x165 271x500 343x500
Blurred Captions Theme Swap
600x126 400x290

3rd Party (Transcribe)

Feature Description
Whisper NOVA Generate subtitles from the video’s audio using Deepgram Nova2 model.
Whisper LARGE2 “” using Whisper Large V2 model.
Whisper FLARE “” using an AI Cloudflare worker. (Coming Soon)

3rd Party (Text to Speech)

Feature Description
Meta With 1100+ voice options the niche language you are studying is surely supported. (Not available for Dubbing)
Edge 318+ voices with regional options. (Not available for Dubbing)
Google 57+ voices with regional options

3rd Party (Translate)

Feature Description
Google (F) Full data translation model. Returns multiple translations, synonyms and lemma. Available for all translation types.
Google (S) / (S2) Two simple translation models. They return different translations but only the most likely one. Available for all translation types.
Google (HTML) Lightning model using a deprecated mobile translate module. Results may vary. Available for all translation types.
Reverso For Reverso supported languages. Returns multiple translations. Available for all translation types.
Meta Word translations can be generated by Meta LLM. (Coming Soon)

3rd Party (Transliterate/Diacritics/Splitting)

Provider Description
Rooster Homebrew transliteration options for Serbian, Arabic, Belarusian, Gujarati, Georgian, Macedonian, Hindi, Khmer, Persian, Hebrew: Latin and Cantonese: Yá»±tyá»±t
RussianGram Russian stress marks and emphasis Latin transliteration. Stress marks are superficial only, the original word is LingQed.
Farasa - QCRI Processes Arabic text and returns a diacritical version. Diacritics are superficial only, the original word is LingQed.
Slovnyk Returns Ukrainian stress marks from original Ukrainian text. Stress marks are superficial only, the original word is LingQed.
ALittleHebrew Returns a more correct Latin transliteration for Hebrew than the Rooster homebrew.
EzLang Returns Zhuyin (Bo-Po-Mo-Fo) from given Chinese characters.
SpacY Identifies word lemma based on position in the caption. For selected languages only.
Thai Word Cut Creates recognizable words from Thai strings. Activated when the main line language is Thai.

Other Tools

Tool Description
AI Chat A general AI chat bot that is baked into the video player.
Segment Recording Record segments of video/audio while playing the video. (Supported websites only)

UI UPDATES

Navigation Menu

193x345 profile menu sources menu website menu

Overlay Settings Menu
Improvements to UI options and user clarity while maintaining complex systems

Viki2 Muumi
690x456 683x499
690x454 690x457

Video Overlay

690x359 690x359
690x359 690x359
690x359 690x359
690x388 690x412

Overlay Settings
Utilize complex settings to create a tailored LingQ lesson / overlay

Summary
audiosourcedrop top line source
Generate Subtitle Trans Generate Subtitle
subtitle options Own Files
Saved Lesson No LingQ Mode

Text to Speech and Translation Settings
Users can now access a range of text to speech and translation services

Summary
image 350x174
350x174 350x174
350x174 350x174
350x174 350x174
350x174 350x174

NFMeta2
image
svtcontent

Feedback / Bug Reports

We would highly appreciate any feedback / bug reports during the Alpha stage. Post a comment or get in contact with us on Discord

Thanks

Massive thank you to @samuelbtaylor and @ryldsamu for their efforts and ideas

9 Likes

I have a question. In version 2 can I import the discussion below the video from youtube to Lingq? I can’t find this option anywhere.

2 Likes

Yep. Was added in an recent update.

1 Like

Rooster, does your plugin handle YouTube TV or are you working on that? Sometimes, I find the same premium series I’m watching on regular YouTube and can import that into LingQ but if it’s not there, I have to do manual recording of the audio to use on LingQ as a workaround. But the time and effort to do that aren’t really worth the return, unfortunately. Anyway, if you have that or add that, I’m buying. Thanks.

1 Like

@davidinla

Hey mate, thanks for the interest. Being outside the U.S I was unaware YouTube TV was even a thing. I’ve done some experimenting on it this morning and have learned quite a bit.

First, it uses a very similar internal structure to normal YouTube which makes the whole process quite simple on our end. Video Meta (Title/Image/Description), Captions and the built in YouTube translator are all working as intended.

Unfortunately it uses protection mechanisms like what was found on the YouTube Red, meaning that the audio/video are only able to be decrypted by the YouTube player and can’t be downloaded/imported for external use.

This would rule out any live streams and caption-less videos but we can get a version up for Vods/replays

2 Likes

Thanks for the swift reply. I followed what you said until the end. You seemed to say can’t be done live because of encryption, but yes, it can be done for Video on Demand and replays? I’m not sure exactly how that would work or if I’m understanding correctly.

I’d like to get the Spanish transcript onto LingQ, so I can read through them after watching. I am already getting live Spanish subtitles on the YouTube TV app.

Can you clarify? And again thank you.

1 Like

Sorry about the jumbled reply I didn’t properly read what I wrote.

To clarify: Where subtitles are available on YouTube Tv we can support import to LingQ and the video overlay. If subtitles are not provided for any reason then we can’t send the audio to Whisper to create the lesson.

1 Like

That sounds pretty great for what I need to do. What would the procedure be after I contribute and download your tools? You can point me somewhere if the information already exists somewhere. Is there a link to get the tools, with the caveat that I only want to get them if they work to get YouTube TV transcripts onto LingQ when the subtitles already exist. You can contact me offline if you prefer.

Thanks for your help and work on this project.

1 Like

The functionality doesn’t exist yet for YouTube TV. I’ll send you a pm when it is.

Cheers

1 Like

Thank you. I misunderstood. The plugin works with the Paramount Plus online though?

1 Like

Paramount+ is available although hasn’t been tested a lot at this stage. If you look at the table on the first post you can find details about which stage each website is at. There are a stack more websites to be added to the list as well so stay tuned.

1 Like

Rooster Video Tools V2 Update

27 October 2024 - (3 Weeks since initial Alpha Release on YouTube)

Version 2 development has been going exceptionally well. With the help of various enthusiasts in the community we have managed to identify and fix bugs, add new features, port more websites and improve basically everything from the Version 1 Video Tools.

Support for 29 video websites are currently included in the extension and we believe this number will balloon to 50 over the next 2 weeks. Our goal has been to deliver the ultimate language learning experience for as many languages as we can and in as many contexts as possible. It finally feels like that goal is being delivered and in a scalable way.

We are hoping to conclude development of the main program within the next 2 weeks in the transition to BETA stage. If you have any specific FEATURE development requests now is the time to make them as we will be focusing on other things during that time.

Thanks very much to those who have participated in the Alpha so far.

Some more development pics below





1 Like

Rooster Video Tools V2 Update

3 November 2024 - (4 Weeks since initial Alpha Release)
Version: 0.49 ALPHA

Since inception, the greatest challenge has been developing reliable techniques that are user friendly, can be scaled without blowing the budget and that don’t require significant maintenance.

I’ve found the answer to that problem lies somewhere in-between using external services and developing and hosting the solution yourself.

This week we have implemented premium cloud services for mp3 conversion and transcription which are currently available for testing. The Nova model returns extremely accurate word-level timestamps which can be utilized in the captions overlay. In addition to this we have added a new translation source being Meta LLM for word and phrase translations.

Paying for these premium features and extending of those features in the future will require a change to how MasterLingQ is packaged and priced. We are currently considering changing the Video Tools V2 into a standalone extension with a monthly sub and PAYG approach to premium features. Using this approach the existing lifetime license holders can be rewarded and the door can be opened for more users. We will have more to say about this closer to end November.

Nova Transcription Model on Netflix


1 Like

Very bad news. I dont like subcription and I dont support your project V02.

1 Like

Hey atrapScA,

Regarding the pricing: The changes wouldn’t hurt people who already paid + the Pay as you go model would still always be available.

Meaning you don’t have to pay for a subscription in order to use the premium features.

And also: The premium features are mostly related to AI features we couldn’t otherwise make free. Meaning if you can live with the free features as they are now, you can still easily use the tool no problem.

You actually wouldn’t lose anything and only gain the option so my question now is if there is another reason you don’t support the V2 Videotools or if it is just the new premium features you can always opt out of.

1 Like

I tried Rooster and may come back to it.
Can it helps with Irish which is not yet on Lingq? I’d like to read books/ text and track known/ unknown words = translations + audio (perhaps abair.ie).

1 Like

Although we have rudimentary support for Non-LingQ languages, I wouldn’t comfortably tell you you can learn Irish well with it just yet. However, we work on a Database especially for Languages LingQ doesn’t support yet.
Just stay tuned.

Are you on the discord yet? If not, just join and be prepared to see what comes around :slight_smile:

1 Like

Sorry for the confusion. There will be no sub for lifetime members.

This only affects new customers

2 Likes

I’m sorry if this is a bit offtopic. Just asking here since it is somehow related and I think the chances of getting a substantive answer here are higher than messaging Support.

I was browing some old code of mine that used to work to import a Youtube video. It is very straightforward: it sends a request to “lessons/import/” with some Youtube URL and lets LingQ do his thing.

My issue is that it now fails when there are only auto-generated subtitles, while I’m almost positive that this worked in the past. Does anyone remember? This is the behaviour of the LingQ Import Addon anyway.

Is there something that I’m missing? Without wanting you to go into the details of your code, am I now forced manually replicate their Addon if I want to use that endpoint with Youtube videos with auto-generated subtitles?

I will perfectly understand if you don’t want to answer, I just wonder why and when it stopped working…

1 Like

@rafarafa

I have basically given up on the LingQ → Youtube API since the removal of transcribe. As you would know most of the processes happen server side and i’m unaware of any params to control the YouTube one.
A good thing about YouTube is that the data is accessible with just a url. I have a simple JS parser here that you can see, the stream urls return malformed but the subtitle urls are legitimate. There is still an issue of parsing the Autogenerated segments to vtt, you could create one yourself or change the format= on the url from json3 to vtt for a not so great vtt version.

Can send me a PM if you need the script for the parser.

https://theartofdrowning.com/lingq/youtube/parser/

3 Likes