Mark pointed out that performance was slow when they had previously attempted this. Would it be any quicker if the Firefox plugin didn’t highlight anything for you automatically and instead you had to highlight on demand? When the user would highlight a word the plugin would then look to see if a LingQ has already been created for that term on that user’s account. Also, once the api supports it, allow the user to create a LingQ if the LingQ doesn’t exist.
My programming skills are rather limited and I’m not familiar with the LingQ api so I have no idea if this would be a better or worse way to go about it. Any thoughts?
I have no idea how the vocabulary highlighter plugin would handle 20000 lingqs or more, i’ll give it a shot after I get around the writing the script. However, if I ever get around to writing a full on Firefox add-on for LingQ, I doubt it will have performance problems. Not trying to toot my own horn here, put memory caching schemes and performance programming is what I do for 40 hours a week at my day job.
Update: I had about an hour or so this evening to play with using the LingQ API to download my LingQs and generate the xml file for VocabularyHighlighter. I have generated a database with just about 5000 LingQs in it and the VocabularyHighlighter add-on seems to be handling it fine. VocabularyHighlighter has a few convenient functions too which make it rather nice. You can toggle all highlighting on and off pretty quickly, so you can choose to only highlight your LingQs when you are on a page you care about. The import database functionality as it turns out is also very nice, because it allows you to choose whether to replace the entire current database or to merge, which means that you could quickly switch languages by switching databases, and easily update databases by doing a merge. I will try to make the tool available to you all this week to generate the database file based off of your LingQs. The step after that will be to make a tool to allow synching of vocabulary added through VocabularyHIghlighter to an easy import file for importing the LingQs into LingQ. The last piece of the puzzle would be to replace VocabularyHighlighter with a LingQ branded FF add-on with LingQ-like LingQ creation. Oh to have more free time (or programming help eh guys? nudge nudge)… at any rate, we are well on our way. Look for updates soon.
@odiernod If there’s anything I can do to help at all, please let me know. Sounds very promising from what you’ve said. I work with .NET for my job, but I’ve never looked at writing an Firefox addon; I know a bit of Java, but I’ve never used JavaScript. Would be an experience, that’s for sure.
A concern for me at least, would be that for the language I want to study, I’d need to work out a way to quickly split words quickly and efficiently as well. Although, I guess that the LingQ list could be used to combat this, and anything unknown to the database could just be left as is, for the user to highlight as they see fit.
Edit: Just done some reading up, I didn’t realise that FF add-ons could use SQLite which makes it sound a lot easier to work with, I had thought that this would be unavailable. I use SQL a lot everyday for my work, so I may well be able to lend a hand.
Hi Lyise, the “XML File” that I generate off of the list of LingQs returned from the LingQ API is in fact just an xml serialized SQLite database. It is my goal at the moment to provide some tools for LingQ users built around the VocabularyHighlighter firefox addon just to get something functional as soon as possible, and then we can start gathering feedback for a full out Add-on. In order to make a serious LingQ add-on however, the LingQ API will need to be bolstered a bit. The biggest drawback at the moment is the lack of a CreateLingQ method. I sent an email to the author of VocabularyHighlighter asking if I could use the source for his add-on as a starting point for a LingQ add-on, I’ll let you know if he gets back to me. If you wanted to get familiar with Firefox add-ons by creating an add-on to highlight text on a page from an sqlite database that would give us a base to go off of if i get turned down by the vocabularyhighlighter guy. The fact that you are studying an Asian language will help as well, as I mainly stick with the European ones and would be less able to help on bugs/feature requests for stuff such as character splitting.
As an Asian language learner, Rikaichan and Peraperakun are great because they do the splitting for you based on the entries in their dictionary. I have always thought this would have been the best way for LingQ. No splitting algorithm, just split based on existing LingQs. If there are none, then no splitting until there are some.
I also would like to see the option of seeing a dictionary as I browse with the LingQ addon. I do not want to just collect words with no hints and then add them to LingQ. I want to be able to read properly and then revise the LingQs later.
Basically Rikiachan and Peraperakun’ s look up bar do all of that for me but with some major limits: they do not export a csv file that I can properly import to LingQ, and even if they did they force me to choose one definition which may not be optimal; they do not grab the phrase; there is no highlighting.
Even with those limits I prefer to use them and just import the word list to LingQ, which I then re-LingQ and add hints to. I just like the seamlessness of it compared to importing text and then kind of feeling guilty about not finishing stuff.
Thanks and I am grateful for anything : ) Just thought I would give some feedback
＠odiernod I’ve had a look through some of the code for Vocabulary Highlighter today, it doesn’t seem as bad as I thought. I guess a work around for a Create LingQ method would be to output a CSV which can be uploaded via the vocab page, and then an option on the extension to “synchronise” LingQs afterwards. I’ll have a play around with some highlighting and see how it goes; whether or not you get permission from the authour of Vocabulary Highlighter it’d be nice to know.
@dooo I think it might be best, which languages lacking spaces at least, to only flag known LingQs and leave the rest as is, as you’ve described. At the end of the day, the guys at LingQ has been working on this for a fair while, and it’s not going to be a minor task to replicate that. Plus, I think for an add-on like this, it wouldn’t really be that suitable for novices of the language, and you don’t need a huge amount of knowledge to be able to make an educated guess on a split, check if that’s correct, and try something else if it’s wrong. Not as pretty as LingQ’s method, but a starting point.
As for a dictionary look up, I imagine it’d be easy enough to make a pop-up window with a pre-filled search for an online dictionary. For example, jisho.org (which I use for Japanese) is just http://jisho.org/words?jap=search (e.g. 範 - Jisho.org )
Edit: Okay, the LingQ forums broke the 2nd link there, but copy and paste works. Silly %s
I think as long as we are going to seriously start working on this, a code repository might be in order. I think I’ll set something up in Google Code tonight or tomorrow.
@Lyise, even if he doesn’t give us permission, rebuilding it using his source as a model shouldn’t take long.
As for the pop up dictionary, yeah we could do just like LingQ does, have an iFrame which shows the dictionary in it with the word in the proper place in the url string. That would certainly be the coolest piece of functionality, but also the least important, as there are plenty of current dictionary firefox add-ons that people can use in the meantime.
Its not the prettiest interface, but it works. I originally wrote this as a Windows app, but I know some of us here are not fans of Windows, so I provided a web interface for it instead so everyone can use it.
Notes: I have not tried Asian languages, they may or may not work.
I have filtered out status 4 LingQs, only status 3 and below make it to the xml file.
Validation of viewstate MAC failed. If this application is hosted by a Web Farm or cluster, ensure that configuration specifies the same validationKey and validation algorithm. AutoGenerate cannot be used in a cluster.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Web.HttpException: Validation of viewstate MAC failed. If this application is hosted by a Web Farm or cluster, ensure that configuration specifies the same validationKey and validation algorithm. AutoGenerate cannot be used in a cluster.
Weird, I have an X. Does the escape key on your keyboard work?
Well, even if you don’t decide to use it, thanks for being a tester for my service! We’ll make sure there is an x for you when we roll out our LingQ add-on.
I just pointed Vocabulary Highlighter developer Son Nguyen to this forum thread. Over the years I’ve been providing him with feedback on how to improve the add-on, particularly in combination with reading Chinese text. At the time he optimized his code considerably after I suggested people might want to import large collections of words and characters, as opposed to importing lists of words currently being studied. I know Son also has an interest in developing add-ons for Google Chrome.
I’m not sure if Son wants to participate in this thread, and whether or not he will be able to respond not being a member of the site, but I’ll definitely let you know as soon as he gets back to me.
Thank you CleverClogs: It would really speed up our LingQ Firefox add-on development efforts if he allows us to use his VocabularyBuilder code as a base.
Have you checked out the LingQs to Vocabulary Highlighter import tool I wrote yet? I posted a link to it a few posts back. I used Unicode encoding so it “should” work with Asian languages, but I need a little feedback.
