@odiernod I tried your conversion tool, but your code breaks on my entering the language code for Chinese. I tried ZH, CH and CN - all with the same 404 error. I get the impression I’m using the wrong language code. Could your code offer a drop-down list of tried-and-true, supported language codes? Can you catch errors?
@cleverclogs, the language code for chinese is zh, so I’m not sure where the error is. I could make it pretty, catch errors and such, but I wanted to throw it out there as quick as possible to get people to be able to use it. If it becomes popular I will invest the time to clean it up a bit. I guess I’ll have to make myself a chinese profile to see where the error is coming from.
@odiernod - ok, I know what the error was: I entered “ZH” in all uppercase instead of “zh”. As soon as I figured that out the .xml file got downloaded to my system without issues. I’ll play with it after dinner-time rush hour is over here. To be continued…
@odiernod Brilliant, your tool works great.
@odiernod thanks for putting together this page, but I think I might have a few too many LingQs (I have almost 23,000) - the page timed out the first time with the below error (it worked the second time):
Exception Details: System.Net.WebException: The operation has timed out
Source Error:
Line 21: string lang = txtLang.Text;
Line 22: //TODO check length of lang
Line 23: List langLingqs = LingQApi.GetLingQs(apiKey, lang);
Line 24: string filename = lang.ToUpper() + “LingQVocabulary” + “.xml”;
Line 25: byte[] output = VocabDictBuilder.BuildVocabDictionary(langLingqs, filename);
Source File: \boswinfs03\home\users\web\b1443\whl.odiernod\LingQTools\VHLingQGenerator.aspx.cs Line: 23
Out of curiosity for cleverclogs’ problem - could it be the case that is entered? If I use “ja” Japanese is fine, if I use “JA”, I get the 404 error that cleverclogs is getting - would it be possible to change:
Line 23: List langLingqs = LingQApi.GetLingQs(apiKey, lang);
to:
Line 23: List langLingqs = LingQApi.GetLingQs(apiKey, lang.ToLower());
Edit: Nevermind there were a load of responses while I was typing, oops!
ah, when I get home tonight I will make it case insensitive. Lyise’s stack trace also alerted me that a) I need to turn off stack traces in errors, and b) I forgot to finish implementing security measures. No one try to cross site script me!
Out of curiosity Lyise, how many LingQs did you have? It looks like it times out waiting for LingQ to respond. I’ll have to increase the timeout during the call.
@odiernod I wasn’t sure if you’ve deliberately left that one while testing or not to be honest. EDIT: just saw your question, I have just shy of 23,000 LingQs (22,916 to be precise) and after the XML was produced, I have 15,000.
I’ll be honest, I was expecting Vobaulary Highlighter to be a lot slower than it is, given I’ve put a 6.5MB XML into it (15,000 LingQs) it runs really quickly. The two (Odiernod’s produced XML and Vocabulary Highlighter) work together really well. This suddenly seems a lot more promising than I had thought.
Ah yes, all 22,916 of those LingQs get pulled through the LingQ API, in one shot as there is no paging, and that is sure to take a long time. I then go through and filter out all status 4 lingqs, dropping you down to 15000.
Next up will be creating the vocabulary import file to import your vocab created through the VH addon back into LingQ. I am hesitant to provide a web solution to this as it would require not only re-getting all of the user’s lingqs, but also requiring them to upload an export of their dictionary to my tool in order to do the comparison and return to them the file to import. This would be handled much better through the add-on itself, which is why I am hoping to hear back from the developer soon. In the mean time I have another little project I am working on, and soon maybe I’ll be able to get back to studying languages 
Don’t forget, here is the project page for the future LingQ firefox addon: Google Code Archive - Long-term storage for Google Code Project Hosting.
＠Odiernod I had a play around the other day and it took me several minutes to wait for LingQ to generate and download my list of LingQs, so I was expecting it to take a while. Easy test for time outs, though!
I’ve been having a play around with Firefox myself to try and get some basics down, but haven’t had much time at the minute. I’ve read through a fair bit of the code for VH to get a (very) brief overview of how that works, and some things that we wouldn’t need (e.g. VH works out for itself that “apply” and “applies” are the same word). Hopefully I’ll get more time later in the week, or the weekend, to get over the initial hump of getting used to the way Firefox wants me to talk to it. The javascript itself, on the other hand looks pretty nice from what I’ve picked up elsewhere.
I agree with you about it being easier to do the synchronising with the add-on itself, I imagine we can use the LingQID for this, so that it could conceivably handle a change in hint, phrase, status and word, as well as quickly checking for matches and where there’s a new “LingQ” on the add-on, that isn’t yet on LingQ itself (in the case of the latter, I imagine it would then need to produce the CSV to upload to LingQ).
@Odiernod, my list of only 400 Chinese words took just seconds to convert using your tool, even less to display it as xml in the VH Import Database form, but the import (more precisely: database merge) process itself is stalling at the moment. I’ve let it run for about 10 minutes. I believe that should have sufficed to finalize the process. I’m afraid something went wrong, but I wouldn’t know where to start debugging. I’ll need to force Firefox to shut down. I’ll give it another shot some time later.
@Lyise I have yet to look too deeply into the addon. I extracted the xpi and thats about as far as I got. I’m glad that you are doing the legwork on it! 
I too should familiarize myself with this xul structure and speaking with Firefox.
I just made a plea for API updates on the LingQApi functionality. While creating the csv is a good workaround, creating new lingqs through the api as the user creates them would be much preferred of course, and a much more attractive system for members and possibly new users.
Dominick, I’ve tested your tool and it worked quickly with no problems on the three languages I am studying. Thank you very much. I am truly impressed with the Highlighter too, I did not know that kind of things existed.
Having Lingq features on any website would be fantastic, I believe it would be a huge step forward for Lingq.
@CleverClogs
That happened to me a few times as well. I just pressed the escape key, closed and reopened the VH settings, and the imports were there. Hopefully its the same for you
@odiernod - Wow! This works great, as does the Vocabulary Highlighter add-on. I tried it for French in which I have thousands of LingQs and it works well. Obviously, to figure out a way to incrementally update new LingQs both in the the LingQ database and in the highlighter app db would be great. We will take a look at the API and see what we can do to support your efforts!
@mark, thanks for the help and encouragement. I’m excited about what we’ll be able to produce as well, it will really turn LingQing into a powerhouse.
Updates made to the tool import page, the lang code is no longer case sensitive, I increased the wait timeout to the LingQAPI for those with a crap ton of LingQs, so it should no longer time out on you guys.
I changed the encoding of the xml file to UTF-8, so the file sizes it produces are much smaller now, it should still support the asian character sets, please let me know if it no longer does.
Before the current latest updates, it did my Dutch lingqs and it took less than 30 seconds. Around 20,000 of them - so, the performance isn’t too bad at all.
@Odiernod Ok, I successfully escaped the import process without needing to restart Firefox (wasn’t actually aware that I could). Thank you so much for that hint, and also for making your conversion tool so generously available to us all.
BTW, on reopening the VH options panel, my previous database had vanished. I get the impression the database merge process stalled and failed because my existing database and the one created by your tool somehow didn’t like each other, maybe because my database had definitions for pens and yours hadn’t. LingQ’d words now get properly highlighted by VH. Best of luck and thanks again!
Let me know how urgent everybody feels is it that we get the ability to export your LingQs created in Vocabulary Highlighter. I was planning on waiting until the LingQ API was updated to do something like that, but its already caught me a couple times where I started to create a “lingq” on another page through VH and went “oh crap, I’m not in LingQ right now” so I was wondering if anyone else was making any use of this or not.
I’d personally wait too (if I was coding it) and as a user, I AM waiting for it (for that functionality to arrive). Looks like you’re doing great work there Odiernod.
Hey, I wonder if it would be able to have a word count displayed somewhere. Like this - 100/1000 (10%) - which could be added to toolbars. Just an idea.
I’m looking forward to this, but not using it to add words. For me, status changes would be very useful. Wait for API changes, so the LingQ example can contain the word in context.