Project

General

Profile

Bug #9049

Feature #10034: Translation web platform

Search for easier ways to open the website to translation

Added by BitingBird over 3 years ago. Updated 8 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
-
Start date:
12/28/2015
Due date:
% Done:

100%

QA Check:
Feature Branch:
weblate:tails/
Type of work:
Research

Description

git is not easy for less-geeky translators. Transifex is easy, but it's not adapted for the website (no branches, only strings without context provided, which doesn't work to translate doc pages).

We should search what else exists out there. What do other well-translated sites use? Note that our website changes a lot, so more or less static solutions don't work.


Subtasks

Feature #10800: Check if Pontoon fits our requirements for a translation platformRejectedmuri

Feature #11104: Try Pootle as translation platformRejected

History

#1 Updated by sajolida over 3 years ago

  • Assignee set to emmapeel
  • QA Check set to Info Needed

I know that emmapeel has been looking at Weblate (http://weblate.org/) which has a better Git integration, allows you to browse a tree of sources (instead of having a list of files like Transifex). So I'm assigning this ticket to her for more info.

#2 Updated by intrigeri over 3 years ago

The way I would handle this is:

  1. list the features and desirable behaviour the current setup gives us;
  2. sort them by MUST / SHOULD / MAY (e.g. BitingBird suggested the other day that we could lower the bar a bit for new translation teams, and allow them to translate only the master branch -- if we agree on this, then perhaps a lot more candidate software will seem acceptable; I suspect some of the other historical blockers for using web-based translation tools could be re-negotiated in the same way, under today's light);
  3. evaluate possible solutions, based on the specs agreed upon earlier.

Now, this entire process can go through several iterations, instead of trying to come up with the perfect specs right away. And if others prefer to work on this in different ways, I'm very happy :)

I'd just like to avoid seeing people pissed off because they discover requirements they were not aware of, after spending hours evaluating some piece of software that clearly doesn't satisfy said requirements.

#3 Updated by emmapeel over 3 years ago

intrigeri wrote:

The way I would handle this is:

  1. list the features and desirable behaviour the current setup gives us;
  2. sort them by MUST / SHOULD / MAY (e.g. BitingBird suggested the other day that we could lower the bar a bit for new translation teams, and allow them to translate only the master branch -- if we agree on this, then perhaps a lot more candidate software will seem acceptable; I suspect some of the other historical blockers for using web-based translation tools could be re-negotiated in the same way, under today's light);
  3. evaluate possible solutions, based on the specs agreed upon earlier.

Now, this entire process can go through several iterations, instead of trying to come up with the perfect specs right away. And if others prefer to work on this in different ways, I'm very happy :)

I'd just like to avoid seeing people pissed off because they discover requirements they were not aware of, after spending hours evaluating some piece of software that clearly doesn't satisfy said requirements.

I think this is a great way to go. Personally I felt very difficult to start the translation process because the system looked very confusing when I was explaining it to other possible translators. But I really didn't looked about solutions till now.

So gathering requirements, apart of tails-i10n, should be done in -dev?

I think that git is not a skill many good translators have or will have any time soon, and it would be great that we can use their help.

First I envisioned a total solution but now I feel we just need a proxy between the translators and git.

I have been looking into Weblate and it works great for translators, as you can

  • see which files need translation, or get notified by email
  • you can leave comments and see the translations other people made
  • you can download and edit offline, and then upload again
  • you get a series of benefits of common work: glossary, inconsistent translation checks, no repeated work, etc
  • you can suggest translations without being logged in
  • you can do one line now and the rest tomorrow
  • can help new translators to translate the most important pages to be able to go on production
  • has machine learning and translation memory services, including a local service than can be fed .po files for consistency with a simple command in the host :
    build_tmdb -d /var/lib/tm/db -s en -t cs locale/cs/LC_MESSAGES/django.po
    
related to updates,
  • it is a bit burocratic to add a new translation component,
  • the importing otions didnt went very well with the ikiwiki filetree but it can be fixed by hand.
but once added:
  • it's linked to its own repo/branch for each translation component you add an refreshes alone with lazy commit after a while or when you call an update (all this can be automated)
  • the kind of work it does (.po associated files) can deal well with two separate git paths with rebases once in a while
  • can update from upstream all the time and will increase the verification and consistency of translations
  • doesn't need to be a backdoor, can be firewalled by one person per group very easily
  • you can always get on the server and play with the repo!

My take after a couple of days playing with the weblate is:

  • I would at least at first and with a per*language team basis, just start centralising the translations there, to gather un logged people inputs and have a map of what it is to translate
  • Bridge that repo and the tails master through a human firewall: one person in the team can take care of getting those commits on tails master. all the other translators can just translate!

once a file is finished, translator says so and can be reviewed like any other .po file in the world. the reviewer can copy it to overwrite the file on her repo, and review it. if its ok can ask for a pull or if its too complicated somebody of the team can do it.

we can always add more automated setups but i think they can lead to trouble. although as each file can have it's own repo, we could use it to translate things different than the website, and then the automated updates may make sense. right now i think it is more a buffer for translation work to be held without impacting the website work and wihout forcing a steep learning curve in people already busy that can translate on their free time or at work meetings without having to learn more than english.

and after testing I can confirm it is quite addictive, in a sick windows-defragmentation snowcrash way.

#4 Updated by intrigeri over 3 years ago

So gathering requirements, apart of tails-i10n, should be done in -dev?

I don't think it's worth putting -dev@ in the loop. We can do that later if we come up with proposals/compromises that affect the development process, for example the release manager's work (in that specific case, perhaps putting anonym and I in the loop is enough).

#5 Updated by Dr_Whax over 3 years ago

Another option worth considering which was shown to me at CTFestival is the "Transifex live"[1] option. Which allows you to translate the site in context, it might have some limitations, but apparently there is a possibility to request Transifex to implement certain features.

[1] http://docs.transifex.com/developer/live/gettingstarted

#6 Updated by BitingBird over 3 years ago

Seems like Transifex Live requires you to enable JS to display translations. You can't simply import the .po (if you can, it's not documented).

It's sad, the "translate in context" was great. One less candidate...

#7 Updated by u over 3 years ago

In the past I've been using Pootle as a standalone translation webinterface for po files. Weblate however seems to be able to integrate with different Git branches and can make commits with correct authorship, as one can read here: http://blog.cihar.com/archives/2012/05/11/pootle-vs-weblate/

Using something like this would allow people to either use the webinterface or the command line i guess.

These tools also feature the points raised by emmapeel:
  • see which files need translation, or get notified by email
  • you can leave comments and see the translations other people made
  • you get a series of benefits of common work: glossary, inconsistent translation checks, no repeated work, etc
  • you can do one line now and the rest tomorrow
  • can help new translators to translate the most important pages to be able to go on production
  • has machine learning and translation memory services
    etc.

#8 Updated by emmapeel over 3 years ago

Dr_Whax wrote:

Another option worth considering which was shown to me at CTFestival is the "Transifex live"[1] option. Which allows you to translate the site in context, it might have some limitations, but apparently there is a possibility to request Transifex to implement certain features.

[1] http://docs.transifex.com/developer/live/gettingstarted

Transifex live could be used to gather strings somehow and reuse them, but is not reading the .po files, just string on the webpage, and those may not be the same.

For example, the strings that generate links

#9 Updated by u over 3 years ago

emmapeel wrote:

Dr_Whax wrote:

Another option worth considering which was shown to me at CTFestival is the "Transifex live"[1] option. Which allows you to translate the site in context, it might have some limitations, but apparently there is a possibility to request Transifex to implement certain features.

[1] http://docs.transifex.com/developer/live/gettingstarted

Transifex live could be used to gather strings somehow and reuse them, but is not reading the .po files, just string on the webpage, and those may not be the same.

Correct.

For example, the strings that generate links.

Correct, as our files are based on markdown, not HTML.

Also, sometimes different PO files are loaded on a single compiled HTML page..

The other problem of Transifex Live is, as BitingBird already said (but I had to read it myself ;)), that it requires to add a piece of 3rd party Javascript to the Tails website in order to load the translations. Not an option.

#10 Updated by u over 3 years ago

A very helpful person from Transifex wrote me this, which we could check out too:
..........................................................................
There's not a native support for Github inside Transifex but there is the
TXgh https://github.com/jsilland/txgh integration that is a neat little
Sinatra server that serves as a bridge between Transifex and Github. It
ensures you always have up-to-date translations in Github, and that the
source text in your Transifex resources are kept up-to-date with the text
in Github. The server does that by updating the content in Transifex for
every change to a source translation file in Github. In addition, for every
change to a translation in Transifex, the server creates a commit and push
it to Github with the new translations. You can read more about it in our
docs http://docs.transifex.com/developer/integrations/github.

I believe that setting it up will help you a lot manage your localization
process as it will automate it and eliminate all the back and forth with
files. Keep in mind that you can also use the command line Client
http://docs.transifex.com/developer/client/ to automate and speed up
things.
..........................................................................

Setting this up requires some prerequisites which make us less independent.
However, it would allow for a large existing translator base, which we would not necessarily have if we simply use our own webinterface.

#11 Updated by u over 3 years ago

Started a discussion about the issue on tails-l10n mailing list: https://mailman.boum.org/pipermail/tails-l10n/2015-March/002072.html

#12 Updated by sajolida over 3 years ago

  • Blueprint set to https://tails.boum.org/blueprint/translation_platform/

Please compile requirements and other information on this blueprint: https://tails.boum.org/blueprint/translation_platform/

#13 Updated by emmapeel about 3 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 50
  • QA Check changed from Info Needed to Ready for QA
  • Feature Branch set to weblate:tails/

So, we are testing a weblate instance.

It is pushing to https://git-tails.immerda.ch/weblate/tails

That branch also has Italian, Spanish, Catalan, Persian and maybe other languages, not sure how to go about it.

I added the new languages in some commits, and there were also initial small commits from Weblate in many files.

I also committed unmerged files (oops!) but I've fixed them afterwards.

#14 Updated by intrigeri about 3 years ago

  • Category changed from Internationalization to Infrastructure

(You seem to be confusing i18n with l10n. Our website is already i18n-ready.)

#15 Updated by sajolida about 3 years ago

  • Parent task set to #10034

#16 Updated by sajolida over 2 years ago

  • QA Check deleted (Ready for QA)

I don't think this is "Ready for QA".

#17 Updated by u 9 months ago

Although we have a fait accompli of using weblate nowadays, I will leave this ticket open for a while, just in case we realize we do want to change to something else nevertheless.

#18 Updated by u 8 months ago

  • Status changed from In Progress to Resolved

Let's stick with our setup!

Also available in: Atom PDF