Project

General

Profile

Bug #12641

Feature #5630: Reproducible builds

Comment changes in POT files make ISO builds non-reproducible

Added by intrigeri 4 months ago. Updated 6 days ago.

Status:
Fix committed
Priority:
Normal
Assignee:
-
Category:
Build system
Target version:
Start date:
06/05/2017
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
bugfix/12641+potfile_line_comments
Type of work:
Code
Blueprint:
Easy:
Affected tool:

Description

Sometimes we update a program of ours and as a result the line numbers encoded in comments, in the POT file generated by refresh-translations, differ. And then POT + PO files will get updated every time we build an ISO, resulting in non-reproducible builds. This happens quite frequently (e.g. yesterday), and generally I'm baby-sitting Jenkins results closely enough to notice, and then I run refresh-translations, commit and push to fix the problem. It's a waste of my time, and I don't even want to consider sharing this dumb work better. We'd better simply eliminate the root cause of this problem :)

To this end, I think we instead should:

  • make refresh-translations ignore changes in comments by default, just like it already ignores changes in the POT creation date header; this way ISO builds won't be made needlessly unreproducible by such trivial changes;
  • give refresh-translations a --force option, that we'll use in the release process to ensure we regularly update even those usually ignored comments.

I guess both intltool_update_po and intltool_update_pot functions and the refresh-translations script need to be changed. Perhaps import-translations as well but I'm less sure (one would need to look exactly when/how/why we run it to make a good decision).

Who wants to do it? Either anonym or Ulrike, I guess? If you prefer I can handle it myself, stealing from your time budget.


Related issues

Blocked by Tails - Bug #14459: Some branches fail to build an ISO because we merge their base branch too late Resolved 08/28/2017

Associated revisions

Revision 5efee3d7 (diff)
Added by intrigeri 7 days ago

normalize_pot(): convert to extended regexp and avoid extreme quoting caused by shell double quotes (refs: #12641).

Revision 8391f358 (diff)
Added by intrigeri 7 days ago

normalize_pot(): split long line somewhat (refs: #12641).

Revision 3ec831a4
Added by anonym 7 days ago

Merge remote-tracking branch 'origin/bugfix/12641+potfile_line_comments' into devel

Fix-committed: #12641

History

#1 Updated by u 3 months ago

  • Assignee changed from anonym to u

I can take that one.

#2 Updated by u 15 days ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from u to anonym
  • QA Check set to Ready for QA
  • Feature Branch set to 451f:tails/bugfix/12641+potfile_line_comments

I was not really able to test this in a real life environment, as I never update the translations of our programs. However, I tried using some fake pot files and it seemed to work. Can you please review and improve this if necessary? Thanks!

#3 Updated by anonym 14 days ago

  • % Done changed from 0 to 50
  • Feature Branch changed from 451f:tails/bugfix/12641+potfile_line_comments to bugfix/12641+potfile_line_comments

I've pushed a few improvements and bug fixes, and implemented --force. Let's see what jenkins thinks!

#4 Updated by u 14 days ago

anonym wrote:

I've pushed a few improvements and bug fixes, and implemented --force. Let's see what jenkins thinks!

Thanks! <333

#5 Updated by intrigeri 9 days ago

  • Blocked by Bug #14459: Some branches fail to build an ISO because we merge their base branch too late added

#6 Updated by intrigeri 9 days ago

FTBFS on Jenkins due to #14459.

#7 Updated by anonym 7 days ago

  • Assignee changed from anonym to intrigeri

Note: the feature branch was force pushed right before posting this comment! Pull!

It builds! https://jenkins.tails.boum.org/job/build_Tails_ISO_bugfix-12641-potfile-line-comments/14/

I hope it also does what it should -- I must admit I haven't felt I've had enough time (and the time has been so fragmented!) to really get to understand what's going on in these scripts, mostly due to my unfamiliarity with this side of the gettext toolbox. But I am on the optimistic side of hope, at least! :)

#8 Updated by anonym 7 days ago

u, I would also like to point you to 0f0a980df96e19e31ca37714feb6be7270cfb747. I did RTFM for diff(1) after spending almost two hours on getting a breathtakingly ugly diff + while read + grep monstrosity to work. Blessed be GNU diffutils! :)

#9 Updated by intrigeri 7 days ago

  • Assignee changed from intrigeri to anonym
  • % Done changed from 50 to 70

Works fine according to my test matrix! I've pushed two code style improvements on top, that don't break anything I've tested. Please review and merge :)

#10 Updated by anonym 7 days ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 70 to 100

#11 Updated by anonym 6 days ago

  • Assignee deleted (anonym)
  • QA Check changed from Ready for QA to Pass

Also available in: Atom PDF