Project

General

Profile

Bug #14946

Feature #5630: Reproducible builds

Topic branches that lag behind their base branch don't build reproducibly with mergebasebranch build option

Added by intrigeri 11 months ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Build system
Target version:
Start date:
11/10/2017
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
bugfix/14946-mergebasebranch-breaks-reproducibility
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

intrigeri wrote on #12633#note-36:

I think there's a bug in the handling of the base branch merge or something related: https://jenkins.tails.boum.org/view/Tails_ISO/job/reproducibly_build_Tails_ISO_bugfix-14796-topicons-position/8/artifact/build-artifacts/diffoscope.html shows differing (merge) commit IDs, while the build logs pretend we're merging the exact same commit ("at commit ..."). Neither of these two commit IDs are in the official tails.git, hence my guess that a different merge commit ID is created for some reason unknown to me. I don't know what's going on here but it seems fishy. Please investigate and if this is Someone Else's Problem™ file a ticket for them, otherwise fix it. I suspect it would help debugging if we passed --verbose to this git merge, and/or displayed the ID of the resulting merge commit, or similar.

I think I've got it: two otherwise identical merge commits done at different times get different IDs. So if I got it right:

  • base branches are not affected by this bug, which is nice
  • any topic branch is affected by this bug only if it is lagging behind its base branch; that's a rather common case so it feels wrong to report such confusing info to the reviewer; it's easy to workaround (merge base branch,& push) but then you have to wait about 2h20min for the next results, IMO our already painful (some would say "bureaucratic") reviewe'n'merge process really does not need to be made more cumbersome.

I've played locally with using faketime to workaround this problem and it seems to work fine: faketime -f "YYYY-MM-DD hh:mm:ss" git merge --no-edit $whatever always creates the same merge commit ID. Note that -f is important otherwise the clock keeps running and if for some reason things get slow, then the commit can be created after the specified timestamp.

In passing, too bad git merge has no --date option, while git commit does.


Related issues

Related to Tails - Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs Resolved 10/22/2017
Blocks Tails - Bug #14924: reproducibly_build_Tails_ISO_stable Jenkins job always fails Resolved 11/06/2017

Associated revisions

Revision 469cc719 (diff)
Added by intrigeri 11 months ago

Fix reproducibility of builds of topic branches that lag behind their base branch with the mergebasebranch build option enabled (refs: #14946)

Two otherwise identical merge commits done at different times get different IDs,
and we happen to embed in the ISO the ID of the commit we're building from.

Revision 0c043dbf (diff)
Added by intrigeri 11 months ago

Fix reproducibility of builds of topic branches that lag behind their base branch with the mergebasebranch build option enabled (refs: #14946)

Two otherwise identical merge commits done at different times get different IDs,
and we happen to embed in the ISO the ID of the commit we're building from.

Revision cd1a448c
Added by bertagaz 11 months ago

Merge remote-tracking branch 'origin/bugfix/14946-mergebasebranch-breaks-reproducibility'

Fix-committed: #14946, #14933

History

#1 Updated by intrigeri 11 months ago

  • Related to Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#2 Updated by intrigeri 11 months ago

  • Subject changed from Our CI can't reproduce builds of topic branches that lag behind their base branch to Topic branches that lag behind their base branch don't build reproducibly with mergebasebranch build option
  • Category changed from Continuous Integration to Build system

(This is not specific to our CI.)

#3 Updated by intrigeri 11 months ago

  • Blocked by Bug #14933: stable branch is not reproducible: differences in some .fa.html website files added

#4 Updated by intrigeri 11 months ago

(I'll base my work on the branch for #14933 so there's a chance the resulting ISO builds reproducibly.)

#5 Updated by intrigeri 11 months ago

  • Parent task set to #5630

#6 Updated by intrigeri 11 months ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10
  • QA Check set to Ready for QA
  • Feature Branch set to bugfix/14946-mergebasebranch-breaks-reproducibility

#8 Updated by intrigeri 11 months ago

Please merge into master → stable → devel, and I'll merge into the active not-yet-reviewed topic branches so Jenkins stops showing them as non-reproducible and stops storing tons of ISOs.

#9 Updated by bertagaz 11 months ago

  • Assignee changed from anonym to bertagaz

Taking over the reviewing as part of the follow up of #12633.

#10 Updated by intrigeri 11 months ago

  • Blocks Bug #14924: reproducibly_build_Tails_ISO_stable Jenkins job always fails added

#11 Updated by bertagaz 11 months ago

  • Status changed from In Progress to Fix committed
  • Assignee deleted (bertagaz)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

bertagaz wrote:

Taking over the reviewing as part of the follow up of #12633.

Good catch, as said on IRC, I would have spent time trying to hack the merge in the build process, rather than think about the timedate affecting the merge. So I've merged it just in time for 3.3.

#12 Updated by anonym 11 months ago

  • Blocked by deleted (Bug #14933: stable branch is not reproducible: differences in some .fa.html website files)

#13 Updated by anonym 11 months ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF