Project

General

Profile

Feature #12633

Feature #5630: Reproducible builds

Lower the workload caused by reproducible builds Jenkins jobs

Added by intrigeri 5 months ago. Updated 30 days ago.

Status:
Confirmed
Priority:
Elevated
Assignee:
Category:
Continuous Integration
Target version:
Start date:
06/03/2017
Due date:
% Done:

0%

QA Check:
Feature Branch:
Type of work:
Sysadmin
Blueprint:
Easy:
Affected tool:

Description

Currently, when I push a commit to a base branch, 3 ISOs are built from the same commit:

  • one ISO is built by the build_Tails_ISO_$branch job;
  • two ISOs are built by the reproducibly_build_Tails_ISO_$branch job and then compared.

We really need to build only 2 of these ISOs to get what we want, so there's room for improvement wrt. our CI infrastructure latency and throughput => usefulness.

I suggest instead having any successful build_Tails_ISO_$branch job trigger another job, that would:

  • get as input the ISO built by build_Tails_ISO_$branch (a checksum would be enough in the best case, but we really need that ISO to compare when they differ), and any relevant parameter e.g. the commit ID and base branch commit ID;
  • build a second ISO image (ideally with different build settings e.g. a time offset applied and a different CPU, but this is certainly not a blocker for this ticket)
  • run diffoscope to compare these ISOs
  • if these ISOs are the same, then delete all artifacts; otherwise, archive both ISO build artifacts + the diffoscope output

This looks very similar to what we already do for the test_Tails_ISO_* jobs, so I expect it will be pretty cheap to implement.

And as a bonus, in most cases the 2 ISOs will be built on two different isobuilders, which increases the value of the reproducibility testing :)


Related issues

Related to Tails - Feature #12654: Introduce more variations in our reproducibility CI tests Confirmed 06/07/2017
Blocks Tails - Feature #13436: Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA Confirmed 07/07/2017

History

#1 Updated by intrigeri 4 months ago

  • Related to Feature #12654: Introduce more variations in our reproducibility CI tests added

#2 Updated by intrigeri 4 months ago

  • Assignee changed from bertagaz to intrigeri

I'll take care of it unless you want it.

#3 Updated by bertagaz 4 months ago

intrigeri wrote:

I'll take care of it unless you want it.

As you wish. That's easy and fun to set up and good to play with our Jenkins jobs setup, but if you ever want to do something else, feel free to reassign to me.

#4 Updated by intrigeri 4 months ago

As you wish. That's easy and fun to set up and good to play with our Jenkins jobs setup, but if you ever want to do something else, feel free to reassign to me.

Sure!

#5 Updated by intrigeri 3 months ago

  • Blocks Feature #13436: Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA added

#7 Updated by intrigeri 3 months ago

  • Description updated (diff)

#8 Updated by intrigeri 3 months ago

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_3.2 to Tails_3.3

I've scheduled some time to do this on October 9-10. Setting elevated prio as we'll be close to the end of our contract.

#9 Updated by intrigeri 30 days ago

  • Assignee changed from intrigeri to bertagaz

bertagaz wrote:

intrigeri wrote:

I'll take care of it unless you want it.

As you wish. That's easy and fun to set up and good to play with our Jenkins jobs setup, but if you ever want to do something else, feel free to reassign to me.

Please go ahead.

Also available in: Atom PDF