Project

General

Profile

Feature #11123

Bug #7161: Support more than 24 HTTP mirrors

Have the Upgrader use the new mirror pool design

Added by intrigeri about 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Infrastructure
Target version:
Start date:
02/13/2016
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
{tails,iuk,perl5lib}:feature/11123-new-mirror-pool
Type of work:
Code

Description

The status quo, if we don't do anything, would be that we go on pointing to http://dl.amnesia.boum.org/ in UDFs, and then all IUK downloads will be pulled from the remaining DNS round-robin pool. Is it acceptable, or do we want the Upgrader to use the new mirror pool?


Related issues

Blocks Tails - Feature #11284: Update the DNS round-robin mirror pool to only include a few fast and reliable HTTP mirrors Resolved 03/25/2016

Associated revisions

Revision a52ff653 (diff)
Added by intrigeri almost 2 years ago

Add mirror-pool-dispatcher as a Git submodule.

We are going to run some of its code from Tails Upgrader.

refs: #11123

Revision 610f7de2 (diff)
Added by intrigeri almost 2 years ago

Install tails-transform-mirror-url + its dependencies, and add it to the Node.js modules search patch.

refs: #11123

Revision a714df5c
Added by anonym almost 2 years ago

Merge remote-tracking branch 'origin/feature/11123-new-mirror-pool' into devel

Fix-committed: #11123

History

#1 Updated by intrigeri about 2 years ago

  • Assignee set to intrigeri

Oops, seems that I failed to put this on my radar.

#2 Updated by intrigeri about 2 years ago

  • Target version changed from Tails_2.2 to Tails_2.3

#3 Updated by intrigeri about 2 years ago

  • Priority changed from Normal to High

Rationale: if we decide that the Upgrader needs to use the new mirror pool, then the corresponding code will need to be written early enough to go into Tails 2.4. I'd like to know ASAP if I need to schedule time for this.

#4 Updated by intrigeri about 2 years ago

  • Subject changed from Design how the Upgrader will use the new mirror pool design to Have the Upgrader use the new mirror pool design

I've started drafting the code, and will push this a bit further. If it gets too complicated I might reconsider using the fallback DNS pool for downloading IUKs, but so far I'm confident I can get this done in time for 2.4.

#5 Updated by intrigeri about 2 years ago

  • Status changed from Confirmed to In Progress
  • Type of work changed from Discuss to Code

#6 Updated by intrigeri about 2 years ago

  • Target version changed from Tails_2.3 to Tails_2.4

#7 Updated by intrigeri almost 2 years ago

  • Feature Branch set to {tails,iuk,perl5lib}:feature/11123-new-mirror-pool

#8 Updated by intrigeri almost 2 years ago

  • Blocked by Feature #11384: Record the current state of our mirror pool in JSON added

#9 Updated by intrigeri almost 2 years ago

  • % Done changed from 0 to 20

So, I have something that works in the context of the iuk.git's test suite. Next step is to build Debian packages and test in a running Tails, which will only be possible once #11384 is done.

#10 Updated by intrigeri almost 2 years ago

FTR, #11384 has seen enough progress so it's not a blocker for testing this. But of course, it remains a blocker for merging this.

#11 Updated by intrigeri almost 2 years ago

Tested in Tails 2.2, after installing tails-{iuk,perl5lib} built from their debian_feature-11123-new-mirror-pool branch, and manually applying the changes from feature/11123-new-mirror-pool: it fails to run tails-transform-mirror-url because one construct used in mirror-dispatcher.js is not supported by Jessie's nodejs (while it is on sid).

#12 Updated by intrigeri almost 2 years ago

  • Assignee changed from intrigeri to anonym
  • % Done changed from 20 to 50
  • QA Check set to Ready for QA

tl;dr: it now works! Packages uploaded.

That bug was fixed by adding fallback code to mirror-pool-dispatcher.js, to support this use case. And now I see Tails Upgrader use a mirror URL it got using mirror-pool-dispatcher.js, in the testing environment described above.

anonym, can you please review the 3 topic branches and then reassign to me? We can't merge until the blocking ticket is fully done, but still you might want to save yourself (and perhaps me) from a future rush, with an early review :)

#13 Updated by u almost 2 years ago

@intrigeri, could you please add a comment to bin/tails-transform-mirror-url in mirror-pool-dispatcher.git, so that we know what this file is for and by what it's used? I think that would be useful. Thanks!

#14 Updated by intrigeri almost 2 years ago

@intrigeri, could you please add a comment to bin/tails-transform-mirror-url in mirror-pool-dispatcher.git, so that we know what this file is for and by what it's used? I think that would be useful. Thanks!

Good idea! Done.

#15 Updated by intrigeri almost 2 years ago

Updated the submodule so that it points to the latest JS code.

#16 Updated by intrigeri almost 2 years ago

  • Blocked by deleted (Feature #11384: Record the current state of our mirror pool in JSON)

#17 Updated by intrigeri almost 2 years ago

intrigeri wrote:

FTR, #11384 has seen enough progress so it's not a blocker for testing this. But of course, it remains a blocker for merging this.

#11384 is basically done, and our mirror pool is ready to receive requests => please merge this for 2.4 (otherwise, we have a problem: these branches are not exactly of the kind we want to merge during the 2.5 point-release cycle, and until they are merged, Tails Upgrader will use the fallback DNS pool at dl.a.b.o; but that pool will very soon be reduced to ~5 mirrors, and we want to use it as little as possible, so it would be much better if downloads from the Upgrader started to use our new mirror pool before).

#18 Updated by intrigeri almost 2 years ago

  • Blocks Feature #11284: Update the DNS round-robin mirror pool to only include a few fast and reliable HTTP mirrors added

#19 Updated by anonym almost 2 years ago

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

#20 Updated by anonym almost 2 years ago

  • Status changed from Fix committed to Resolved

#21 Updated by BitingBird almost 2 years ago

  • Priority changed from High to Normal

Also available in: Atom PDF