Project

General

Profile

Feature #9796

HTTPS mirrors

Added by sajolida over 2 years ago. Updated 4 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
06/21/2017
Due date:
% Done:

67%

QA Check:
Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

We should allow or force our mirrors to server ISO images through HTTPS. This would be a cheap protection from simple MitM and other network attacks as they have been reported to us already (<>).

team: intrigeri, u


Subtasks

Bug #12833: Think about fallback DNS round-robin pool & HTTPSConfirmedu

Bug #12834: Update our validation schema/script to reject JSON files that contain HTTP mirrorsResolvedu

Feature #12835: Adjust contribute/how/mirror to only support HTTPSResolvedu

Bug #12836: Adjust our Puppet code to only support HTTPS mirrorsConfirmedintrigeri

Bug #12837: Drop HTTP mirrorsResolvedu

Feature #13597: Adjust bin/validate-config to only allow HTTPS URLsResolved


Related issues

Related to Tails - Feature #11623: Provide SHA-Checksum and HTTPS Download Rejected 08/06/2016
Related to Tails - Feature #9102: Get tails.boum.org on the Chrome HSTS preload list Resolved 03/24/2015

History

#1 Updated by intrigeri over 2 years ago

IMO, next step is to wait for Let's Encrypt to go live.

#2 Updated by sajolida over 2 years ago

  • Target version set to 2017

#3 Updated by Dr_Whax over 1 year ago

  • Description updated (diff)
  • Assignee set to u

#4 Updated by sajolida over 1 year ago

  • Related to Feature #11623: Provide SHA-Checksum and HTTPS Download added

#5 Updated by sajolida over 1 year ago

Note that right now we're forcing people to do a verification which is almost as strong as HTTPs, either through the Firefox extension, BitTorrent, or OpenPGP. So what would it bring to have HTTPS only mirrors? Security in depth? Anything else?

#6 Updated by u over 1 year ago

sajolida wrote:

Note that right now we're forcing people to do a verification which is almost as strong as HTTPs, either through the Firefox extension, BitTorrent, or OpenPGP. So what would it bring to have HTTPS only mirrors? Security in depth? Anything else?

Yes, if you don't download over Tor, HTTPS can prevent your ISP/employer/etc to see what you're downloading exactly. Although, indeed, the ISP/employer/etc can see the domain name connected to, probably the prior DNS query and could make a guess from the downloaded filesize. But I still find that quite a good additional layer of encryption.

#7 Updated by intrigeri about 1 year ago

  • Blocks Feature #9102: Get tails.boum.org on the Chrome HSTS preload list added

#8 Updated by intrigeri about 1 year ago

  • Blocks deleted (Feature #9102: Get tails.boum.org on the Chrome HSTS preload list)

#9 Updated by intrigeri about 1 year ago

  • Related to Feature #9102: Get tails.boum.org on the Chrome HSTS preload list added

#10 Updated by intrigeri about 1 year ago

u wrote:

sajolida wrote:

Note that right now we're forcing people to do a verification which is almost as strong as HTTPs, either through the Firefox extension, BitTorrent, or OpenPGP. So what would it bring to have HTTPS only mirrors? Security in depth? Anything else?

Yes, if you don't download over Tor, HTTPS can prevent your ISP/employer/etc to see what you're downloading exactly. Although, indeed, the ISP/employer/etc can see the domain name connected to, probably the prior DNS query and could make a guess from the downloaded filesize. But I still find that quite a good additional layer of encryption.

Right.

Also, whatever post-download verification, be it done by DAVE, a BitTorrent client, or OpenPGP, does not protect against exploitation with malicious data of the code used for downloading.

Another reason is that dl.a.b.o is the only reason why the nice boum.org folks cannot have HSTS for their domain; it would be nice if we could stop causing this problem to them.

And finally: it's 2016, soon 2017, and the days of plaintext HTTP are over; it must die (we can't do that for the whole WWW ourselves, but at least we can stop contributing to the "it's OK to download software over plaintext HTTP" mindset; mind you, not all users get the subtle difference between "it's mostly OK since we verify post-download" and "it's OK in all cases").

None of these reasons, taken in isolation, would be enough to make me do this work. But all together, they certainly are motivating enough :)

#11 Updated by sajolida about 1 year ago

I'm convinced now :)

#12 Updated by u 9 months ago

Next steps:

  • Wait for Chromium 47 to be released (should happen tomorrow) and to be updated in Debian.
  • Issue an email request to our current mirror operators asking them to provide HTTPS.
  • Modify the mirror pool dispatcher script used in the Tails Upgrader.
  • Modify the mirror pool dispatcher web JS script to allow only HTTPS.
  • Release a new version of DAVE using the updated mirror pool dispatcher script.

Anything else?

#13 Updated by u 9 months ago

  • Type of work changed from Sysadmin to Code

#14 Updated by intrigeri 9 months ago

  • Modify the mirror pool dispatcher script used in the Tails Upgrader.
  • Modify the mirror pool dispatcher web JS script to allow only HTTPS.

IMO it would be easier to ensure our mirror pool only contains HTTPS mirrors, i.e. "update our pre-commit hook in mirrors.git to ensure no non-TLS mirror is in the list". And then we don't need to modify the mirror pool dispatcher everywhere.

#15 Updated by u 9 months ago

current state:

$ grep url_prefix mirrors.json  | grep https | wc -l
37
$ grep url_prefix mirrors.json  | grep "http:" | wc -l
13

=> So 25.5% of our 51 mirrors are currently using http only.

5 of the remaining 13 http mirrors are using *.dl.amnesia.boum.org - so if i get it right this should be fixed by HSTS?

So we can contact the remaining 8 and ask them to set up HTTPS.

#16 Updated by u 8 months ago

I think we should set a deadline for the mirror operators because otherwise this will never happen. What do you think about June 1st? That's in about 2 months. Is that enough time to set up HTTPS? Or should we say June 15th?

#17 Updated by u 8 months ago

Here is a sample email text:

Hi,

you are currently hosting a Tails mirror. Thank you very much!

We plan to switch our mirror pool to HTTPS only in _July 2017_. 
Your mirror is one of 13 out of 50 Tails mirrors which do not currently provide TLS.

Are you able and willing to set up HTTPS for your Tails mirror? Let's Encrypt makes it pretty cheap and easy theses days.

Mirrors which do not provide HTTPS shall be removed from our mirror pool on _July 1st 2017_.

Thank you very much for helping Tails and hosting a mirror.

Obviously, the date may be different, I just try to push this forward a little bit. Please improve and comment.

#18 Updated by intrigeri 8 months ago

Here is a sample email text:

Looks good to me! Thanks for pushing this forward :)

What do you think about June 1st? That's in about 2 months. Is that enough time to set up HTTPS? Or should we say June 15th?

Two months, starting from the time when we send them the advance warning, sounds reasonable to me. Personally I won't have time to work on the next steps (drop non-HTTPS mirrors, adjust our pre-commit hook) before June 20. But feel free to set the deadline earlier if you can handle it :)

#19 Updated by u 8 months ago

I'll start sending it out now, using June 20th as a deadline. Then we can see who handles it. I can certainly take over the dropping part.

#20 Updated by u 8 months ago

mail sent.

#21 Updated by intrigeri 8 months ago

mail sent.

\o/

#22 Updated by sajolida 8 months ago

Cool!

#23 Updated by intrigeri 8 months ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

BTW, what's the plan wrt. the fallback DNS round-robin pool?

#24 Updated by u 8 months ago

We are now down to 6 out of 49 mirrors using HTTP only. One of them is deactivated since > 6 months, because it was unresponsive.

#25 Updated by intrigeri 6 months ago

The deadline is now in 5 days, and according to bin/stats, among our 45 enabled mirrors enabled, only 4 don't provide HTTPS :) Yeepee, congrats!

I suggest we complete the work on the regular mirror pool without blocking on the DNS round-robin one: stop accepting HTTP mirrors, adjust contribute/how/mirror + our Puppet code to only support HTTPS, drop HTTP mirrors, and update our validation schema/script to reject JSON files that contain HTTP mirrors. This way, most downloads will benefit from the improvement sooner. And once we're there, we can start thinking about what to do wrt. the fallback DNS round-robin pool. Makes sense?

#26 Updated by u 6 months ago

intrigeri wrote:

The deadline is now in 5 days, and according to bin/stats, among our 45 enabled mirrors enabled, only 4 don't provide HTTPS :) Yeepee, congrats!

I suggest we complete the work on the regular mirror pool without blocking on the DNS round-robin one: stop accepting HTTP mirrors, adjust contribute/how/mirror + our Puppet code to only support HTTPS, drop HTTP mirrors, and update our validation schema/script to reject JSON files that contain HTTP mirrors. This way, most downloads will benefit from the improvement sooner. And once we're there, we can start thinking about what to do wrt. the fallback DNS round-robin pool. Makes sense?

Sounds good to me. I'll create tickets for each of these items.

#27 Updated by u 6 months ago

  • Assignee changed from u to intrigeri

Reassigning this main ticket to you as all subtickets also currently belong to you.

#28 Updated by BitingBird 4 months ago

  • Target version changed from 2017 to 2018

Also available in: Atom PDF