Feature #8183

Ship a 64-bit (x86_64) instead of 32-bit userspace

Added by Dr_Whax over 2 years ago. Updated 7 months ago.

Status:ResolvedStart date:10/11/2016
Priority:NormalDue date:
Assignee:intrigeri% Done:

100%

Category:Hardware support
Target version:Tails_3.0
QA Check: Blueprint:
Feature Branch:feature/8183-64bit-userspace Easy:
Type of work:Code Affected tool:

Description

Currently, Tails ship an x86 (32-bit) userland but will load an x86_64 (64-bit) kernel if the system you're using has 64-bit support.

Supporting the "32-bit userspace on 64-bit hardware" combination has historically caused lots of trouble, both for developers and for users (e.g. #11518, #9969, #5606). Also, software built for 64-bit processors is more interesting from a security standpoint (e.g. it's harder to bruteforce offsets/addresses, ASLR becomes stronger in that sense as is PIE support).

So, we have a few good reasons to consider switching our userspace to 64-bit. This implies to drop support for 32-bit hardware. Is it acceptable to do that in Tails 3.0, that we will release at some point between 2017Q2 and 2018Q1?

32-bit vs. 64-bit kernel stats among WhisperBack bug reports:

32-bit % 64-bit %
2014Q2 31 15 171 85
2014Q3 53 18 244 82
2014Q4 34 13 226 86
2015Q1 30 10 243 89
2015Q2 27 15 155 85
2015Q3 36 14 213 86
2015Q4 17 7 210 92
2016Q1 32 8 349 91
2016Q2 14 6 201 93
2016Q3 18 7 215 92

Note that a good share of the 32-bit systems are virtual machines: e.g. in 2016Q1, 11 of the 32 32-bit systems were VirtualBox and VMware. It seems safe to assume that the hardware able to run Tails in a VM is most likely 64-bit, and is running a 64-bit host OS (this seems plausible given our current hardware requirements, and e.g. the VirtualBox ones are probably due to https://www.virtualbox.org/ticket/11037 that forces us to tell users to set up a 32-bit VM). So we should just ignore the 32-bit VMs when looking at these stats.

Other than those, we have (32 - 11) / (32 + 349) = 5.5% of bare metal 32-bit systems. On #8183#note-29 we have analyzed these systems, and to sum up, among these 21 bare metal systems:

  • 4 supports only 64-bit CPU so will still work once we switch to full 64-bit (let's blame syslinux CPU auto-detection) => no regression
  • 1 supports max. 512MB of RAM => is not supported currently
  • 3 unknown
  • 10 will be 10+ years old when we release Tails 3.x, and support max. 2GB of RAM => we can be that this hardware won't last much longer
  • the 3 remaining systems are from 2009 or 2012, and support max. 2GB of RAM

=> even including the 10+ years old systems in the equation, we're talking of dropping support for 16 (4.2%) of systems that currently report bugs about Tails.

arch.sh Magnifier (418 Bytes) sajolida, 04/10/2015 11:43 AM


Related issues

Related to Tails - Feature #5606: Add VirtualBox host software Confirmed
Related to Tails - Bug #9969: Re-enable dkms modules building with Linux 4.x Resolved 08/11/2015
Related to Tails - Feature #11638: Tell 32-bit users why Tails cannot start Resolved 08/15/2016
Related to Tails - Bug #11663: Clarify the scope of hardware support Confirmed 08/19/2016
Related to Tails - Feature #11734: Test the 64-bit ISO with 32-bit UEFI hardware Resolved 08/26/2016
Related to Tails - Bug #11873: "Upgrade from ISO" fails from 32-bit Tails with 64-bit ISO Resolved 10/11/2016
Related to Tails - Feature #12163: Announce in advance that 3.0 will be 64-bit only Resolved 01/30/2017
Blocks Tails - Bug #7505: Video is broken with switchable graphics In Progress 07/06/2014
Blocks Tails - Feature #11829: Switch to the Debian-packaged aufs kernel module Resolved 09/23/2016
Blocks Tails - Bug #11518: Linux 4.x QXL 64-bit kernel modesetting breaks 32-bit X.Org Resolved 06/08/2016

Associated revisions

Revision 3e07d3ef
Added by intrigeri about 1 year ago

Only build the VirtualBox kernel modules for the 32-bit kernel.

It's both hard and useless to build it for 64-bit in the current state
of things, as long as we're shipping a 32-bit userspace.

Also, install virtualbox-* from jessie-backports, since the version in
Jessie is not compatible with Linux 4.x.

refs: #9969, #8183

Revision 6149b6f8
Added by intrigeri 10 months ago

Tor Browser "release" process: also import 64-bit tarballs.

refs: #8183

Revision 5a7a53c9
Added by intrigeri 10 months ago

Switch to a 64-bit userspace: first steps.

refs: #8183

Revision f4282f22
Added by intrigeri 10 months ago

Switch to a 64-bit userspace: migrate contributors' documentation.

Note that IMO it's not worth migrating the end-users' documentation
at this stage: it can be done mostly mechanically, and doing it now
would send us deep into merge conflict hell until this branch is merged.

refs: #8183

Revision 435cc0d7
Added by intrigeri 10 months ago

Don't install I2P, so that I can make progress.

refs: #8183

Revision eb5f96ba
Added by intrigeri 10 months ago

Test suite: drop obsolete check about kernel architecture.

refs: #8183

Revision a3e0745a
Added by intrigeri 10 months ago

Test suite: adjust to the fact that we now support running as a 64-bit guest in VirtualBox.

refs: #8183

Revision 03a4f50f
Added by intrigeri 10 months ago

Test suite: simplify memory erasure scenarios' name.

(x86-64 is now post-modern.)

refs: #8183

Revision 6f0a2e77
Added by intrigeri 10 months ago

Firewall: drop rules that depend on the i2psvc user.

I had to temporarily disable I2P in 435cc0d.

refs: #8183

Revision 19582877
Added by intrigeri 10 months ago

Test suite: drop code that is not used anymore.

refs: #8183

Revision 301f13e5
Added by intrigeri 10 months ago

aufs hook: drop obsolete loop construct.

refs: #8183

Revision 2a1c8a94
Added by intrigeri 10 months ago

Simplify code.

refs: #8183

Revision 02f0cb86
Added by intrigeri 10 months ago

Adjust menu and file names to better reflect what they are used for.

refs: #8183

Revision b04913ad
Added by intrigeri 7 months ago

Merge branch 'feature/8183-64bit-userspace' into feature/stretch

Closes: #8183

Revision 8c129ade
Added by intrigeri 5 months ago

Hardware requirements: don't pretend most computers from 2005-2008 will support Tails 3.0.

refs: #11663, #8183

Revision 7d6b2376
Added by intrigeri 5 months ago

Hardware requirements: update processor link (x86 → x86-64).

refs: #11663, #8183

History

#1 Updated by BitingBird over 2 years ago

  • Tracker changed from Bug to Feature
  • Category set to Hardware support
  • Status changed from New to Confirmed

#2 Updated by intrigeri over 2 years ago

I've asked frontdesk if they want to collect stats about what kernel (32 vs. 64-bit) is used on machines that report bugs.

#3 Updated by intrigeri over 2 years ago

Stats for 2014 Q2-Q3:

vmlinuz % vmlinuz2 %
2014Q2 31 15 171 85
2014Q3 53 18 244 82

Note that I've dropped the 2014Q1 stats, as they do not mean anything wrt. 32-bit vs. 64-bit: before Tails 0.23 (March 18, 2014), vmlinuz2 was a 686-pae kernel, not a 64-bit one.

#4 Updated by intrigeri over 2 years ago

These statistics were generated with:

find -iname "*.eml" | while read email ; do
   echo "${email}" 
   gpg --quiet --output "${email%.eml}.asc" --decrypt "${email}" 
done

for folder in 2014Q{1,2,3} ; do
   echo "${folder}" 
   echo "vmlinuz" 
   grep --files-with-matches "BOOT_IMAGE=/live/vmlinuz " ${folder}/*.asc | wc -l
   echo "vmlinuz2" 
   grep --files-with-matches "BOOT_IMAGE=/live/vmlinuz2 " ${folder}/*.asc | wc -l
done

#5 Updated by Dr_Whax over 2 years ago

I think that 18% of users on only X86 hardware is quite a lot and we can't just simply drop them. There's simply no excuse to do that.

Perhaps we should drop this for now, revisit this idea when we have reached our Tails 3.0 goals and see how much work it would be at that point in time?

#6 Updated by intrigeri over 2 years ago

I think that 18% of users on only X86 hardware is quite a lot and we can't just simply drop them. There's simply no excuse to do that.

Agreed. My point, in starting to gather these stats now, was not to make a decision immediately. It was rather to provide data points when we come back to it, say in 6-10 months, when Tails/Jessie is almost ready and (oh, surprise) has hardware requirements that are unlikely to be satisfied by more that a third of these 32-bits systems.

I wouldn't be surprised if the 32-bit/64-bit ratio dropped quite a bit until then anyway: hardware gets old, hardware dies. The remaining 32-bit boxes used in production around me won't live another full year.

Perhaps we should drop this for now, revisit this idea when we have reached our Tails 3.0 goals and see how much work it would be at that point in time?

I agree, assuming you meant 2.0.

#7 Updated by intrigeri over 2 years ago

  • Related to Bug #8485: Consider including an x86 kernel with SMP support added

#8 Updated by intrigeri over 2 years ago

  • Type of work changed from Code to Research

#10 Updated by intrigeri about 2 years ago

  • Assignee set to sajolida
  • Target version set to Tails_1.4
  • QA Check set to Info Needed

sajolida, may you please compute these stats for 2014Q4 and 2015Q1? Then, you can drop the milestone, the assignee and qa check.

#11 Updated by sajolida about 2 years ago

Here you go:

./2014Q4

32-bit 34 13%
64-bit 226 86%

./2015Q1

32-bit 30 10%
64-bit 243 89%

With the script in attachment.

#12 Updated by intrigeri about 2 years ago

OK, thanks! It seems that 32-bit is slowly losing ground.

#13 Updated by intrigeri about 2 years ago

  • Assignee changed from sajolida to intrigeri
  • Target version changed from Tails_1.4 to Tails_1.7
  • QA Check deleted (Info Needed)

#14 Updated by intrigeri over 1 year ago

  • Assignee changed from intrigeri to sajolida
  • QA Check set to Info Needed

sajolida, as promised I'm nagging you six months later: may you please run your stats script again, report back here, and reassign to me?

#15 Updated by sajolida over 1 year ago

Here you go:

./2015Q2

32-bit 27 15%
64-bit 155 85%

./2015Q3

32-bit 36 14%
64-bit 213 86%

#16 Updated by u over 1 year ago

sajolida wrote:

Here you go:

./2015Q2

32-bit 27 15%
64-bit 155 85%

./2015Q3

32-bit 36 14%
64-bit 213 86%

woah!

#17 Updated by sajolida over 1 year ago

  • Assignee changed from sajolida to intrigeri
  • QA Check deleted (Info Needed)

#18 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_1.7 to 2016

Thanks! So in a year there's been no statistically meaningful drop of 32-bit usage. Let's revisit 6 months after Tails/Jessie is out.

#19 Updated by sajolida over 1 year ago

#20 Updated by sajolida over 1 year ago

./2015Q4 is better:

32-bit 17 7%
64-bit 201 92%

#21 Updated by sajolida about 1 year ago

./2016Q1 is stable despite 2.0:

| 32-bit | 32 | 8% |
| 64-bit | 349 | 91% |

#22 Updated by sajolida about 1 year ago

But actually a good share of these are virtual machines :)

grep -rl "BOOT_IMAGE=/live/vmlinuz " * | while read file ; do grep -A 1 system-product-name $file ; done | grep -v system-product-name | sort | uniq -c | sort -rn

     10 VirtualBox
      3 Satellite A100
      3 MS-7728
      2 Inspiron 510m                   
      2                                 
      1 VMware Virtual Platform
      1 TravelMate C310              .     
      1 Stylistic ST5112
      1 MM061                           
      1 Hi-Fi A85S3
      1 dynabook SS S31 120S/2W
      1 Aspire 5610Z    
      1 Aspire 2020
      1 AOA110
      1 AMILO Pro Edition V3405
      1 1101HA
      1 1015CX

#23 Updated by Dr_Whax about 1 year ago

The question in my opinion that gets raised now is, at which % of 32-bit users are we going to say, this is low enough that we should consider shipping an X86_64 iso. Will we wait another month to see if usage of 32-bit kernels are staying the same and then move to make a decision? Since then, it's been 8/9 months since Tails/jessie.

#24 Updated by intrigeri about 1 year ago

sajolida wrote:

But actually a good share of these are virtual machines :)

I think it's safe to assume that the hardware able to run Tails in a VM is most likely 64-bit, and is running a 64-bit host OS (this seems plausible given our current hardware requirements, and e.g. the 10 VirtualBox ones are probably due to https://www.virtualbox.org/ticket/11037 that forces us to tell users to set up a 32-bit VM). So IMO we should just ignore the 32-bit VMs from these stats.

Other than those, we have (32 - (10+1)) / (32 + 349) = 5.5% of bare metal 32-bit systems.

DrWhax wrote:

The question in my opinion that gets raised now is, at which % of 32-bit users are we going to say, this is low enough that we should consider shipping an X86_64 iso. Will we wait another month to see if usage of 32-bit kernels are staying the same and then move to make a decision?

I think that at about 5% (given the above), we can already seriously start considering the switch to 64-bit: waiting for the long tails of 32-bit hardware to die can very well take years, and IMO we've already reached the acceptable limit to move on. And we already have 2 years of data, which sounds more than enough to have confidence in our estimates.

Still, IMO it would be nice to combine the switch to 64-bit with another, easier to understand, change in hardware requirements. I'm thinking e.g. of the move to Debian Stretch. I would not be ready to commit to it formally, but it would be nice to informally make the "if I get a 'new' box to be able to upgrade to Tails 2.0, then it'll work for all 2.x releases and I won't have to get another computer before 3.x" reasoning valid. Granted, that's quite a weak argument. I'm not sure. I could also live with switching now (as in: whenever any of us has time to make it happen, which would be ~2016Q4 I guess).

#25 Updated by intrigeri about 1 year ago

Note that shipping both 32-bit and 64-bit kernels makes e.g. #10298 much harder than it should be (#9969); that's one argument in favour of switching to 64-bit, instead of spending lots of time to support two kernels now, and then dropping all that work in 6-12 months.

#26 Updated by intrigeri about 1 year ago

  • Related to Bug #9969: Re-enable dkms modules building with Linux 4.x added

#27 Updated by intrigeri about 1 year ago

  • Related to Bug #11518: Linux 4.x QXL 64-bit kernel modesetting breaks 32-bit X.Org added

#28 Updated by intrigeri about 1 year ago

  • Related to deleted (Bug #8485: Consider including an x86 kernel with SMP support)

#29 Updated by intrigeri about 1 year ago

sajolida wrote:

But actually a good share of these are virtual machines :)

I've looked at the bare metal systems on that list.

Some of those are irrelevant for the discussion at hand:

  • dynabook SS S31: 2009, unknown by memory upgrade wizards, probably soldered RAM given form factor
  • Hi-Fi A85S3 and MS-7728: supports only 64-bit CPU, no idea why it's listed there, let's blame syslinux auto-detection
  • AOA110: 2007-2008, max. 512MB RAM => not supported

Relevant for this discussion:

  • Aspire 2020: 2004, max. 2GB RAM
  • Dell Inspiron 510m: 2004, max. 2GB RAM
  • TravelMate C310: 2005, max. 2GB RAM
  • Toshiba Satellite A100: 2006, max. 2GB RAM
  • Dell Inspiron 6400 MM061: 2006, max. 2GB RAM
  • Aspire 5610Z: 2007, max. 2GB RAM
  • Fujitsu AMILO Pro V3405: 2007, max. 2GB RAM
  • Asus Eee PC 1101HA: 2009, max. 2GB RAM
  • Eee PC 1015CX: 2012, max. 2GB RAM

My next question would be: do we actually support these systems in practice? I.e. were the bug reported about issues that prevent one from using Tails, and if yes, were we able to fix it? We've seen issues with old graphics cards vs. GNOME Shell (#11096), that we were not able to solve.

sajolida, if it's not too painful for you, it would be great if you could have a look at the bug report threads regarding these (relevant) systems, to give us an idea of whether they actually work on Tails currently. If you want to save time, only look at the ones that are from 2007 or later (I suspect that the 10+ years old hardware among those won't last much longer).

#30 Updated by intrigeri 11 months ago

FTR sajolida sent me some data, but he did it using a sharing service that expires stuff after a while, and sadly I didn't look at it early enough so I could not access it. Anyway, I think we have enough info to make a decision wrt. Tails 3.0 (I could do #10298 without blocking on this topic so it's less an emergency than what I thought a couple months ago), so I'll sum this up and will send a proposal on tails-dev@, and add the issue to the next monthly meeting's agenda.

#31 Updated by intrigeri 11 months ago

  • Subject changed from Investigate shipping an x86_64 iso to Investigate shipping an x86_64 ISO
  • Description updated (diff)

#32 Updated by intrigeri 11 months ago

  • Subject changed from Investigate shipping an x86_64 ISO to Investigate shipping 64-bit (x86_64) instead of 32-bit userspace

#33 Updated by intrigeri 11 months ago

  • Type of work changed from Research to Discuss

#35 Updated by intrigeri 11 months ago

  • Subject changed from Investigate shipping 64-bit (x86_64) instead of 32-bit userspace to Ship a 64-bit (x86_64) instead of 32-bit userspace
  • Assignee deleted (intrigeri)
  • Priority changed from Low to Normal
  • Target version changed from 2016 to Tails_3.0
  • Type of work changed from Discuss to Code

The people attending the last monthly meeting agreed with this proposal, and at the summit we agreed too.

#36 Updated by intrigeri 11 months ago

  • Related to Feature #11638: Tell 32-bit users why Tails cannot start added

#37 Updated by intrigeri 10 months ago

  • Feature Branch set to feature/8183-64bit-userspace

#38 Updated by intrigeri 10 months ago

  • Blocks Bug #7505: Video is broken with switchable graphics added

#39 Updated by intrigeri 10 months ago

  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri

#40 Updated by intrigeri 10 months ago

http://nightly.tails.boum.org/build_Tails_ISO_feature-8183-64bit-userspace/lastSuccessful/archive/ has 64-bit ISOs now. Due to #11694 one has to manually switch to VT 2 (with CTRL-ALT-F2) after clicking "Login" in the Greeter. At first glance, ISOs built from this branch work just as fine as those built from feature/stretch, but the state of the test suite on these branches is not good enough yet to be sure.

#41 Updated by sycamoreone 10 months ago

  • Related to Bug #11663: Clarify the scope of hardware support added

#42 Updated by intrigeri 9 months ago

  • Blocks Feature #11829: Switch to the Debian-packaged aufs kernel module added

#43 Updated by intrigeri 7 months ago

  • Related to Feature #11734: Test the 64-bit ISO with 32-bit UEFI hardware added

#44 Updated by intrigeri 7 months ago

  • Related to Bug #11873: "Upgrade from ISO" fails from 32-bit Tails with 64-bit ISO added

#45 Updated by intrigeri 7 months ago

  • Related to deleted (Bug #11518: Linux 4.x QXL 64-bit kernel modesetting breaks 32-bit X.Org)

#46 Updated by intrigeri 7 months ago

  • Blocks Bug #11518: Linux 4.x QXL 64-bit kernel modesetting breaks 32-bit X.Org added

#47 Updated by sajolida 7 months ago

I removed the FAQ we had on 64-bit proc with 89539bb.

#48 Updated by intrigeri 7 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

#49 Updated by sajolida 7 months ago

  • Description updated (diff)

#50 Updated by intrigeri 5 months ago

  • Related to Feature #12163: Announce in advance that 3.0 will be 64-bit only added

Also available in: Atom PDF