Project

General

Profile

Bug #13206

Install by cloning sometimes silently fails from a stick installed with UUI

Added by goupille 6 months ago. Updated about 1 month ago.

Status:
Confirmed
Priority:
Elevated
Assignee:
Category:
Installation
Target version:
Start date:
09/29/2017
Due date:
% Done:

33%

QA Check:
Dev Needed
Feature Branch:
Type of work:
Research
Blueprint:
Starter:
Affected tool:
Installer

Description

some users reported that, after clicking on "install by cloning", the window just shut down without any error message. It was reported by users trying to clone from a Tails 3.0 installed with UUI and by someone using Tails 3.0 inside virtualbox. sadly there are no logs to provide.

Team: spriver, sajolida


Subtasks

Feature #14734: Test "Clone the current Tails" onto a stick installed by UUI but with an NTFS system partitionResolved

Bug #14736: Test that it is still possible to install Tails with UUIIn Progresssajolida

Bug #14907: UUI should preselect "Format in FAT32" when Tails is selectedConfirmedsajolida


Related issues

Related to Tails - Bug #10984: Test Rufus Confirmed 01/22/2016
Related to Tails - Feature #14448: Consider documenting Rufus as a workaround in case UUI doesn't work Confirmed 08/24/2017
Related to Tails - Feature #14447: Replace UUI with Win32 Disk Imager Confirmed 08/24/2017
Related to Tails - Bug #14959: Test UEFI boot with a Tails installed with UUI Resolved 11/11/2017

History

#1 Updated by intrigeri 6 months ago

  • Subject changed from Install by cloning sometimes silently fails to Install by cloning sometimes silently fails from a stick installed with UUI
  • Category set to Installation

#2 Updated by intrigeri 6 months ago

  • Status changed from New to Confirmed
  • Assignee changed from intrigeri to goupille
  • QA Check set to Info Needed

The info I need is:

  • how exactly was the device installed/upgraded
  • the full output of the tails-installer-launcher command, started in a Terminal

Thanks!

#3 Updated by goupille 5 months ago

  • Assignee changed from goupille to intrigeri
  • QA Check changed from Info Needed to Dev Needed

I sent the asked informations to intrigeri, re-assigning the ticket

#4 Updated by goupille 5 months ago

the user reported that "upgrade from iso" "install by cloning" and "upgrade by cloning" had the same result, the window shutdown with no error message.

#5 Updated by sajolida 4 months ago

Happy to try to test this myself on my Windows machine if needed.

#6 Updated by emmapeel 4 months ago

On the logs of another affected user, on a Tails installed with UUI, I can read:

ug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]: Traceback (most recent call last):
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/tails_installer/tails-installer", line 105, in <module>
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     main()
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/tails_installer/tails-installer", line 99, in main
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     win = TailsInstallerWindow(opts=opts, args=sys.argv)
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 325, in __init__
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     path=config['running_liveos_mountpoint'])
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/python2.7/dist-packages/tails_installer/source.py", line 67, in __init__
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     self.size = _dir_size(self.path)
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/python2.7/dist-packages/tails_installer/utils.py", line 57, in _dir_size
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     for item in os.listdir(source):
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]: OSError: [Errno 13] Permission denied: '/lib/live/mount/medium'

The USB stick is easily open with the File manager, and the problem happens with it mounted or not (user tested the USB stick)

#7 Updated by sajolida 4 months ago

#8 Updated by sajolida 4 months ago

And once someone managed to use Rufus successfully it would be interesting to see if Tails Installer behaves differently on a USB stick installed using Rufus. Linking this ticket to #10984.

#9 Updated by emmapeel 4 months ago

intrigeri wrote:

The info I need is:
[...]
  • the full output of the tails-installer-launcher command, started in a Terminal

Another friendly user has sent this output:


(tails-installer-launcher:16536): Gtk-WARNING **: Allocating size to GtkBox
0x5586c31b48b0 without calling gtk_widget_get_preferred_width/height(). How
does the code know the size to allocate?

(tails-installer-launcher:16536): Gtk-WARNING **: Allocating size to GtkBox
0x5586c31b48b0 without calling gtk_widget_get_preferred_width/height(). How
does the code know the size to allocate?

Traceback (most recent call last):

  File "/usr/lib/tails_installer/tails-installer", line 105, in <module>

    main()

  File "/usr/lib/tails_installer/tails-installer", line 99, in main

    win = TailsInstallerWindow(opts=opts, args=sys.argv)

  File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 325,
in __init__

    path=config['running_liveos_mountpoint'])

  File "/usr/lib/python2.7/dist-packages/tails_installer/source.py", line
67, in __init__

    self.size = _dir_size(self.path)

  File "/usr/lib/python2.7/dist-packages/tails_installer/utils.py", line
57, in _dir_size

    for item in os.listdir(source):

OSError: [Errno 13] Permission denied: '/lib/live/mount/medium'

#10 Updated by emmapeel 4 months ago

A user having this problem with UUI has made a USB stick with Rufus, and the problem was not there anymore.

Ha!

Message-Id: <>

#11 Updated by sajolida 4 months ago

It's good that you reference the Message-Id!

#12 Updated by sajolida 4 months ago

If the foundations team thinks that providing an older version of UUI might be an option, I'm happy to dig into my archives.

#13 Updated by BitingBird 3 months ago

  • Description updated (diff)
  • Target version set to 2018

#14 Updated by intrigeri 3 months ago

  • Assignee changed from intrigeri to emmapeel
  • % Done changed from 0 to 100
  • QA Check changed from Dev Needed to Info Needed

Thanks for the logs, that's interesting. There seems to be something wrong (early guess: either filesystem type, or filesystem options, or permissions) with the system installed by UUI.

goupille, emmapeel, I need:

  • the full output of the export DEBUG=1 ; tails-installer-launcher command (started in a Terminal) when this happens
  • the output of ls -lAR /lib/live/mount/medium
  • the output of mount -l
  • the output of df -T
  • the output of lsblk

Thanks in advance!

#15 Updated by intrigeri 3 months ago

In a bug report (96cd2bb82a726017b5e53c70590c1388) I've received I see:

/dev/sda1 ntfs 1.9G 1.2G 749M 61% /lib/live/mount/medium

i.e. whatever software was used to install Tails on that medium created, or reused, a NTFS filesystem. Either that software is buggy, or the user didn't follow our doc that says "Select the Fat32 Format Drive option", or our doc is outdated wrt. current UUI and the user was confused.

=> Help desk, next time you see people with this problem, ask them the output of mount -l and if /lib/live/mount/medium is NTFS, point them to our installation doc so they get themselves a correct Tails. And if you see this problem again with a proper FAT32 system partition, see my previous comment about what info I need.

sajolida & anonym: I'm not sure if we're lucky that this drive booted at all: I have no idea if booting from NTFS gives a correctly working Tails or not, and I'm pretty sure that drive won't boot in UEFI mode. Ideally, we should tell the user earlier (without waiting for something to break in an obvious way like this ticket shows) that their Tails is wrong. Cheap way: add a check + desktop notification pointing to the installation doc. Better, but harder way: add a check run at or before the Greeter, and instead of the regular Greeter window display an error + the installation doc. I don't know how high priority this would be.

#16 Updated by sajolida 2 months ago

  • Related to Feature #14448: Consider documenting Rufus as a workaround in case UUI doesn't work added

#17 Updated by sajolida 2 months ago

#18 Updated by rolan 2 months ago

Programs like Universal-USB-installer will format as NTFS without warning despite the selected "FAT32 option" when installing to a thumbdrives larger than 32GB.

#19 Updated by intrigeri about 2 months ago

  • Assignee changed from emmapeel to sajolida

rolan wrote:

Programs like Universal-USB-installer will format as NTFS without warning despite the selected "FAT32 option" when installing to a thumbdrives larger than 32GB.

Is it "32GB or larger" or "strictly larger than 32GB"?

Interesting, this could explain a lot, thanks! Next steps could be:

  1. (optional) help desk checks bug reports to see if the affected drives are >=32GB; I'm writing "optional" as I bet that some affected users simply did not check the FAT32 option; anyone up to do that?
  2. someone (sajolida, who's responsible for this part of our installation proces?) tries to reproduce; next steps assume that rolan is correct;
  3. check with UUI upstream if that behaviour is expected, and if it could be changed; according to https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system a FAT filesystem can be up to 2TB large so I'm curious why UUI would force NTFS. If sajolida doesn't want to handle this, perhaps I could do it myself (assuming this counts as Foundation Team core work);
  4. if UUI can't be fixed in this respect, document that drives larger than 32GB are not supported and display a warning when starting Tails from a NTFS system partition (as said earlier on this ticket, this could be done e.g. in live-boot i.e. very early, or after login).

#20 Updated by sajolida about 2 months ago

  • Assignee changed from sajolida to intrigeri
  • QA Check changed from Info Needed to Ready for QA

I'll report on the test from #14734 here, so that all the information is in a single thread.

Summary: UUI behaves as instructed regarding partition formatting, including on >=32GB USB stick, but Tails Installer doesn't work when run from an NTFS partition.

Details of the tests and results:

- UUI 1.9.7.7: The version we served from https//tails.boum.org/uui/Universal-USB-Installer.exe until today
  - [X] Install on FAT32 32GB without "Format in FAT32" 
    → OK! FAT32 with previous files preserved. I didn't test that Tails.
  - [X] Install on FAT32 32GB with "Format in FAT32" 
    → OK! FAT32 with previous files removed. I didn't test that Tails.
  - [X] Install on NTFS 32GB without "Format in FAT32" 
    → EXPECTED! NTFS with previous files preserved. I didn't test that Tails.
  - [X] Install on NTFS 32GB with "Format in FAT32" 
    → OK! FAT32 with previous files removed. Intermediary Tails boots fine.
  - [X] Install on NTFS 32GB with "Format in FAT32" and clone onto the intermediary Tails
    → OK! Tails Installer runs fine and final Tails boots fine.

- UUI 1.9.7.9: The version we serve starting from today
  - [X] Install on FAT32 32GB without "Format in FAT32" 
    → OK! FAT 32 with previous files preserved. I didn't test that Tails.
  - [X] Install on FAT32 32GB with "Format in FAT32" 
    → OK! FAT 32 with previous files removed. I didn't test that Tails.
  - [X] Install on NTFS 32GB without "Format in FAT32" 
    → EXPECTED! NTFS with previous files preserved.
  - [X] Install on NTFS 32GB without "Format in FAT32" and clone to final Tails
    → REPRODUCED! NTFS with previous files preserved. Tails Installer fails to start on 3.2. I didn't write the error message but it was something about permission denied and /lib/live/mount/medium. I can get the full error message if you want.
  - [X] Install on NTFS 32GB with "Format in FAT32" 
    → OK! FAT32 with previous files removed. Intermediary Tails boots fine.
  - [X] Install on NTFS 32GB with "Format in FAT32" and clone to final Tails
    → OK! Tails Installer runs fine and final Tails boots fine.

So I didn't observed what rolan was describing and, for me, there's no bug in UUI and our instructions are fine.

Now, what we could ask UUI is whether they can preselect "Format in FAT32" when Tails is selected, explaining that installing on NTFS leads to broken stuff (I would need more help on describing why the solution to this problem cannot be to make Tails work properly from an NTFS partition). UUI might be reluctant to do that because it's erasing people's file on the device but, on the other hand, it's what we're advertising on our official instructions anyway.

If you think that's a good idea, maybe create another subticket to contact UUI and I can do it (unless the Foundations Team wants to take over).

#21 Updated by sajolida about 2 months ago

If UUI doesn't do the change, for whatever reason, we could also:

  • (a) Improve the documentation to insist more on "Format in FAT32". But this won't catch everybody.
  • (b) Check how hard it would be to make it possible to clone from an intermediary Tails on NTFS. I'm not saying that we should fix Tails on NTFS, but make it possible to go on with the rest of the instructions even if the intermediary Tails is on NTFS prevent a bunch of failures.
  • (c) Add a check early like intrigeri suggested in #13206#note-15. Note that if we put this check before or instead of Tails Greeter, then people won't be able to see it in their favorite language so, especially if we can do (b), I would rather be in favor of something on the desktop inviting them to clone to a final Tails. If we can't do (b), the check should be after Tails Greeter possibly instead of the desktop.

#22 Updated by intrigeri about 1 month ago

  • Assignee changed from intrigeri to sajolida
  • QA Check changed from Ready for QA to Dev Needed

I'll report on the test from #14734 here, so that all the information is in a single thread.
[…]
Details of the tests and results:

Amazing, thanks!

So I didn't observed what rolan was describing and, for me, there's no bug in UUI and our instructions are fine.

OK. So the engineer's point of view is that everything is fine. But in the real world, the users' & help desk's point of view is that very often "it doesn't work". So I'm glad you're proposing improvements despite these test results :)

Now, what we could ask UUI is whether they can preselect "Format in FAT32" when Tails is selected, explaining that installing on NTFS leads to broken stuff (I would need more help on describing why the solution to this problem cannot be to make Tails work properly from an NTFS partition). UUI might be reluctant to do that because it's erasing people's file on the device but, on the other hand, it's what we're advertising on our official instructions anyway.

Good idea. Here's the info you requested:

  • We don't want to support NTFS because it supports ownership & permissions, and Tails requires the desktop user (amnesia, 1000:1000) to have full read access to the files we install on the system partition (otherwise Tails Installer would need even more admin credentials, which is bad from both a security and UX perspective). Given we have no control over what user UUI is run as on Windows, this implies we have no guarantee that the installed system files will be readable by the desktop user on Tails. So we need a filesystem that doesn't support ownership & permissions. To the best of my knowledge, the only such filesystem that works with UEFI is FAT.
  • Regardless of NTFS vs. FAT32, we don't want to support installing on an existing filesystem that might already have data and it's impossible to guarantee anything wrt. how pre-existing files may affect Tails' behavior. So we need "Format".

It logically follows that we need "Format in FAT32".

(As said above, EFI support might be another argument against NTFS, but I didn't actually test EFI on NTFS so let's not mention it.)

If you think that's a good idea, maybe create another subticket to contact UUI and I can do it (unless the Foundations Team wants to take over).

#14907

I'll let you adjust metadata (e.g. call it UX core work or not, set a target version or not, priority etc.) as you wish. I bet our help desk would call it high priority given the amount of pain they hear about (confirmed by what I see on XMPP).

#23 Updated by intrigeri about 1 month ago

If UUI doesn't do the change, for whatever reason, we could also:

  • (a) Improve the documentation to insist more on "Format in FAT32". But this won't catch everybody.

Yes.

  • (b) Check how hard it would be to make it possible to clone from an intermediary Tails on NTFS. I'm not saying that we should fix Tails on NTFS, but make it possible to go on with the rest of the instructions even if the intermediary Tails is on NTFS prevent a bunch of failures.

I think the only way to do this (without having to require an admin password being set and prompt for it) is to delegate the actual files copy to some privileged helper program (run e.g. via sudo). I don't know how hard it would be to do it safely, hopefully not more than 2 days of work. It increases the Tails attack surface though.

Regardless of how much work it requires, I'm not convinced it's a good idea: people do use Tails installed via UUI in production, and as long as it boots, some will keep doing that no matter what, and it's hard to prevent them from doing so (both technically and in terms of UX, e.g. there are people who can run Tails installed by UUI but not the final one installed with Tails Installer). So I think that "I'm not saying that we should fix Tails on NTFS" is highly theoretical and hard to make real. IMO either we clearly make "Tails on NTFS" unsupported (e.g. via (c) below) or we start supporting it for real, which requires more work than just (b).

  • (c) Add a check early like intrigeri suggested in #13206#note-15. Note that if we put this check before or instead of Tails Greeter, then people won't be able to see it in their favorite language

Good catch!

the check should be after Tails Greeter possibly instead of the desktop.

I don't think that's technically required (but it might be the best option in terms of UX, you tell me). We need to display the error/warning message in the chosen language once a language has been chosen. This could happen in the Greeter, e.g.:

When I start Tails installed on NTFS
Then I see an error message about it in the Greeter
And the Login button is disabled
When I select my preferred language in the Greeter
Then the error message is localized (like the rest of the Greeter's UI)
And the Login button remains disabled

Anyway, let's not put too much time into this until #14907 reaches some conclusion :)

#24 Updated by sajolida about 1 month ago

Quick note to myself: What about having an animated GIF on the selection of "Format FAT32" in the instructions :)

#25 Updated by intrigeri about 1 month ago

intrigeri wrote:

I bet our help desk would call it high priority given the amount of pain they hear about (confirmed by what I see on XMPP).

This made it into the hot topics for help desk last month so I'll bump priority.

#26 Updated by intrigeri about 1 month ago

  • Related to Bug #14959: Test UEFI boot with a Tails installed with UUI added

Also available in: Atom PDF