Project

General

Profile

Feature #5847

Feature #10237: Refactor and clean up the automated test suite

Improve how we pass data between steps

Added by Tails over 5 years ago. Updated over 1 year ago.

Status:
In Progress
Priority:
Normal
Assignee:
-
Category:
Test suite
Target version:
-
Start date:
Due date:
% Done:

10%

QA Check:
Feature Branch:
test/5847-fix-global-vs-class-variables-issue
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:

Description

See e.g. the changes commit 61928d8 brings to torified_gnupg.feature. There the class variable gnupg_recv_key_res is used between steps. Imagine if the step that assigns it (i.e. I fetch the ... OpenPGP key...) is done before the background snapshot is saved, and some step that uses gnupg_recv_key_res (e.g. GnuPG uses the ... keyserver...) is done after the background snapshot is restored. It will work for the first scenario, but in the following scenarios gnupg_recv_key_res will be empty, because class variables are cleared between scenarios.

Perhaps steps like these should only be allowed either when no background snapshot is used, or after the background snapshot has been restored? That could be achieved by putting the following at their beginnings:

   raise if @skip_steps_while_restoring_background

Or we don't allow using class variables between steps them at all, and instead we have to use global variables (i.e. $gnupg_recv_key_res), which are not cleared ever.


Related issues

Blocked by Tails - Feature #6094: Test suite: background snapshot improvements Resolved 10/15/2015

History

#1 Updated by sajolida about 5 years ago

  • Subject changed from test suite: assigning class variables in skippable steps to Assigning class variables in skippable steps
  • Category set to Test suite
  • Starter set to No

#2 Updated by intrigeri over 4 years ago

  • Description updated (diff)

#3 Updated by anonym about 3 years ago

The new snapshot system (#6094) will make this a non-issue. There will be no skippable steps any more, and all scenarios will be completely self-contained. Adding a blocker; once merged we can stop using global ($-prefixed) variables in all steps when sharing something to a consecutive step in the same scenario, and we can close this ticket. Yay!

#4 Updated by anonym about 3 years ago

  • Blocked by Feature #6094: Test suite: background snapshot improvements added

#5 Updated by anonym about 3 years ago

anonym wrote:

The new snapshot system (#6094) will make this a non-issue. There will be no skippable steps any more, and all scenarios will be completely self-contained. Adding a blocker; once merged we can stop using global ($-prefixed) variables in all steps when sharing something to a consecutive step in the same scenario, and we can close this ticket. Yay!

Actually this is not true... non-global variables modified in steps that are run when creating the checkpoints will be lost in all but the scenario it was created in (i.e. the first one needing that checkpoint). That's the exact sort of thing that could cause very nasty issues.

It can still be done, however. For instance, we could have a dictionary/hash, say @vars, that is saved and restored along with the snapshot; this will be the only sanctioned way to store data between steps (example: @vars = { 'desktop_theme' => 'gnome', 'language' => 'english', ... }. For that we'd need to create a Snapshot (or Checkpoint?) class that essentially just stores the snapshot and @vars, along with the methods needed for interacting with the snapshot. It would probably make the new snapshot code from #6094 clearer. One concern I have is where how to store them. Currently the VM object (and hence libvirt domain) is created anew at least once for each @product scenario, so it cannot belong there. There's actually little reason for that, I think (we just need to reset it if we need a "fresh" computer), so we should probably work on making the test suite live with a single VM object, which then also can store the collection of Snapshot:s.

#6 Updated by anonym about 3 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10
  • Feature Branch set to test/5847-fix-global-vs-class-variables-issue

anonym wrote:

One concern I have is where how to store them. Currently the VM object (and hence libvirt domain) is created anew at least once for each @product scenario, so it cannot belong there. There's actually little reason for that, I think (we just need to reset it if we need a "fresh" computer), so we should probably work on making the test suite live with a single VM object

I've just pushed a (probably WIP, it hasn't been tested much) commit to the test/5847-fix-global-vs-class-variables-issue branch doing this.

#7 Updated by anonym about 3 years ago

  • Subject changed from Assigning class variables in skippable steps to Improve how we pass data between steps
  • Parent task set to #10237

#8 Updated by anonym about 3 years ago

Once we have this it should be considered if we can drop the references to 'USB drive "current"' and similar by instead storing the device name between steps some how.

#9 Updated by intrigeri over 2 years ago

  • Type of work changed from Discuss to Code

#10 Updated by u over 1 year ago

@anonym: is this ticket still relevant?

Also available in: Atom PDF