Project

General

Profile

Feature #10382

Feature #5288: Run the test suite automatically on autobuilt ISOs

Implement the specified notification system for test suite failures on Jenkins

Added by intrigeri over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Continuous Integration
Target version:
Start date:
10/16/2015
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
Type of work:
Sysadmin

Description

This is what follow from #10296 once we deem it's time to go ahead.


Related issues

Blocked by Tails - Feature #10296: Reevaluate how the email notifications for failed automatic tests scale Resolved 09/28/2015

History

#1 Updated by intrigeri over 2 years ago

  • Parent task set to #5288

#2 Updated by intrigeri over 2 years ago

  • Tracker changed from Bug to Feature

#4 Updated by intrigeri over 2 years ago

  • Blocked by Feature #10296: Reevaluate how the email notifications for failed automatic tests scale added

#5 Updated by intrigeri over 2 years ago

  • Blueprint set to https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/

#6 Updated by bertagaz over 2 years ago

FYI this will be achieved by reverting commit 6dc5489 from the puppet-tails repo.

Then the test jobs in our JJB repo will still have the email notification activated, but the generation script will output only one email to be notified for the build and test jobs, which will be the same that we set up for the build jobs (tails-rm or last commiter).

#7 Updated by bertagaz over 2 years ago

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_1.7 to Tails_1.8

Postponing and raising priority. Will work on that on the coming days.

#9 Updated by intrigeri over 2 years ago

IMO this ticket should be marked as blocked by... whatever blocks it. #10296 is useful in itself, but it's simply too vague to express what we have to do before we can turn on the big red switch. I guess most of the blockers are subtasks of #10288. Once they're marked as blocking this ticket, then it'll be easier to sort them apart of the mass of #10288 subtasks, which will help a lot anonym and kytv organize their work :)

#10 Updated by intrigeri over 2 years ago

intrigeri wrote:

IMO this ticket should be marked as blocked by... whatever blocks it. [...] I guess most of the blockers are subtasks of #10288.

For what it's worth, I've had a quick look and I'm not so sure: the concerning amount of jobs that are aborted due to inactivity seems to be the top reason of non-successful test suite runs (out of the last 10 runs of test_Tails_ISO_devel, 6 aborted that way, 2 were succesful, and 2 failed; on stable it's "only" 3 aborted jobs out of 10).

I don't know if email notifications go out for aborted jobs, so I don't know if/how this bug is blocking this very ticket, or rather #5288. bertagaz?

In any case, as requested a month ago (#9157#note-18): the aborted jobs bug seems to be very relevant to making #5288 deliver what it promised, so it's a pretty serious one, please track it with a ticket; its exact initial priority/timeline is a factor of the above question.

#11 Updated by intrigeri over 2 years ago

(Created #10717, gonna post some research there soon.)

#12 Updated by bertagaz over 2 years ago

  • Target version changed from Tails_1.8 to Tails_2.0

Postponing

#13 Updated by intrigeri over 2 years ago

Given #10502#note-16 and #10717#note-6, I think we should now go ahead and enable notifications for non-stable branches. We'll do that on Tuesday unless there are objections, or earlier if e.g. anonym and bertagaz say they agree.

#14 Updated by anonym over 2 years ago

intrigeri wrote:

Given #10502#note-16 and #10717#note-6, I think we should now go ahead and enable notifications for non-stable branches.

+1

#15 Updated by intrigeri over 2 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA

Deployed, we'll see in a few days how it goes.

#16 Updated by bertagaz over 2 years ago

You'll see that there was a tiny bug like in test_Tails_ISO_web-10400-farsi-only #4 :

06:50:16 Sending e-mails to: $NOTIFY_TO
06:50:16 ERROR: Invalid Addresses

I've fixed that in commit 7170700 of the puppet-tails repo.

I've started a new build to test it: https://jenkins.tails.boum.org/job/build_Tails_ISO_web-10400-farsi-only/6/

#17 Updated by intrigeri over 2 years ago

I've fixed that in commit 7170700 of the puppet-tails repo.

Looks OK, let's see how it goes :)

#18 Updated by intrigeri over 2 years ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

Works!

Also available in: Atom PDF