Project

General

Profile

Bug #12726

Feature #5630: Reproducible builds

There should be a date on the notes in the News section of the website

Added by goupille 4 months ago. Updated about 9 hours ago.

Status:
In Progress
Priority:
Elevated
Assignee:
Category:
-
Target version:
Start date:
06/17/2017
Due date:
% Done:

50%

QA Check:
Ready for QA
Feature Branch:
doc/12726-sanity-check-website
Type of work:
Website
Blueprint:
Easy:
Affected tool:

Description

at least to know when was the last release...


Related issues

Related to Tails - Bug #14767: ikiwiki does not order news items deterministically In Progress 10/03/2017

Associated revisions

Revision 598a022e (diff)
Added by anonym 7 days ago

bin/sanity-check-website: add script to sanity check the website.

Currently it ensures all blog posts and security advisories have valid
Ikiwiki 'meta date' directives, since we depend on it for
reproducibility (refs: #14767).

Will-fix: #12726

Revision 71e27f1d (diff)
Added by anonym 7 days ago

build-website: sanity check the website source before building it

... and make the check fatal unless `--non-fatal-sanity-check` is
passed to `build-webiste`.

Will-fix: #12726

Revision 2f9f0581 (diff)
Added by anonym about 10 hours ago

Revert "Adding templates from latest ikiwiki version"

This reverts commit 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0.

Will-fix: #12726

History

#1 Updated by intrigeri 4 months ago

Two comments:

  1. IIRC we've removed this info as it makes the ISO unreproducible. We could probably fix that by always explicitly setting a [[!meta date=]] directive.
  2. The release date is displayed in the "Install Tails x.y" block already, that looks very prominent to me.
  3. I don't know why the release date is so important in itself. Just curious: who needs this info, and why?

#2 Updated by goupille 4 months ago

intrigeri wrote:

  1. IIRC we've removed this info as it makes the ISO unreproducible. We could probably fix that by always explicitly setting a [[!meta date=]] directive.

I didn't know why it had been removed, and I can't say I understand your proposal :)

  1. The release date is displayed in the "Install Tails x.y" block already, that looks very prominent to me.

it displays only the current release.

  1. I don't know why the release date is so important in itself. Just curious: who needs this info, and why?

For a reason I can't remember, I needed to know when tails 3.0~rc1 was released, and ended up scrolling @tails_live account on Twitter... I thought that that it was weird not having this kind of information on Tails website, like, in the body of the release note, that's all...

#3 Updated by u 4 months ago

  • Assignee changed from sajolida to intrigeri

There are also publication dates in the RSS feed: https://tails.boum.org/news/index.en.rss

I agree with Goupille that having such a date is common practice on websites.
It helps to see if/that a project is still active and when some news were posted, not only the release.

I'd say that if it is easy to add the !meta date= directive, we should do that, otherwise, leave it as is. What do you think, intrigeri?

#4 Updated by intrigeri 4 months ago

  • Assignee changed from intrigeri to anonym

OK, perhaps. That's part of the reproducible builds work then (as we removed this info to help make the build reproducible).

#5 Updated by intrigeri 4 months ago

  • Parent task set to #5630

#6 Updated by sajolida 4 months ago

As someone who writes a lot of our blog posts, I think it would be good to always have a [[!meta date= and use it as the displayed date of the post.

And yes, it's a common good practice to have date on blog post and news in general.

#7 Updated by intrigeri 4 months ago

  • Target version set to Tails_3.2

(Let's be consistent with the "Deliverable for" value.)

#8 Updated by intrigeri about 1 month ago

  • Target version changed from Tails_3.2 to Tails_3.3

104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 caused this regression by removing the <TMPL_VAR CTIME> bits in the ikiwiki templates we use. This change has been live since 2.12 (2017-04-19).

To fix that we need to:

  1. ensure all blog posts have a well-formed and correct [[!meta date="…"]]
  2. add a quick'n'dirty grep check in build-website that ensures all blog posts have a well-formed and correct [[!meta date="…"]] (I really don't want us to notice that the website broke ISO reproducibility during the release process, when we merge the master branch)
  3. revert the aforementioned commit
  4. find out what this breaks in terms of reproducibility, why, and fix it

I think this is indeed part of our SponsorT 2016 work. But it can be solved on the master branch so let's handle this later this month, after 3.2 is released.

#9 Updated by intrigeri about 1 month ago

  • Priority changed from Normal to Elevated

(End of the contract is Nov 1st)

#10 Updated by intrigeri 17 days ago

  • Related to Bug #14767: ikiwiki does not order news items deterministically added

#11 Updated by intrigeri 17 days ago

intrigeri wrote:

To fix that we need to:

  1. ensure all blog posts have a well-formed and correct [[!meta date="…"]]
  2. add a quick'n'dirty grep check in build-website that ensures all blog posts have a well-formed and correct [[!meta date="…"]] (I really don't want us to notice that the website broke ISO reproducibility during the release process, when we merge the master branch)

Including translations, see e.g. #14767 for a real-life case where a tiny, hard to spot mistake made by a translator made Tails 3.2 not build reproducibly.

#12 Updated by anonym 7 days ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA
  • Feature Branch set to doc/12726-sanity-check-website

intrigeri wrote:

104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 caused this regression by removing the <TMPL_VAR CTIME> bits in the ikiwiki templates we use. This change has been live since 2.12 (2017-04-19).

To fix that we need to:

  1. ensure all blog posts have a well-formed and correct [[!meta date="…"]]

I went with this definition of "well-formed and correct" for blog posts source files (.mdwn, .html):

  • Must have exactly one line matching: /[[!meta\s+date="(.*)"]]/.
  • The date must be strictly rfc2822-compliant.

And this for the PO files:

  • Must have exactly one line matching: /msgid "[[!meta\s+date=\"(.*)\"]]\n"/.
  • ...which must be immediately followed by a line that:
    • is exactly msgstr "", or
    • matches /msgstr "[[!meta\s+date=\"([^"]*)\"]]\n"/.
  • The date in the msgstr (unless empty) must be strictly rfc2822-compliant.

Sounds good?

I optimistically implemented this in bin/sanity-check-website, pushed to the feature branch. What do you think?

(I wrote it in Ruby to encourage sajolida to extend it, if there's other things to sanity check.)

  1. add a quick'n'dirty grep check in build-website that ensures all blog posts have a well-formed and correct [[!meta date="…"]] (I really don't want us to notice that the website broke ISO reproducibility during the release process, when we merge the master branch)

Agreed. And this check should be fatal by default, but possible to opt-out from by passing an option, e.g. for contributors if they happen to want to build the wiki after we e.g. repeat #14767 by mistake and they don't feel it is their responsibility to fix it just to get the wiki building. Implemented in the feature branch.

  1. revert the aforementioned commit
  2. find out what this breaks in terms of reproducibility, why, and fix it

"this" = "the aforementioned commit"? If so, why? As long as sanity-check-website is happy => 'meta date' directives are ok, then we should be good, no?

#13 Updated by intrigeri 6 days ago

I went with this definition of "well-formed and correct" for blog posts source files (.mdwn, .html):
[...]
And this for the PO files:
[...]

Sounds good?

Yes.

I'll review the code next week.

  1. revert the aforementioned commit
  2. find out what this breaks in terms of reproducibility, why, and fix it

"this" = "the aforementioned commit"?

Yes.

If so, why? As long as sanity-check-website is happy => 'meta date' directives are ok, then we should be good, no?

I guess we should be good but we'll only know for sure once this has seen broad exposure to reproducibility testing: this ticket is about fixing one problem we had worked around with 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0, but by doing so we might have incidentally fixed other reproducibility issues.

#14 Updated by intrigeri 5 days ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed
  • s/and drop them for/and drop them from/ maybe?
  • build-website uses a $p variable that's not defined. Did you mean $arg instead? Please test this code with 0, 1 and 2 args (apart of --non-fatal-sanity-check).
  • a comment would be welcome for the !!DateTime.rfc2822(date) thing
  • wrt. meta_date = m[1]: don't we have named captured groups in Ruby?
  • there's a stray space at end of line in sanity-check-website
  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

#15 Updated by anonym 4 days ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

  • s/and drop them for/and drop them from/ maybe?

Thanks!

  • build-website uses a $p variable that's not defined. Did you mean $arg instead? Please test this code with 0, 1 and 2 args (apart of --non-fatal-sanity-check).

Whoops! Fixed, and verified working as intended.

  • a comment would be welcome for the !!DateTime.rfc2822(date) thing

Isn't !!... (or not(not(...))) a commonly accepted pattern to coerce to boolean in languages with such a type? I recall seeing it used in Ruby, JavaScript and Python, at least.

  • wrt. meta_date = m[1]: don't we have named captured groups in Ruby?

Good idea! Implemented.

  • there's a stray space at end of line in sanity-check-website

Fixed!

  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

Ah, indeed!

BTW, your example is not relevant, right, since it does show="1"?

And what about !map pages=...? To me it seems just as affected as !inline.

To get an idea of how we pass pages=... (with a glob) to various Ikiwiki directives I ran:

git grep --extended-regexp '\[\[!\S+[^\]]*pages="[^"]*\*[^"]*"' -- wiki

and got:

wiki/src/blueprint.mdwn:[[!map pages="page(blueprint/*)" show=title]]
wiki/src/blueprint/ARM_platforms.mdwn:[[!map pages="blueprint/ARM_platforms/*"]]
wiki/src/blueprint/DAVE_2.mdwn:[[!map pages="blueprint/DAVE_2/assets/*"]]
wiki/src/blueprint/SponsorW.mdwn:[[!map pages="blueprint/SponsorW/*"]]
wiki/src/blueprint/automated_builds_and_tests.mdwn:[[!map pages="blueprint/automated_builds_and_tests/*"]]
wiki/src/blueprint/bootstrapping/assistant.mdwn:[[!map pages="blueprint/bootstrapping/assistant/router/3rd_iteration*" show=title]]
wiki/src/blueprint/greeter_revamp_UI/mockups.mdwn:[[!map pages="blueprint/tails-greeter:_revamp_UI/mockups/*"]]
wiki/src/blueprint/monthly_report.mdwn:[[!map pages="blueprint/monthly_report/**" show="title"]]
wiki/src/brokenlinks.mdwn:[[!brokenlinks pages="* and !recentchanges and !blueprint/* and lang(EN)"]]
wiki/src/brokenlinks.mdwn:[[!brokenlinks pages="* and !recentchanges and !blueprint/*"]]
wiki/src/contribute/build.mdwn:[[!map pages="contribute/build/*"]]
wiki/src/contribute/design.mdwn:[[!map pages="contribute/design/*"]]
wiki/src/contribute/how/mirror.mdwn:[[!map pages="torrents/files/*.torrent"]]
wiki/src/contribute/how/promote/material.mdwn:[[!map pages="contribute/how/promote/material/*"]]
wiki/src/contribute/meetings.mdwn:[[!inline pages="page(contribute/meetings/*)" template="titlepage" sort="-title" limit="0"]]
wiki/src/contribute/release_process.mdwn:[[!map pages="contribute/release_process/*"]]
wiki/src/contribute/reports.mdwn:[[!map pages="page(contribute/reports/*)" show="title"]]
wiki/src/contribute/reports/SponsorS.mdwn:[[!map pages="contribute/reports/SponsorS/*"]]
wiki/src/contribute/reports/SponsorW.mdwn:[[!map pages="contribute/reports/SponsorW/*"]]
wiki/src/contribute/working_together.mdwn:[[!map pages="contribute/working_together/*"]]
wiki/src/contribute/working_together/roles.mdwn:[[!map pages="contribute/working_together/roles/*"]]
wiki/src/doc/first_steps/upgrade/release_notes.mdwn:[[!inline pages="page(news/version_*) and currentlang()" show="1" sort="-meta(date) age -path"]]
wiki/src/doc/upgrade/error.mdwn:[[!map pages="doc/upgrade/error/* and currentlang()" show="title"]]
wiki/src/index.html:[[!inline pages="page(news/*) and !news/*/* and !news/discussion and (currentlang() or news/report_2* or news/test_*-rc? or news/test_*beta? or news/test_*alpha?)" show="2" feeds="no" archive="yes" sort="-meta(date) age -path"]]
wiki/src/index.html:[[!inline pages="page(security/*) and !security/audits and !security/audits.* and !security/audits/* and !security/*/* and !security/discussion and currentlang()" show="2" feeds="no" archive="yes" sort="-meta(date) age -path"]]
wiki/src/news.html:[[!inline pages="page(news/*) and !news/*/* and !news/discussion and (currentlang() or news/report_2* or news/test_*-rc? or news/test_*beta? or news/test_*alpha?)" show="10" sort="-meta(date) age -path"]]
wiki/src/news.html:[[!inline pages="page(news/*) and !news/*/* and !news/discussion and (currentlang() or news/report_2* or news/test_*-rc? or news/test_*beta? or news/test_*alpha?) and tagged(announce)" 
wiki/src/recentchanges.mdwn:[[!inline pages="internal(recentchanges/change_*) and !title(change to forum/*) and !*/Discussion" 
wiki/src/security.mdwn:[[!inline pages="page(security/*) and !security/audits and !security/audits.* and !security/audits/* and currentlang()" 
wiki/src/security.mdwn:[[!inline pages="page(security/*) and tagged(security/probable) and currentlang()" 
wiki/src/security.mdwn:[[!inline pages="page(security/*) and tagged(security/fixed) and (currentlang() or security/Numerous_security_holes_in_0* or security/Numerous_security_holes_in_1*)" 
wiki/src/security/audits.mdwn:[[!inline pages="security/audits/* and currentlang()" 
wiki/src/tags.mdwn:[[!map pages="tags/*"]]
wiki/src/torrents/rss.mdwn:[[!inline pages="torrents/files/*.torrent" show="30" sort="age"]]

We have:

  1. four instances of sort="-meta(date) age -path" (all of these are for news/* and security/*)
  2. one instance of sort="age": the docs say "[l]ist pages from the most recently created to the oldest", which isn't clear to me what it means (ctime? mtime, but it already has its own sorting method? same as meta(date)?)
  3. tons of instances not specifying any sorting => default to @sort="age title"
  4. one instance of sort="-title"

So, (1) is currently handled by the sanity check. For the rest, I hope our build system standardizes enough in the environment so (4) is not an issue, and for (2-3) I don't know, since it's unclear how sort="age" works. But given that we haven't seen any issues with (2), I think it reasonable for us to simply assume (2-4) aren't problems until proven otherwise, i.e. our current sanity check is good enough.

What do you think?

#16 Updated by intrigeri 4 days ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Info Needed
  • a comment would be welcome for the !!DateTime.rfc2822(date) thing

Isn't !!... (or not(not(...))) a commonly accepted pattern to coerce to boolean in languages with such a type? I recall seeing it used in Ruby, JavaScript and Python, at least.

Right. I had missed the "?" in rfc2822_date? so I was wondering what's going on. Sorry for the noise.

  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

Ah, indeed!

BTW, your example is not relevant, right, since it does show="1"?

Right… assuming the single listed page is always the same, regardless of whatever may vary in the build environment. Is it?

I'll do the next code review pass once this is clarified.

And what about !map pages=...? To me it seems just as affected as !inline.

I see some explicit sorting in the map plugin so as long as locales don't vary we should be good.

We have:

  1. four instances of sort="-meta(date) age -path" (all of these are for news/* and security/*)
  2. one instance of sort="age": the docs say "[l]ist pages from the most recently created to the oldest", which isn't clear to me what it means (ctime? mtime, but it already has its own sorting method? same as meta(date)?)
  3. tons of instances not specifying any sorting => default to @sort="age title"
  4. one instance of sort="-title"

So, (1) is currently handled by the sanity check. For the rest, I hope our build system standardizes enough in the environment so (4) is not an issue, and for (2-3) I don't know, since it's unclear how sort="age" works. But given that we haven't seen any issues with (2), I think it reasonable for us to simply assume (2-4) aren't problems until proven otherwise, i.e. our current sanity check is good enough.

OK. Note that we've standardized lots of stuff (for unrelated reasons) to sort="-meta(date) age -path" that should avoid to avoid cases like 2. For avoidance of doubt we could do the same in other places too. And we've fixed some stuff in ikiwiki already so the deterministic setting is now taken into account in more places.

#17 Updated by anonym 4 days ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Info Needed to Ready for QA

intrigeri wrote:

  • a comment would be welcome for the !!DateTime.rfc2822(date) thing

Isn't !!... (or not(not(...))) a commonly accepted pattern to coerce to boolean in languages with such a type? I recall seeing it used in Ruby, JavaScript and Python, at least.

Right. I had missed the "?" in rfc2822_date? so I was wondering what's going on. Sorry for the noise.

  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

Ah, indeed!

BTW, your example is not relevant, right, since it does show="1"?

Right… assuming the single listed page is always the same, regardless of whatever may vary in the build environment. Is it?

Ah, indeed. I guess I was wrong; I imagine that e.g. #14767 would still have occurred with show="1" since the first item was affected. But it is odd that your example wasn't in the diffoscope report we have to #14767.

Any way, your example uses sort="-meta(date) age -path" on news/version_*, which we sanity check the meta date for, so we're good for that reason.

I'll do the next code review pass once this is clarified.

Please, go ahead!

And what about !map pages=...? To me it seems just as affected as !inline.

I see some explicit sorting in the map plugin so as long as locales don't vary we should be good.

Good!

We have:

  1. four instances of sort="-meta(date) age -path" (all of these are for news/* and security/*)
  2. one instance of sort="age": the docs say "[l]ist pages from the most recently created to the oldest", which isn't clear to me what it means (ctime? mtime, but it already has its own sorting method? same as meta(date)?)
  3. tons of instances not specifying any sorting => default to @sort="age title"
  4. one instance of sort="-title"

So, (1) is currently handled by the sanity check. For the rest, I hope our build system standardizes enough in the environment so (4) is not an issue, and for (2-3) I don't know, since it's unclear how sort="age" works. But given that we haven't seen any issues with (2), I think it reasonable for us to simply assume (2-4) aren't problems until proven otherwise, i.e. our current sanity check is good enough.

OK. Note that we've standardized lots of stuff (for unrelated reasons) to sort="-meta(date) age -path" that should avoid to avoid cases like 2. For avoidance of doubt we could do the same in other places too. And we've fixed some stuff in ikiwiki already so the deterministic setting is now taken into account in more places.

Good! So, in summary, the only thing we want to sanity check at the moment is: when !inline:ing pages with sort="meta(date)" (+ any other sorting), ensure the included pages has a valid meta date directive. And we simplify this by assuming the only such !inline targets are in news and security.

I hope we feel that that simplification is good enough, otherwise we'd have to reimplement ikiwikis pages=... parsing. :)

#18 Updated by intrigeri 3 days ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed

Any way, your example uses sort="-meta(date) age -path" on news/version_*, which we sanity check the meta date for, so we're good for that reason.

Right :)

I'll do the next code review pass once this is clarified.

Please, go ahead!

Done, code review passes!

I've run ./bin/sanity-check-website and it raised 370 problems such as:

wiki/src/news/mediapart.mdwn: 'meta date' directive contains non-rfc2822 date: Thu Dec 15 12:34:56 2016

Is this expected? What's your plan about this?

So, in summary, the only thing we want to sanity check at the moment is: when !inline:ing pages with sort="meta(date)" (+ any other sorting), ensure the included pages has a valid meta date directive. And we simplify this by assuming the only such !inline targets are in news and security.

Let me extend this a bit. There are two potential problems here:

  • non-deterministic inlined posts' date: any inline can be affected as long as it uses a template that displays the date; this is the problem that made us do 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 initially; your validation code addresses that… as long as it covers all affected inline directives; I think it does.
  • non-deterministic sorting of inlined posts: earlier you've been assuming that this isn't a problem until proven otherwise; I checked the code and sort="age" uses ctime, which is overridden by the git plugin to the date when the page was first added to the Git repo, which is deterministic; but that's not enough because it happens that a bunch of pages are added in the same commit (e.g. the commits that create PO files for new pages), so when no sort= is specified, the default sort="age title" will fallback to sort="title"; we have no guarantees that page titles are unique (e.g. 1. all translated versions of a page share the same title unless there's a meta title= directive that's translated in all languages; I believe we've never seen this problem by pure chance, because we never (?) inline pages in multiple languages; 2. if we inline pages from multiple directories, it could be that there's a page called the same in several of them; here again I think we're lucky we currently have no such case in wiki/src but it might change in the future). I think the only way to be 100% sure the sorting of inlined pages is deterministic is to add a sorting criteria that we know will sort on unique values, i.e. path ⇒ to be on the safe side, all inline directives that have no explicit sort=, or a sort= that sorts on values that are not always unique (e.g. age and/or title) should be modified to use something like sort="-meta(date) age -path". Of course, we don't care about pages (with inline directives) that we don't include in the ISO, e.g. we exclude usr/share/doc/tails/website/torrents/rss.html from the SquashFS (which explains why we've not seen any sorting problem there recently, see 149515aa7367afe22d9c3c40ae56ea5fd1e39669). At first glance only wiki/src/security/audits.mdwn needs to be fixed as proposed but please double-check. I know this is "just to be safe", doesn't fix any current issue, and is even somewhat off-topic on this ticket, but the fix is a one-liner so perhaps you won't mind doing it?

Also, your branch does not revert 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 so merging it won't be enough to close this ticket.

I've merged current master into the topic branch so it builds again on Jenkins (so far it has always FTBFS before the code you added was run, and I'd like to see it run there before we merge this :)

Finally, the result of the sanity check result seems to be ignored when building an ISO (https://jenkins.tails.boum.org/job/build_Tails_ISO_doc-12726-sanity-check-website/8/consoleFull) which is not good enough IMO: it's just too easy to miss it and release a Tails that's not reproducible.

#19 Updated by anonym about 9 hours ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

I've run ./bin/sanity-check-website and it raised 370 problems such as:

[...]

Is this expected? What's your plan about this?

The plan was to fix all these in some (semi-)automated way once we had agreed, which we now have, so I've now done it.

So, in summary, the only thing we want to sanity check at the moment is: when !inline:ing pages with sort="meta(date)" (+ any other sorting), ensure the included pages has a valid meta date directive. And we simplify this by assuming the only such !inline targets are in news and security.

Let me extend this a bit. There are two potential problems here:

  • non-deterministic inlined posts' date: any inline can be affected as long as it uses a template that displays the date; this is the problem that made us do 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 initially; your validation code addresses that… as long as it covers all affected inline directives; I think it does.
  • non-deterministic sorting of inlined posts: earlier you've been assuming that this isn't a problem until proven otherwise; I checked the code and sort="age" uses ctime, which is overridden by the git plugin to the date when the page was first added to the Git repo, which is deterministic; but that's not enough because it happens that a bunch of pages are added in the same commit (e.g. the commits that create PO files for new pages), so when no sort= is specified, the default sort="age title" will fallback to sort="title"; we have no guarantees that page titles are unique (e.g. 1. all translated versions of a page share the same title unless there's a meta title= directive that's translated in all languages; I believe we've never seen this problem by pure chance, because we never (?) inline pages in multiple languages; 2. if we inline pages from multiple directories, it could be that there's a page called the same in several of them; here again I think we're lucky we currently have no such case in wiki/src but it might change in the future). I think the only way to be 100% sure the sorting of inlined pages is deterministic is to add a sorting criteria that we know will sort on unique values, i.e. path ⇒ to be on the safe side, all inline directives that have no explicit sort=, or a sort= that sorts on values that are not always unique (e.g. age and/or title) should be modified to use something like sort="-meta(date) age -path". Of course, we don't care about pages (with inline directives) that we don't include in the ISO, e.g. we exclude usr/share/doc/tails/website/torrents/rss.html from the SquashFS (which explains why we've not seen any sorting problem there recently, see 149515aa7367afe22d9c3c40ae56ea5fd1e39669). At first glance only wiki/src/security/audits.mdwn needs to be fixed as proposed but please double-check. I know this is "just to be safe", doesn't fix any current issue, and is even somewhat off-topic on this ticket, but the fix is a one-liner so perhaps you won't mind doing it?

I've fixed wiki/src/security/audits.mdwn.

Also, your branch does not revert 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 so merging it won't be enough to close this ticket.

Reverted.

I've merged current master into the topic branch so it builds again on Jenkins (so far it has always FTBFS before the code you added was run, and I'd like to see it run there before we merge this :)

Thanks!

Finally, the result of the sanity check result seems to be ignored when building an ISO (https://jenkins.tails.boum.org/job/build_Tails_ISO_doc-12726-sanity-check-website/8/consoleFull) which is not good enough IMO: it's just too easy to miss it and release a Tails that's not reproducible.

Apparently auto/{build,clean,config} didn't set -e, now fixed.

Also available in: Atom PDF