Project

General

Profile

Bug #12726

Feature #5630: Reproducible builds

There should be a date on the notes in the News section of the website

Added by goupille 6 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
-
Target version:
Start date:
06/17/2017
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
doc/12726-sanity-check-website
Type of work:
Website
Blueprint:
Starter:
Affected tool:

Description

at least to know when was the last release...


Related issues

Related to Tails - Bug #14767: ikiwiki does not order news items deterministically Resolved 10/03/2017
Related to Tails - Bug #14933: stable branch is not reproducible: differences in some .fa.html website files Fix committed 11/08/2017

Associated revisions

Revision 598a022e (diff)
Added by anonym 2 months ago

bin/sanity-check-website: add script to sanity check the website.

Currently it ensures all blog posts and security advisories have valid
Ikiwiki 'meta date' directives, since we depend on it for
reproducibility (refs: #14767).

Will-fix: #12726

Revision 71e27f1d (diff)
Added by anonym 2 months ago

build-website: sanity check the website source before building it

... and make the check fatal unless `--non-fatal-sanity-check` is
passed to `build-webiste`.

Will-fix: #12726

Revision 2f9f0581 (diff)
Added by anonym about 2 months ago

Revert "Adding templates from latest ikiwiki version"

This reverts commit 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0.

Will-fix: #12726

Revision 33f6a5a4 (diff)
Added by anonym about 2 months ago

Vagrant build box: install ruby, needed for building the wiki.

... since the introduction of bin/sanity-check-website.

Refs: #12726

Revision ca6a9512 (diff)
Added by anonym about 2 months ago

Adapt wiki templates so 'meta date' is RFC 2822-compliant.

Refs: #12726

Revision 7e369a3a (diff)
Added by intrigeri about 2 months ago

Document the need for ruby when building the website (refs: #12726).

Revision 69149e89
Added by intrigeri about 2 months ago

Merge branch 'doc/12726-sanity-check-website' (Closes: #12726).

History

#1 Updated by intrigeri 6 months ago

Two comments:

  1. IIRC we've removed this info as it makes the ISO unreproducible. We could probably fix that by always explicitly setting a [[!meta date=]] directive.
  2. The release date is displayed in the "Install Tails x.y" block already, that looks very prominent to me.
  3. I don't know why the release date is so important in itself. Just curious: who needs this info, and why?

#2 Updated by goupille 6 months ago

intrigeri wrote:

  1. IIRC we've removed this info as it makes the ISO unreproducible. We could probably fix that by always explicitly setting a [[!meta date=]] directive.

I didn't know why it had been removed, and I can't say I understand your proposal :)

  1. The release date is displayed in the "Install Tails x.y" block already, that looks very prominent to me.

it displays only the current release.

  1. I don't know why the release date is so important in itself. Just curious: who needs this info, and why?

For a reason I can't remember, I needed to know when tails 3.0~rc1 was released, and ended up scrolling @tails_live account on Twitter... I thought that that it was weird not having this kind of information on Tails website, like, in the body of the release note, that's all...

#3 Updated by u 6 months ago

  • Assignee changed from sajolida to intrigeri

There are also publication dates in the RSS feed: https://tails.boum.org/news/index.en.rss

I agree with Goupille that having such a date is common practice on websites.
It helps to see if/that a project is still active and when some news were posted, not only the release.

I'd say that if it is easy to add the !meta date= directive, we should do that, otherwise, leave it as is. What do you think, intrigeri?

#4 Updated by intrigeri 6 months ago

  • Assignee changed from intrigeri to anonym

OK, perhaps. That's part of the reproducible builds work then (as we removed this info to help make the build reproducible).

#5 Updated by intrigeri 6 months ago

  • Parent task set to #5630

#6 Updated by sajolida 6 months ago

As someone who writes a lot of our blog posts, I think it would be good to always have a [[!meta date= and use it as the displayed date of the post.

And yes, it's a common good practice to have date on blog post and news in general.

#7 Updated by intrigeri 5 months ago

  • Target version set to Tails_3.2

(Let's be consistent with the "Deliverable for" value.)

#8 Updated by intrigeri 3 months ago

  • Target version changed from Tails_3.2 to Tails_3.3

104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 caused this regression by removing the <TMPL_VAR CTIME> bits in the ikiwiki templates we use. This change has been live since 2.12 (2017-04-19).

To fix that we need to:

  1. ensure all blog posts have a well-formed and correct [[!meta date="…"]]
  2. add a quick'n'dirty grep check in build-website that ensures all blog posts have a well-formed and correct [[!meta date="…"]] (I really don't want us to notice that the website broke ISO reproducibility during the release process, when we merge the master branch)
  3. revert the aforementioned commit
  4. find out what this breaks in terms of reproducibility, why, and fix it

I think this is indeed part of our SponsorT 2016 work. But it can be solved on the master branch so let's handle this later this month, after 3.2 is released.

#9 Updated by intrigeri 3 months ago

  • Priority changed from Normal to Elevated

(End of the contract is Nov 1st)

#10 Updated by intrigeri 2 months ago

  • Related to Bug #14767: ikiwiki does not order news items deterministically added

#11 Updated by intrigeri 2 months ago

intrigeri wrote:

To fix that we need to:

  1. ensure all blog posts have a well-formed and correct [[!meta date="…"]]
  2. add a quick'n'dirty grep check in build-website that ensures all blog posts have a well-formed and correct [[!meta date="…"]] (I really don't want us to notice that the website broke ISO reproducibility during the release process, when we merge the master branch)

Including translations, see e.g. #14767 for a real-life case where a tiny, hard to spot mistake made by a translator made Tails 3.2 not build reproducibly.

#12 Updated by anonym 2 months ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA
  • Feature Branch set to doc/12726-sanity-check-website

intrigeri wrote:

104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 caused this regression by removing the <TMPL_VAR CTIME> bits in the ikiwiki templates we use. This change has been live since 2.12 (2017-04-19).

To fix that we need to:

  1. ensure all blog posts have a well-formed and correct [[!meta date="…"]]

I went with this definition of "well-formed and correct" for blog posts source files (.mdwn, .html):

  • Must have exactly one line matching: /[[!meta\s+date="(.*)"]]/.
  • The date must be strictly rfc2822-compliant.

And this for the PO files:

  • Must have exactly one line matching: /msgid "[[!meta\s+date=\"(.*)\"]]\n"/.
  • ...which must be immediately followed by a line that:
    • is exactly msgstr "", or
    • matches /msgstr "[[!meta\s+date=\"([^"]*)\"]]\n"/.
  • The date in the msgstr (unless empty) must be strictly rfc2822-compliant.

Sounds good?

I optimistically implemented this in bin/sanity-check-website, pushed to the feature branch. What do you think?

(I wrote it in Ruby to encourage sajolida to extend it, if there's other things to sanity check.)

  1. add a quick'n'dirty grep check in build-website that ensures all blog posts have a well-formed and correct [[!meta date="…"]] (I really don't want us to notice that the website broke ISO reproducibility during the release process, when we merge the master branch)

Agreed. And this check should be fatal by default, but possible to opt-out from by passing an option, e.g. for contributors if they happen to want to build the wiki after we e.g. repeat #14767 by mistake and they don't feel it is their responsibility to fix it just to get the wiki building. Implemented in the feature branch.

  1. revert the aforementioned commit
  2. find out what this breaks in terms of reproducibility, why, and fix it

"this" = "the aforementioned commit"? If so, why? As long as sanity-check-website is happy => 'meta date' directives are ok, then we should be good, no?

#13 Updated by intrigeri 2 months ago

I went with this definition of "well-formed and correct" for blog posts source files (.mdwn, .html):
[...]
And this for the PO files:
[...]

Sounds good?

Yes.

I'll review the code next week.

  1. revert the aforementioned commit
  2. find out what this breaks in terms of reproducibility, why, and fix it

"this" = "the aforementioned commit"?

Yes.

If so, why? As long as sanity-check-website is happy => 'meta date' directives are ok, then we should be good, no?

I guess we should be good but we'll only know for sure once this has seen broad exposure to reproducibility testing: this ticket is about fixing one problem we had worked around with 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0, but by doing so we might have incidentally fixed other reproducibility issues.

#14 Updated by intrigeri 2 months ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed
  • s/and drop them for/and drop them from/ maybe?
  • build-website uses a $p variable that's not defined. Did you mean $arg instead? Please test this code with 0, 1 and 2 args (apart of --non-fatal-sanity-check).
  • a comment would be welcome for the !!DateTime.rfc2822(date) thing
  • wrt. meta_date = m[1]: don't we have named captured groups in Ruby?
  • there's a stray space at end of line in sanity-check-website
  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

#15 Updated by anonym 2 months ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

  • s/and drop them for/and drop them from/ maybe?

Thanks!

  • build-website uses a $p variable that's not defined. Did you mean $arg instead? Please test this code with 0, 1 and 2 args (apart of --non-fatal-sanity-check).

Whoops! Fixed, and verified working as intended.

  • a comment would be welcome for the !!DateTime.rfc2822(date) thing

Isn't !!... (or not(not(...))) a commonly accepted pattern to coerce to boolean in languages with such a type? I recall seeing it used in Ruby, JavaScript and Python, at least.

  • wrt. meta_date = m[1]: don't we have named captured groups in Ruby?

Good idea! Implemented.

  • there's a stray space at end of line in sanity-check-website

Fixed!

  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

Ah, indeed!

BTW, your example is not relevant, right, since it does show="1"?

And what about !map pages=...? To me it seems just as affected as !inline.

To get an idea of how we pass pages=... (with a glob) to various Ikiwiki directives I ran:

git grep --extended-regexp '\[\[!\S+[^\]]*pages="[^"]*\*[^"]*"' -- wiki

and got:

wiki/src/blueprint.mdwn:[[!map pages="page(blueprint/*)" show=title]]
wiki/src/blueprint/ARM_platforms.mdwn:[[!map pages="blueprint/ARM_platforms/*"]]
wiki/src/blueprint/DAVE_2.mdwn:[[!map pages="blueprint/DAVE_2/assets/*"]]
wiki/src/blueprint/SponsorW.mdwn:[[!map pages="blueprint/SponsorW/*"]]
wiki/src/blueprint/automated_builds_and_tests.mdwn:[[!map pages="blueprint/automated_builds_and_tests/*"]]
wiki/src/blueprint/bootstrapping/assistant.mdwn:[[!map pages="blueprint/bootstrapping/assistant/router/3rd_iteration*" show=title]]
wiki/src/blueprint/greeter_revamp_UI/mockups.mdwn:[[!map pages="blueprint/tails-greeter:_revamp_UI/mockups/*"]]
wiki/src/blueprint/monthly_report.mdwn:[[!map pages="blueprint/monthly_report/**" show="title"]]
wiki/src/brokenlinks.mdwn:[[!brokenlinks pages="* and !recentchanges and !blueprint/* and lang(EN)"]]
wiki/src/brokenlinks.mdwn:[[!brokenlinks pages="* and !recentchanges and !blueprint/*"]]
wiki/src/contribute/build.mdwn:[[!map pages="contribute/build/*"]]
wiki/src/contribute/design.mdwn:[[!map pages="contribute/design/*"]]
wiki/src/contribute/how/mirror.mdwn:[[!map pages="torrents/files/*.torrent"]]
wiki/src/contribute/how/promote/material.mdwn:[[!map pages="contribute/how/promote/material/*"]]
wiki/src/contribute/meetings.mdwn:[[!inline pages="page(contribute/meetings/*)" template="titlepage" sort="-title" limit="0"]]
wiki/src/contribute/release_process.mdwn:[[!map pages="contribute/release_process/*"]]
wiki/src/contribute/reports.mdwn:[[!map pages="page(contribute/reports/*)" show="title"]]
wiki/src/contribute/reports/SponsorS.mdwn:[[!map pages="contribute/reports/SponsorS/*"]]
wiki/src/contribute/reports/SponsorW.mdwn:[[!map pages="contribute/reports/SponsorW/*"]]
wiki/src/contribute/working_together.mdwn:[[!map pages="contribute/working_together/*"]]
wiki/src/contribute/working_together/roles.mdwn:[[!map pages="contribute/working_together/roles/*"]]
wiki/src/doc/first_steps/upgrade/release_notes.mdwn:[[!inline pages="page(news/version_*) and currentlang()" show="1" sort="-meta(date) age -path"]]
wiki/src/doc/upgrade/error.mdwn:[[!map pages="doc/upgrade/error/* and currentlang()" show="title"]]
wiki/src/index.html:[[!inline pages="page(news/*) and !news/*/* and !news/discussion and (currentlang() or news/report_2* or news/test_*-rc? or news/test_*beta? or news/test_*alpha?)" show="2" feeds="no" archive="yes" sort="-meta(date) age -path"]]
wiki/src/index.html:[[!inline pages="page(security/*) and !security/audits and !security/audits.* and !security/audits/* and !security/*/* and !security/discussion and currentlang()" show="2" feeds="no" archive="yes" sort="-meta(date) age -path"]]
wiki/src/news.html:[[!inline pages="page(news/*) and !news/*/* and !news/discussion and (currentlang() or news/report_2* or news/test_*-rc? or news/test_*beta? or news/test_*alpha?)" show="10" sort="-meta(date) age -path"]]
wiki/src/news.html:[[!inline pages="page(news/*) and !news/*/* and !news/discussion and (currentlang() or news/report_2* or news/test_*-rc? or news/test_*beta? or news/test_*alpha?) and tagged(announce)" 
wiki/src/recentchanges.mdwn:[[!inline pages="internal(recentchanges/change_*) and !title(change to forum/*) and !*/Discussion" 
wiki/src/security.mdwn:[[!inline pages="page(security/*) and !security/audits and !security/audits.* and !security/audits/* and currentlang()" 
wiki/src/security.mdwn:[[!inline pages="page(security/*) and tagged(security/probable) and currentlang()" 
wiki/src/security.mdwn:[[!inline pages="page(security/*) and tagged(security/fixed) and (currentlang() or security/Numerous_security_holes_in_0* or security/Numerous_security_holes_in_1*)" 
wiki/src/security/audits.mdwn:[[!inline pages="security/audits/* and currentlang()" 
wiki/src/tags.mdwn:[[!map pages="tags/*"]]
wiki/src/torrents/rss.mdwn:[[!inline pages="torrents/files/*.torrent" show="30" sort="age"]]

We have:

  1. four instances of sort="-meta(date) age -path" (all of these are for news/* and security/*)
  2. one instance of sort="age": the docs say "[l]ist pages from the most recently created to the oldest", which isn't clear to me what it means (ctime? mtime, but it already has its own sorting method? same as meta(date)?)
  3. tons of instances not specifying any sorting => default to @sort="age title"
  4. one instance of sort="-title"

So, (1) is currently handled by the sanity check. For the rest, I hope our build system standardizes enough in the environment so (4) is not an issue, and for (2-3) I don't know, since it's unclear how sort="age" works. But given that we haven't seen any issues with (2), I think it reasonable for us to simply assume (2-4) aren't problems until proven otherwise, i.e. our current sanity check is good enough.

What do you think?

#16 Updated by intrigeri 2 months ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Info Needed
  • a comment would be welcome for the !!DateTime.rfc2822(date) thing

Isn't !!... (or not(not(...))) a commonly accepted pattern to coerce to boolean in languages with such a type? I recall seeing it used in Ruby, JavaScript and Python, at least.

Right. I had missed the "?" in rfc2822_date? so I was wondering what's going on. Sorry for the noise.

  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

Ah, indeed!

BTW, your example is not relevant, right, since it does show="1"?

Right… assuming the single listed page is always the same, regardless of whatever may vary in the build environment. Is it?

I'll do the next code review pass once this is clarified.

And what about !map pages=...? To me it seems just as affected as !inline.

I see some explicit sorting in the map plugin so as long as locales don't vary we should be good.

We have:

  1. four instances of sort="-meta(date) age -path" (all of these are for news/* and security/*)
  2. one instance of sort="age": the docs say "[l]ist pages from the most recently created to the oldest", which isn't clear to me what it means (ctime? mtime, but it already has its own sorting method? same as meta(date)?)
  3. tons of instances not specifying any sorting => default to @sort="age title"
  4. one instance of sort="-title"

So, (1) is currently handled by the sanity check. For the rest, I hope our build system standardizes enough in the environment so (4) is not an issue, and for (2-3) I don't know, since it's unclear how sort="age" works. But given that we haven't seen any issues with (2), I think it reasonable for us to simply assume (2-4) aren't problems until proven otherwise, i.e. our current sanity check is good enough.

OK. Note that we've standardized lots of stuff (for unrelated reasons) to sort="-meta(date) age -path" that should avoid to avoid cases like 2. For avoidance of doubt we could do the same in other places too. And we've fixed some stuff in ikiwiki already so the deterministic setting is now taken into account in more places.

#17 Updated by anonym about 2 months ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Info Needed to Ready for QA

intrigeri wrote:

  • a comment would be welcome for the !!DateTime.rfc2822(date) thing

Isn't !!... (or not(not(...))) a commonly accepted pattern to coerce to boolean in languages with such a type? I recall seeing it used in Ruby, JavaScript and Python, at least.

Right. I had missed the "?" in rfc2822_date? so I was wondering what's going on. Sorry for the noise.

  • I think all pages that have something like inline pages=".../*" may not build reproducibly unless the conditions you're checking are verified; I think there's more than blog posts and security advisories in the ISO, see e.g. wiki/src/doc/first_steps/upgrade/release_notes.mdwn.

Ah, indeed!

BTW, your example is not relevant, right, since it does show="1"?

Right… assuming the single listed page is always the same, regardless of whatever may vary in the build environment. Is it?

Ah, indeed. I guess I was wrong; I imagine that e.g. #14767 would still have occurred with show="1" since the first item was affected. But it is odd that your example wasn't in the diffoscope report we have to #14767.

Any way, your example uses sort="-meta(date) age -path" on news/version_*, which we sanity check the meta date for, so we're good for that reason.

I'll do the next code review pass once this is clarified.

Please, go ahead!

And what about !map pages=...? To me it seems just as affected as !inline.

I see some explicit sorting in the map plugin so as long as locales don't vary we should be good.

Good!

We have:

  1. four instances of sort="-meta(date) age -path" (all of these are for news/* and security/*)
  2. one instance of sort="age": the docs say "[l]ist pages from the most recently created to the oldest", which isn't clear to me what it means (ctime? mtime, but it already has its own sorting method? same as meta(date)?)
  3. tons of instances not specifying any sorting => default to @sort="age title"
  4. one instance of sort="-title"

So, (1) is currently handled by the sanity check. For the rest, I hope our build system standardizes enough in the environment so (4) is not an issue, and for (2-3) I don't know, since it's unclear how sort="age" works. But given that we haven't seen any issues with (2), I think it reasonable for us to simply assume (2-4) aren't problems until proven otherwise, i.e. our current sanity check is good enough.

OK. Note that we've standardized lots of stuff (for unrelated reasons) to sort="-meta(date) age -path" that should avoid to avoid cases like 2. For avoidance of doubt we could do the same in other places too. And we've fixed some stuff in ikiwiki already so the deterministic setting is now taken into account in more places.

Good! So, in summary, the only thing we want to sanity check at the moment is: when !inline:ing pages with sort="meta(date)" (+ any other sorting), ensure the included pages has a valid meta date directive. And we simplify this by assuming the only such !inline targets are in news and security.

I hope we feel that that simplification is good enough, otherwise we'd have to reimplement ikiwikis pages=... parsing. :)

#18 Updated by intrigeri about 2 months ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed

Any way, your example uses sort="-meta(date) age -path" on news/version_*, which we sanity check the meta date for, so we're good for that reason.

Right :)

I'll do the next code review pass once this is clarified.

Please, go ahead!

Done, code review passes!

I've run ./bin/sanity-check-website and it raised 370 problems such as:

wiki/src/news/mediapart.mdwn: 'meta date' directive contains non-rfc2822 date: Thu Dec 15 12:34:56 2016

Is this expected? What's your plan about this?

So, in summary, the only thing we want to sanity check at the moment is: when !inline:ing pages with sort="meta(date)" (+ any other sorting), ensure the included pages has a valid meta date directive. And we simplify this by assuming the only such !inline targets are in news and security.

Let me extend this a bit. There are two potential problems here:

  • non-deterministic inlined posts' date: any inline can be affected as long as it uses a template that displays the date; this is the problem that made us do 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 initially; your validation code addresses that… as long as it covers all affected inline directives; I think it does.
  • non-deterministic sorting of inlined posts: earlier you've been assuming that this isn't a problem until proven otherwise; I checked the code and sort="age" uses ctime, which is overridden by the git plugin to the date when the page was first added to the Git repo, which is deterministic; but that's not enough because it happens that a bunch of pages are added in the same commit (e.g. the commits that create PO files for new pages), so when no sort= is specified, the default sort="age title" will fallback to sort="title"; we have no guarantees that page titles are unique (e.g. 1. all translated versions of a page share the same title unless there's a meta title= directive that's translated in all languages; I believe we've never seen this problem by pure chance, because we never (?) inline pages in multiple languages; 2. if we inline pages from multiple directories, it could be that there's a page called the same in several of them; here again I think we're lucky we currently have no such case in wiki/src but it might change in the future). I think the only way to be 100% sure the sorting of inlined pages is deterministic is to add a sorting criteria that we know will sort on unique values, i.e. path ⇒ to be on the safe side, all inline directives that have no explicit sort=, or a sort= that sorts on values that are not always unique (e.g. age and/or title) should be modified to use something like sort="-meta(date) age -path". Of course, we don't care about pages (with inline directives) that we don't include in the ISO, e.g. we exclude usr/share/doc/tails/website/torrents/rss.html from the SquashFS (which explains why we've not seen any sorting problem there recently, see 149515aa7367afe22d9c3c40ae56ea5fd1e39669). At first glance only wiki/src/security/audits.mdwn needs to be fixed as proposed but please double-check. I know this is "just to be safe", doesn't fix any current issue, and is even somewhat off-topic on this ticket, but the fix is a one-liner so perhaps you won't mind doing it?

Also, your branch does not revert 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 so merging it won't be enough to close this ticket.

I've merged current master into the topic branch so it builds again on Jenkins (so far it has always FTBFS before the code you added was run, and I'd like to see it run there before we merge this :)

Finally, the result of the sanity check result seems to be ignored when building an ISO (https://jenkins.tails.boum.org/job/build_Tails_ISO_doc-12726-sanity-check-website/8/consoleFull) which is not good enough IMO: it's just too easy to miss it and release a Tails that's not reproducible.

#19 Updated by anonym about 2 months ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

I've run ./bin/sanity-check-website and it raised 370 problems such as:

[...]

Is this expected? What's your plan about this?

The plan was to fix all these in some (semi-)automated way once we had agreed, which we now have, so I've now done it.

So, in summary, the only thing we want to sanity check at the moment is: when !inline:ing pages with sort="meta(date)" (+ any other sorting), ensure the included pages has a valid meta date directive. And we simplify this by assuming the only such !inline targets are in news and security.

Let me extend this a bit. There are two potential problems here:

  • non-deterministic inlined posts' date: any inline can be affected as long as it uses a template that displays the date; this is the problem that made us do 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 initially; your validation code addresses that… as long as it covers all affected inline directives; I think it does.
  • non-deterministic sorting of inlined posts: earlier you've been assuming that this isn't a problem until proven otherwise; I checked the code and sort="age" uses ctime, which is overridden by the git plugin to the date when the page was first added to the Git repo, which is deterministic; but that's not enough because it happens that a bunch of pages are added in the same commit (e.g. the commits that create PO files for new pages), so when no sort= is specified, the default sort="age title" will fallback to sort="title"; we have no guarantees that page titles are unique (e.g. 1. all translated versions of a page share the same title unless there's a meta title= directive that's translated in all languages; I believe we've never seen this problem by pure chance, because we never (?) inline pages in multiple languages; 2. if we inline pages from multiple directories, it could be that there's a page called the same in several of them; here again I think we're lucky we currently have no such case in wiki/src but it might change in the future). I think the only way to be 100% sure the sorting of inlined pages is deterministic is to add a sorting criteria that we know will sort on unique values, i.e. path ⇒ to be on the safe side, all inline directives that have no explicit sort=, or a sort= that sorts on values that are not always unique (e.g. age and/or title) should be modified to use something like sort="-meta(date) age -path". Of course, we don't care about pages (with inline directives) that we don't include in the ISO, e.g. we exclude usr/share/doc/tails/website/torrents/rss.html from the SquashFS (which explains why we've not seen any sorting problem there recently, see 149515aa7367afe22d9c3c40ae56ea5fd1e39669). At first glance only wiki/src/security/audits.mdwn needs to be fixed as proposed but please double-check. I know this is "just to be safe", doesn't fix any current issue, and is even somewhat off-topic on this ticket, but the fix is a one-liner so perhaps you won't mind doing it?

I've fixed wiki/src/security/audits.mdwn.

Also, your branch does not revert 104d7f89ffd0a83962eb6eb891e1c2d390cf62b0 so merging it won't be enough to close this ticket.

Reverted.

I've merged current master into the topic branch so it builds again on Jenkins (so far it has always FTBFS before the code you added was run, and I'd like to see it run there before we merge this :)

Thanks!

Finally, the result of the sanity check result seems to be ignored when building an ISO (https://jenkins.tails.boum.org/job/build_Tails_ISO_doc-12726-sanity-check-website/8/consoleFull) which is not good enough IMO: it's just too easy to miss it and release a Tails that's not reproducible.

Apparently auto/{build,clean,config} didn't set -e, now fixed.

#20 Updated by intrigeri about 2 months ago

  • Assignee changed from intrigeri to anonym
  • % Done changed from 50 to 70
  • QA Check changed from Ready for QA to Dev Needed

This branch FTBFS on Jenkins: apparently the new script is run in the Vagrant VM which has no Ruby.

Please adjust templates (at least wiki/src/contribute/how/documentation/release_notes/template.mdwn and wiki/src/blueprint/monthly_report.mdwn) that have example dates which work just fine with ikiwiki, but that the new script rejects for some reason (FYI the ikiwiki meta plugin uses Date::Parse's str2time which is much more tolerant), otherwise we are bound to introduce such problems again.

Other than this, everything looks good, congrats!

#21 Updated by anonym about 2 months ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

This branch FTBFS on Jenkins: apparently the new script is run in the Vagrant VM which has no Ruby.

Gah... I knew this, since I temporarily installed it in my builder to test it. Fixed now. But it will any way FTBFS given the state of master's debian/changelog. But the wiki builds for me locally, at least.

Please adjust templates (at least wiki/src/contribute/how/documentation/release_notes/template.mdwn and wiki/src/blueprint/monthly_report.mdwn) that have example dates which work just fine with ikiwiki

Fixed!

but that the new script rejects for some reason (FYI the ikiwiki meta plugin uses Date::Parse's str2time which is much more tolerant), otherwise we are bound to introduce such problems again.

Ruby has similar functionality via DateTime.parse, which was my first pick, but then I saw this in its API docs: "This method does not function as a validator" (indeed, it will attempt to ignore invalid parts, with results that are hard to anticipate; unlike perl's str2time, a date is always produced). This got me thinking that we really want to be strict here to not repeat something similar to #14767 (assuming an invalid meta date is the same as no meta date). And, in fact, I saw a few strange dates that weren't caught by my automated conversion, like "Sun Dec 13 1 01:02:03 2015" (from wiki/src/security/Numerous_security_holes_in_1.7.mdwn), where the stray "1" apparently becomes "+01" time zone offset via perl's str2time.

#22 Updated by intrigeri about 2 months ago

Merged with two commits on top:

Please review them :)

#23 Updated by intrigeri about 2 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 70 to 100

#24 Updated by intrigeri about 2 months ago

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#25 Updated by intrigeri about 2 months ago

Also, I suggest merging master into stable & devel so this sees more exposure to testing.

#26 Updated by intrigeri about 1 month ago

  • Related to Bug #14933: stable branch is not reproducible: differences in some .fa.html website files added

Also available in: Atom PDF