Project

General

Profile

Feature #11834

Migrate our infrastructure to Puppet 4

Added by intrigeri over 1 year ago. Updated 26 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
09/24/2016
Due date:
% Done:

22%

QA Check:
Feature Branch:
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:

Description

On Stretch we won't have Puppet 3, so we'll have to migrate to Puppet 4.

Lots of interesting discussions on this topic have happened on https://bugs.debian.org/826551, including how to migrate an existing storedconfig database to PuppetDB. Since then, puppetdb made it into Debian.

See upstream's upgrading checklist for details.


Subtasks

Feature #11833: Make our Puppet code compatible with the "future" parserConfirmedintrigeri

Feature #11835: Upgrade Puppet master and clients to 3.8Resolved

Feature #11836: Stop stringifying Puppet factsIn Progressintrigeri

Feature #11837: Upgrade Puppet master to Puppet 4Confirmedintrigeri

Feature #11838: Upgrade Puppet agents to Puppet 4Confirmedintrigeri


Related issues

Related to Tails - Bug #11543: bitcoin.lizard cannot connect to our Puppet master Resolved 06/23/2016
Blocks Tails - Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure) Confirmed 06/30/2017

History

#1 Updated by intrigeri over 1 year ago

  • Blocked by Feature #11833: Make our Puppet code compatible with the "future" parser added

#2 Updated by intrigeri over 1 year ago

  • Blocked by Feature #11835: Upgrade Puppet master and clients to 3.8 added

#3 Updated by intrigeri over 1 year ago

  • Related to Bug #11543: bitcoin.lizard cannot connect to our Puppet master added

#4 Updated by intrigeri over 1 year ago

  • Blocked by deleted (Feature #11833: Make our Puppet code compatible with the "future" parser)

#5 Updated by intrigeri over 1 year ago

  • Blocked by deleted (Feature #11835: Upgrade Puppet master and clients to 3.8)

#6 Updated by intrigeri over 1 year ago

  • Description updated (diff)

#7 Updated by intrigeri 10 months ago

  • Assignee set to intrigeri

#8 Updated by intrigeri 10 months ago

  • Description updated (diff)

#9 Updated by intrigeri 9 months ago

  • Target version set to Tails_3.5

#10 Updated by intrigeri 8 months ago

  • Blocked by Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure) added

#11 Updated by intrigeri 8 months ago

  • Blocked by deleted (Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure))

#12 Updated by intrigeri 8 months ago

  • Blocks Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure) added

#13 Updated by intrigeri 6 months ago

  • Description updated (diff)

PuppetDB is now in Debian :)

#14 Updated by intrigeri 6 months ago

  • Description updated (diff)

#15 Updated by intrigeri 3 months ago

intrigeri wrote:

PuppetDB is now in Debian :)

I don't understand why it did not migrate to testing yet => asked the maintainer.

#16 Updated by intrigeri 3 months ago

intrigeri wrote:

intrigeri wrote:

PuppetDB is now in Debian :)

I don't understand why it did not migrate to testing yet => asked the maintainer.

It seems to be caused at least by libtika-java being RC buggy.

#17 Updated by intrigeri about 1 month ago

  • Status changed from Confirmed to In Progress

As said elsewhere last week, during the 3.5 cycle I want to plan this migration. Here's what I did in order to get a clearer idea of what this project will involve:

  • Started a discussion with the shared Puppet modules group to learn how others are handling this. Jérôme's first answer is inspiring:
    • He went ahead and "dumped a number of shared modules in favor of modules maintained by other people", which is consistent with the long-term plan of this group. I think we should do the same when feasible, instead of investing time to port these poorly maintained modules to Puppet 4: moving to modules maintained by big players in the Puppet community (puppetlabs, voxpopuli, camptocamp) might require a bigger initial investment, but it's clear it'll pay off on the long term. I'll decide on a case by case basis.
    • He's running PuppetDB from sid on Stretch (and apparently Micah does the same). Good news, this gives us a workable solution while I was really wondering what kind of ugly trade-off we would have to make :) I'll try this.
  • Migrated my own laptop (that runs sid) to Puppet 4. This machine runs a subset of the modules we use on our infra, so:

Next steps are (note to myself but also so that you folks can follow along):

  1. Install PuppetDB locally and see how some of the exported resources we use work with it.
  2. Handle #11833, for which I now have a plan (I've realized there's a small chance it magically fixes #11836).
  3. Deal with #11836, somehow (possibly while doing the next step).
  4. Set up a Puppet 4 master + PuppetDB on our production infra, feed it with our manifests + modules (using a different branch on the manifests repo, so I can point to new versions of submodules that are compatible with Puppet 4), make it manage a few non-critical systems that don't export resources used by other systems (most likely something on sib), fix problems.
  5. Press the big red button i.e. make the Puppet 4 master manage all our systems: migrating them one after the other would be ideal, but due to our heavy reliance on exported resources, I think there will need to be a few flag days — best case — or a single one — worst case).
  6. Upgrade all Puppet agents to Puppet 4.

Regarding planning, I'll probably focus on this next week but then my schedule is quite packed already in 2018Q1. Still, I think there's actually a chance that I get this done by the end of March, we'll see. In any case, I'll take note of everything that would be worth reviewing by groente, as part of his Puppet training, like I started to do above :)

#18 Updated by intrigeri about 1 month ago

  • Target version changed from Tails_3.5 to Tails_3.7

#19 Updated by intrigeri 26 days ago

intrigeri wrote:

As said elsewhere last week, during the 3.5 cycle I want to plan this migration.

Done.

Regarding planning, I'll probably focus on this next week

Did not happen, will postpone #11833 and #11836 accordingly.

but then my schedule is quite packed already in 2018Q1. Still, I think there's actually a chance that I get this done by the end of March, we'll see.

I don't think that'll be done in 2018Q1. 2018Q2 seems more realistic.

Also available in: Atom PDF