Project

General

Profile

Feature #11834

Migrate our infrastructure to Puppet 4

Added by intrigeri over 1 year ago. Updated 20 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
09/24/2016
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:

Description

On Stretch we won't have Puppet 3, so we'll have to migrate to Puppet 4.

Lots of interesting discussions on this topic have happened on https://bugs.debian.org/826551, including how to migrate an existing storedconfig database to PuppetDB. Since then, puppetdb made it into Debian.

See upstream's upgrading checklist for details.


Subtasks

Feature #11833: Make our Puppet code compatible with the "future" parserResolvedintrigeri

Feature #11835: Upgrade Puppet master and clients to 3.8Resolved

Feature #11836: Stop stringifying Puppet factsResolvedintrigeri

Feature #11837: Upgrade Puppet master to Puppet 4Resolvedgroente

Feature #11838: Upgrade Puppet agents to Puppet 4Resolvedgroente

Feature #15490: Remove MariaDB on puppet-git.lizardResolvedgroente

Feature #15492: Set up PuppetDB backupsResolvedgroente

Bug #15493: Adjust monitoring check for Puppet runs for Puppet master 4.xResolvedgroente

Bug #15555: PuppetDB is regularly unavailableResolvedgroente


Related issues

Related to Tails - Bug #11543: bitcoin.lizard cannot connect to our Puppet master Resolved 06/23/2016
Blocks Tails - Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure) Confirmed 06/30/2017

History

#1 Updated by intrigeri over 1 year ago

  • Blocked by Feature #11833: Make our Puppet code compatible with the "future" parser added

#2 Updated by intrigeri over 1 year ago

  • Blocked by Feature #11835: Upgrade Puppet master and clients to 3.8 added

#3 Updated by intrigeri over 1 year ago

  • Related to Bug #11543: bitcoin.lizard cannot connect to our Puppet master added

#4 Updated by intrigeri over 1 year ago

  • Blocked by deleted (Feature #11833: Make our Puppet code compatible with the "future" parser)

#5 Updated by intrigeri over 1 year ago

  • Blocked by deleted (Feature #11835: Upgrade Puppet master and clients to 3.8)

#6 Updated by intrigeri over 1 year ago

  • Description updated (diff)

#7 Updated by intrigeri about 1 year ago

  • Assignee set to intrigeri

#8 Updated by intrigeri about 1 year ago

  • Description updated (diff)

#9 Updated by intrigeri 12 months ago

  • Target version set to Tails_3.5

#10 Updated by intrigeri 11 months ago

  • Blocked by Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure) added

#11 Updated by intrigeri 11 months ago

  • Blocked by deleted (Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure))

#12 Updated by intrigeri 11 months ago

  • Blocks Feature #13284: Core work 2017Q2→2019Q1: Sysadmin (Adapt our infrastructure) added

#13 Updated by intrigeri 9 months ago

  • Description updated (diff)

PuppetDB is now in Debian :)

#14 Updated by intrigeri 9 months ago

  • Description updated (diff)

#15 Updated by intrigeri 6 months ago

intrigeri wrote:

PuppetDB is now in Debian :)

I don't understand why it did not migrate to testing yet => asked the maintainer.

#16 Updated by intrigeri 6 months ago

intrigeri wrote:

intrigeri wrote:

PuppetDB is now in Debian :)

I don't understand why it did not migrate to testing yet => asked the maintainer.

It seems to be caused at least by libtika-java being RC buggy.

#17 Updated by intrigeri 4 months ago

  • Status changed from Confirmed to In Progress

As said elsewhere last week, during the 3.5 cycle I want to plan this migration. Here's what I did in order to get a clearer idea of what this project will involve:

  • Started a discussion with the shared Puppet modules group to learn how others are handling this. Jérôme's first answer is inspiring:
    • He went ahead and "dumped a number of shared modules in favor of modules maintained by other people", which is consistent with the long-term plan of this group. I think we should do the same when feasible, instead of investing time to port these poorly maintained modules to Puppet 4: moving to modules maintained by big players in the Puppet community (puppetlabs, voxpopuli, camptocamp) might require a bigger initial investment, but it's clear it'll pay off on the long term. I'll decide on a case by case basis.
    • He's running PuppetDB from sid on Stretch (and apparently Micah does the same). Good news, this gives us a workable solution while I was really wondering what kind of ugly trade-off we would have to make :) I'll try this.
  • Migrated my own laptop (that runs sid) to Puppet 4. This machine runs a subset of the modules we use on our infra, so:

Next steps are (note to myself but also so that you folks can follow along):

  1. Install PuppetDB locally and see how some of the exported resources we use work with it.
  2. Handle #11833, for which I now have a plan (I've realized there's a small chance it magically fixes #11836).
  3. Deal with #11836, somehow (possibly while doing the next step).
  4. Set up a Puppet 4 master + PuppetDB on our production infra, feed it with our manifests + modules (using a different branch on the manifests repo, so I can point to new versions of submodules that are compatible with Puppet 4), make it manage a few non-critical systems that don't export resources used by other systems (most likely something on sib), fix problems.
  5. Press the big red button i.e. make the Puppet 4 master manage all our systems: migrating them one after the other would be ideal, but due to our heavy reliance on exported resources, I think there will need to be a few flag days — best case — or a single one — worst case).
  6. Upgrade all Puppet agents to Puppet 4.

Regarding planning, I'll probably focus on this next week but then my schedule is quite packed already in 2018Q1. Still, I think there's actually a chance that I get this done by the end of March, we'll see. In any case, I'll take note of everything that would be worth reviewing by groente, as part of his Puppet training, like I started to do above :)

#18 Updated by intrigeri 4 months ago

  • Target version changed from Tails_3.5 to Tails_3.7

#19 Updated by intrigeri 4 months ago

intrigeri wrote:

As said elsewhere last week, during the 3.5 cycle I want to plan this migration.

Done.

Regarding planning, I'll probably focus on this next week

Did not happen, will postpone #11833 and #11836 accordingly.

but then my schedule is quite packed already in 2018Q1. Still, I think there's actually a chance that I get this done by the end of March, we'll see.

I don't think that'll be done in 2018Q1. 2018Q2 seems more realistic.

#20 Updated by intrigeri about 2 months ago

Other modules I've upgraded for Puppet 4 compatibility:

  • sysctl: to its latest upstream release (0.0.11); wasn't strictly needed but it fixes a warning
  • sshkeys: our previous upstream was abandonned, switched to https://github.com/FrankVanDamme/puppet-sshkeys which fixes Puppet 4 compatibility among other things; I'm not upgrading to the very latest commits as they're introducing syntax that's not compatible with Puppet 3.x (although they work locally with Puppet 4.x and 5.x).

intrigeri wrote:

Next steps are (note to myself but also so that you folks can follow along):

  1. Install PuppetDB locally and see how some of the exported resources we use work with it.

Done, this works :) I've documented how I did that on #11837.

#21 Updated by intrigeri about 1 month ago

  • Assignee changed from intrigeri to groente
  • QA Check set to Ready for QA

(All subtasks are now Ready for QA :)

#22 Updated by intrigeri about 1 month ago

And while I was at it, I've taken advantage of the data type system that Puppet 4 introduced and ported puppet-tails to it:

I see 106 files changed, 563 insertions(+), 1011 deletions(-) in puppet-tails, pretty good!

#23 Updated by groente 20 days ago

  • Status changed from In Progress to Resolved
  • QA Check changed from Ready for QA to Pass

\o/

Also available in: Atom PDF