Applying Manifest Code

Difficulty: Advanced

Time: Approximately 20 minutes

In this exercise you will further explore Bolt Plans by using the apply keyword to leverage existing content from the Puppet Forge.

You can read more about using bolt apply in Masterless Workflows in a Blog Post written by Bolt developer Michael Smith.

You will deploy two web servers and a load balancer to distribute the traffic evenly between them with the following steps:

Prerequisites

For the following exercises you should have Bolt, Docker, and docker-compose installed. The following guides will help:

Build a Boltdir

Create a new project directory with a Boltdir. If you are using the files included with this lesson your project directory will end up looking like:

lesson11/
├── Boltdir
│   ├── inventory.yaml
│   ├── Puppetfile
│   └── site
│       └── profiles
│           ├── manifests
│           |   └── server.pp
│           └── plans
│               └── nginx_install.pp
└── docker-compose.yml

Acquire targets

This lesson requires three targets. Save the following file in your project directory as docker-compose.yml.

version: '3'
services:
  lb:
    image: rastasheep/ubuntu-sshd:16.04
    ports:
      - "20022:22"
      - "20080:80"
  server_1:
    image: rastasheep/ubuntu-sshd:16.04
    ports:
      - "20023:22"
  server_2:
    image: rastasheep/ubuntu-sshd:16.04
    ports:
      - "20024:22"

Targets can be obtained with the docker-compose up -d command.

You can verify targets are created with docker ps

8b7f33d8fde4        lab_target            "/usr/sbin/sshd -D"   About an hour ago   Up About an hour    0.0.0.0:20023->22/tcp                          11applymanifestcode_server_1_1
90f6abe8dfc8        lab_target            "/usr/sbin/sshd -D"   About an hour ago   Up About an hour    0.0.0.0:20024->22/tcp                          11applymanifestcode_server_2_1
26c47f8c4bad        lab_target            "/usr/sbin/sshd -D"   About an hour ago   Up About an hour    0.0.0.0:20022->22/tcp, 0.0.0.0:20080->80/tcp   lb

Inventory

Build an inventory to organize provisioned targets. This will be the first configuration file in our new project specific Boltdir.

Note: Example outputs in the lab are for targets provisioned with Docker.

If you provisioned your targets with the docker-compose file provided with this exercise save the following in Boltdir/inventory.yaml.

---
version: 2
groups:
  - name: lb
    targets:
      - 0.0.0.0:20022
  - name: servers
    targets:
      - 127.0.0.1:20023
      - localhost:20024
config:
  ssh:
    host-key-check: false
    password: root
    user: root

Make sure your inventory is configured correctly and you can connect to all targets. Run from within the project directory:

bolt command run 'echo hi' -n all

Expected output

Started on 0.0.0.0...
Started on 127.0.0.1...
Started on localhost...
Finished on 0.0.0.0:
  STDOUT:
    hi
Finished on 127.0.0.1:
  STDOUT:
    hi
Finished on localhost:
  STDOUT:
    hi
Successful on 3 targets: 0.0.0.0:20022,127.0.0.1:20023,localhost:20024
Ran on 3 targets in 0.2 sec

Module Content

In order to install module content from the forge Bolt uses a Puppetfile. See Puppetfile Documentation for more information.

Save the following Puppetfile that describes the Puppet Forge content to be installed in the project Boltdir.

forge 'http://forge.puppetlabs.com'
mod 'puppetlabs-stdlib', '4.25.1'
mod 'puppetlabs-concat', '4.2.1'
mod 'puppetlabs-haproxy', '2.1.0'

From within the project directory install the Forge content with the following Bolt command:

bolt puppetfile install

Confirm that a modules directory has been created in the project Boltdir.

Write profile Module

Now that you have downloaded existing modules it is time to write your own module content. Custom module content not managed by the project Puppetfile belongs in a site directory in the Boltdir. After creating a Boltdir/site directory create a new directory called profiles. The profiles module will be our own custom module.

Note: site and site-modules directories can both be used for custom module content.

Abstracting the Server Setup

Start by abstracting the Nginx setup by writing a Puppet Class. Puppet code belongs in a subdirectory of our module called manifests. Save the following class definition in Boltdir/site/profiles/manifests/server.pp.

If you are new to Puppet writing puppet code check out these learning resources. The Learning VM is especially helpful for getting up to speed with Puppet.

class profiles::server(String $site_content) {
  if($facts['os']['family'] == 'redhat') {
    package { 'epel-release':
      ensure => present,
      before => Package['nginx'],
    }
    $html_dir = '/usr/share/nginx/html'
  } else {
    $html_dir = '/var/www/html'
  }

  package { 'nginx':
    ensure => present,
  }

  file { "${html_dir}/index.html":
    content => $site_content,
    ensure  => file,
  }

  service { 'nginx':
    ensure  => 'running',
    enable  => 'true',
    require => Package['nginx'],
  }
}

Note: Vox Pupuli maintains an nginx module that you could swap in for our simple server class to manage more complex nginx configuration.

Now we will write a Plan to utilize the server class.

Apply the Puppet Code

As we have seen in the lab, plan code belongs in the plans subdirectory. Save the following to Boltdir/site/profiles/plans/nginx_install.pp.

Take note of the following features of the plan:

  1. This plan has three parameters, the server targets, the load balancer targets and a string to be statically served by our load balanced Nginx servers.
  2. Notice the apply_prep function call. apply_prep is used to install packages needed by apply on remote targets as well as to gather facts about the targets.
  3. The first apply block configures the Nginx servers. The site content is defined by default to be “Hello from [target name]” where target name is a fact gathered by apply_prep. The site_content parameter can be configured in the bolt plan invocation.
  4. The second apply block uses information about the Nginx servers to configure a load balancer to direct traffic between the two servers.
plan profiles::nginx_install(
  TargetSpec $servers,
  TargetSpec $lb,
  String $site_content = 'hello!',
) {
  if get_targets($lb).size != 1 {
    fail("Must specify a single load balancer, not ${lb}")
  }
  # Ensure puppet tools are installed and gather facts for the apply
  apply_prep([$servers, $lb])

  apply($servers) {
    class { 'profiles::server':
      site_content => "${site_content} from ${$trusted['certname']}\n",
    }
  }

  apply($lb) {
    include haproxy
    haproxy::listen { 'nginx':
      collect_exported => false,
      ipaddress        => $facts['ipaddress'],
      ports            => '80',
    }

    $targets = get_targets($servers)
    $targets.each |Integer $index, Target $target| {
      haproxy::balancermember { "lb_${$index}":
        listening_service => 'nginx',
        server_names      => $target.host,
        ipaddresses       => $target.facts['ipaddress'],
        ports             => '80',
        options           => 'check',
      }
    }
  }
}

Verify the nginx_install plan is available to run using bolt plan show. You should see an output similar to:

aggregate::count
aggregate::targets
canary
facts
facts::info
profiles::nginx_install
puppetdb_fact

Now you are ready to execute the plan.

bolt plan run profiles::nginx_install servers=servers lb=lb

Expected output

Starting: plan profiles::nginx_install
Starting: install puppet and gather facts on 127.0.0.1:20023, localhost:20024, 0.0.0.0:20022
Finished: install puppet and gather facts with 0 failures in 28.45 sec
Starting: apply catalog on 127.0.0.1:20023, localhost:20024
Finished: apply catalog with 0 failures in 32.17 sec
Starting: apply catalog on 0.0.0.0:20022
0.0.0.0:20022: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.
Finished: apply catalog with 0 failures in 10.01 sec
Finished: plan profiles::nginx_install in 1 min, 11 sec

In order to verify the deployment is operating as expected use the following curl commands to see the load balancer delegating to the different web servers.

curl http://0.0.0.0:20080/

We expect the result to vary between based on the load balancer

hello! from localhost

and

hello! from 127.0.0.1

Note: You can also navigate to http://0.0.0.0:20080/ in a web browser. Just be aware that your browser will likely cache the result and therefore you may not see the oscillation between the two servers behind the load balancer.

Next steps

Congratulations! You should now have a basic understanding of Bolt and Bolt Tasks. Here are a few ideas for what to do next: