Applying Manifest Code
Difficulty: Advanced
Time: Approximately 20 minutes
In this exercise you will further explore Bolt Plans by using the apply
keyword to leverage existing content from the Puppet Forge.
You can read more about using bolt apply
in Masterless Workflows in a Blog Post written by Bolt developer Michael Smith.
You will deploy two web servers and a load balancer to distribute the traffic evenly between them with the following steps:
- Build a Project Specific Configuration Using a
Boltdir
- Build an Inventory to Organize Provisioned Targets
- Download Useful Module Content From Puppet Forge
- Write a Puppet Class to Abstract Nginx Web Server Configuration
- Write a Bolt Plan to
apply
Puppet Code and Orchestrate Deployment
Prerequisites
For the following exercises you should have Bolt, Docker, and docker-compose installed. The following guides will help:
Build a Boltdir
Create a new project directory with a Boltdir
. If you are using the files included with this lesson your project directory will end up looking like:
lesson11/
├── Boltdir
│ ├── inventory.yaml
│ ├── Puppetfile
│ └── site
│ └── profiles
│ ├── manifests
│ | └── server.pp
│ └── plans
│ └── nginx_install.pp
└── docker-compose.yml
Acquire targets
This lesson requires three targets. Save the following file in your project directory as docker-compose.yml
.
version: '3'
services:
lb:
image: rastasheep/ubuntu-sshd:16.04
ports:
- "20022:22"
- "20080:80"
server_1:
image: rastasheep/ubuntu-sshd:16.04
ports:
- "20023:22"
server_2:
image: rastasheep/ubuntu-sshd:16.04
ports:
- "20024:22"
Targets can be obtained with the docker-compose up -d
command.
You can verify targets are created with docker ps
8b7f33d8fde4 lab_target "/usr/sbin/sshd -D" About an hour ago Up About an hour 0.0.0.0:20023->22/tcp 11applymanifestcode_server_1_1
90f6abe8dfc8 lab_target "/usr/sbin/sshd -D" About an hour ago Up About an hour 0.0.0.0:20024->22/tcp 11applymanifestcode_server_2_1
26c47f8c4bad lab_target "/usr/sbin/sshd -D" About an hour ago Up About an hour 0.0.0.0:20022->22/tcp, 0.0.0.0:20080->80/tcp lb
Inventory
Build an inventory to organize provisioned targets. This will be the first configuration file in our new project specific Boltdir
.
Note: Example outputs in the lab are for targets provisioned with Docker.
If you provisioned your targets with the docker-compose file provided with this exercise save the following in Boltdir/inventory.yaml
.
---
version: 2
groups:
- name: lb
targets:
- 0.0.0.0:20022
- name: servers
targets:
- 127.0.0.1:20023
- localhost:20024
config:
ssh:
host-key-check: false
password: root
user: root
Make sure your inventory is configured correctly and you can connect to all targets. Run from within the project directory:
bolt command run 'echo hi' -n all
Expected output
Started on 0.0.0.0...
Started on 127.0.0.1...
Started on localhost...
Finished on 0.0.0.0:
STDOUT:
hi
Finished on 127.0.0.1:
STDOUT:
hi
Finished on localhost:
STDOUT:
hi
Successful on 3 targets: 0.0.0.0:20022,127.0.0.1:20023,localhost:20024
Ran on 3 targets in 0.2 sec
Module Content
In order to install module content from the forge Bolt uses a Puppetfile
. See Puppetfile Documentation for more information.
Save the following Puppetfile
that describes the Puppet Forge content to be installed in the project Boltdir
.
forge 'http://forge.puppetlabs.com'
mod 'puppetlabs-stdlib', '4.25.1'
mod 'puppetlabs-concat', '4.2.1'
mod 'puppetlabs-haproxy', '2.1.0'
From within the project directory install the Forge content with the following Bolt command:
bolt puppetfile install
Confirm that a modules
directory has been created in the project Boltdir
.
Write profile
Module
Now that you have downloaded existing modules it is time to write your own module content. Custom module content not managed by the project Puppetfile
belongs in a site
directory in the Boltdir
. After creating a Boltdir/site
directory create a new directory called profiles
. The profiles
module will be our own custom module.
Note:
site
andsite-modules
directories can both be used for custom module content.
Abstracting the Server Setup
Start by abstracting the Nginx setup by writing a Puppet Class. Puppet code belongs in a subdirectory of our module called manifests
. Save the following class definition in Boltdir/site/profiles/manifests/server.pp
.
If you are new to Puppet writing puppet code check out these learning resources. The Learning VM is especially helpful for getting up to speed with Puppet.
class profiles::server(String $site_content) {
if($facts['os']['family'] == 'redhat') {
package { 'epel-release':
ensure => present,
before => Package['nginx'],
}
$html_dir = '/usr/share/nginx/html'
} else {
$html_dir = '/var/www/html'
}
package { 'nginx':
ensure => present,
}
file { "${html_dir}/index.html":
content => $site_content,
ensure => file,
}
service { 'nginx':
ensure => 'running',
enable => 'true',
require => Package['nginx'],
}
}
Note: Vox Pupuli maintains an nginx module that you could swap in for our simple server class to manage more complex nginx configuration.
Now we will write a Plan to utilize the server class.
Apply the Puppet Code
As we have seen in the lab, plan code belongs in the plans
subdirectory. Save the following to Boltdir/site/profiles/plans/nginx_install.pp
.
Take note of the following features of the plan:
- This plan has three parameters, the server targets, the load balancer targets and a string to be statically served by our load balanced Nginx servers.
- Notice the
apply_prep
function call.apply_prep
is used to install packages needed by apply on remote targets as well as to gather facts about the targets. - The first apply block configures the Nginx servers. The site content is defined by default to be “Hello from [target name]” where target name is a fact gathered by
apply_prep
. Thesite_content
parameter can be configured in the bolt plan invocation. - The second apply block uses information about the Nginx servers to configure a load balancer to direct traffic between the two servers.
plan profiles::nginx_install(
TargetSpec $servers,
TargetSpec $lb,
String $site_content = 'hello!',
) {
if get_targets($lb).size != 1 {
fail("Must specify a single load balancer, not ${lb}")
}
# Ensure puppet tools are installed and gather facts for the apply
apply_prep([$servers, $lb])
apply($servers) {
class { 'profiles::server':
site_content => "${site_content} from ${$trusted['certname']}\n",
}
}
apply($lb) {
include haproxy
haproxy::listen { 'nginx':
collect_exported => false,
ipaddress => $facts['ipaddress'],
ports => '80',
}
$targets = get_targets($servers)
$targets.each |Integer $index, Target $target| {
haproxy::balancermember { "lb_${$index}":
listening_service => 'nginx',
server_names => $target.host,
ipaddresses => $target.facts['ipaddress'],
ports => '80',
options => 'check',
}
}
}
}
Verify the nginx_install
plan is available to run using bolt plan show
. You should see an output similar to:
aggregate::count
aggregate::targets
canary
facts
facts::info
profiles::nginx_install
puppetdb_fact
Now you are ready to execute the plan.
bolt plan run profiles::nginx_install servers=servers lb=lb
Expected output
Starting: plan profiles::nginx_install
Starting: install puppet and gather facts on 127.0.0.1:20023, localhost:20024, 0.0.0.0:20022
Finished: install puppet and gather facts with 0 failures in 28.45 sec
Starting: apply catalog on 127.0.0.1:20023, localhost:20024
Finished: apply catalog with 0 failures in 32.17 sec
Starting: apply catalog on 0.0.0.0:20022
0.0.0.0:20022: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.
Finished: apply catalog with 0 failures in 10.01 sec
Finished: plan profiles::nginx_install in 1 min, 11 sec
In order to verify the deployment is operating as expected use the following curl
commands to see the load balancer delegating to the different web servers.
curl http://0.0.0.0:20080/
We expect the result to vary between based on the load balancer
hello! from localhost
and
hello! from 127.0.0.1
Note: You can also navigate to
http://0.0.0.0:20080/
in a web browser. Just be aware that your browser will likely cache the result and therefore you may not see the oscillation between the two servers behind the load balancer.
Next steps
Congratulations! You should now have a basic understanding of Bolt and Bolt Tasks. Here are a few ideas for what to do next:
- Get reusable tasks and plans from the Task Modules Repo
- Search Puppet Forge for Tasks
- Start writing Tasks for one of your existing Puppet modules
- Head over to the Puppet Slack and talk to the Bolt developers and other users
- Try out the Puppet Development Kit (docs) which has a few features to make authoring tasks even easier