Puppet for Cloud and Virtualization
The boom of virtualization and cloud computing has boosted the necessity and the diffusion of software management tools that can accelerate deployment and the scale of new systems.
Puppet can be used to manage various aspects related to cloud computing and virtualization:
- It can configure virtualization and cloud solutions, such as VMware, OpenStack, and Eucalyptus. This is done with different modules for different operating systems.
- It can provide commands to provision instances on different clouds, such as Amazon AWS, Google Compute Engine, and VMware. This is done with the cloud provisioner module and other community modules.
VMware
VMware is a major investor in Puppet Labs and technological collaborations have been done at various levels. Let’s see the most interesting projects
VM provisioning on vCenter and vSphere
Puppet till version 3.8, provides support to manage virtual machine instances using vSphere and vCenter. This is done via a face, which provides the node_vmware Puppet sub-command. Once the local environment is configured, we can create a new VM based on an existing template with the following command:
puppet node_vmware create — name=myserver — template=”/Datacenters/ Solutions/vm/master_template”
We can start and stop an existing VM with these commands: puppet
node_vmware start /Datacenters/Solutions/vm/myserver
puppet node_vmware stop /Datacenters/Solutions/vm/myserver
vCenter configuration
Puppet Labs and VMware products also interoperate on the setup and configuration of vCenter: the VMware application that allows the management of the virtual infrastructure. The module https://github.com/puppetlabs/puppetlabsvcenter can be used to install (on Windows) and configure Center.
It provides native resource types to manage objects such as folders (vc_folder), datacenters (vc_datacenter), clusters (vc_cluster), and hosts (vc_host).
The Puppet code to manage the installation of vCenter on Windows and the configuration of some basic elements may look like the following:
class center {
media => ‘e:\\’,
jvm_memory_option => ‘M’,
}
vc_folder { ‘/prod’:
ensure => present,
}
vc_datacenter { [ ‘/prod/uk’, ‘/prod/it’ ]:
ensure => present,
}
vc_cluster { [ ‘/prod/uk/fe’, ‘/prod/it/fe’ ]:
ensure => present,
}
vc_host {‘10.42.20.11’:
ensure => ‘present’,
username => ‘root’,
password => ‘password’,
tag => ‘fe’,
}
vc_host { ‘10.42.20.12’:
ensure => ‘present’,
username => ‘root’,
password => ‘password’,
tag => ‘fe’,
}
vSphere virtual machine management with resource types
For more modern versions of the puppet enterprise (from Puppet 3.8), Puppet Labs maintains a module to manage vSphere Virtual Machines; this module requires rbvmomi and hocn Ruby gems and can be installed as any other module with:
puppet module install puppetlabs-vsphere
Before using it, we need to configure the module to be able to access the vCenter console; it can be configured using environment variables or a configuration file. The environment variables are:
VCENTER_SERVER=’host’
VCENTER_USER=’username’
VCENTER_PASSWORD=’password’
VCENTER_INSECURE=’true/false’
VCENTER_SSL=’true/false’
VCENTER_PORT=’port’
And if we want to use the configuration file instead, we must place a file with these values in /etc/puppetlabs/puppet/vcenter.conf using this format:
vcenter: {
host: “host”
user: “username”
password: “password”
port: port
insecure: false
ssl: false
}
Once installed and configured, the vsphere_vm resource is available; this allows to manage and list existing vms with puppet resource:
puppet resource vsphere_vm
For example, we could remove a vm with:
puppet resource vsphere_vm /opdx1/vm/eng/sample ensure=absent
And of course, it’s possible to define vms with puppet code:
vsphere_vm { ‘/opdx1/vm/eng/sample’:
ensure => running,
source => ‘/opdx1/vm/eng/source’,
memory => 1024,
cpus => 1,
extra_config => { ‘advanced.setting’ => ‘value’,
}
}
Amazon Web Services
Puppet-based solutions to manage AWS services have been around for some time. There are contributions both from Puppet Labs and the community and they relate to different Amazon services.
Cloud Provisioning on AWS:
Puppet Labs released a Cloud Provisioner module that provides faces to manage instances on AWS and Google Compute Engine. We can install it with:
puppet module install puppetlabs-cloud_provisioner
A new node_aws face is provided and it allows operations on AWS instances. They are performed via Fog, a Ruby cloud services library, and need some prerequisites. We can install them with:
gem install fog
gem install guid
To be able to interface with AWS services, we have to generate Access Credentials from the AWS Management Console. If we use the, now recommended AWS Identity and Access Management (IAM) interface, remember to set at least a Power User policy to the user for which access keys are created. We can place the access key ID and secret access key in ~/.fog, which is the configuration file of Fog:
:default:
aws_access_key_id: AKIAILAJ3HL2DQC37HZA
aws_secret_access_key: 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
Once done, we can interact with AWS. To see the ssh key pair names we have on AWS, we run the following command:
puppet node_aws list_keynames
To create a new instance, we can execute the following command:
puppet node_aws create — type t1.micro — image ami-2f726546 — keyname my_key
We have specified the instance type, the Amazon Machine Image (AMI) to be used and the SSH key we want to use to connect to it.
The output of the command reports the hostname of the newly created instance so that we can SSH to it with a command such as the following one:
ssh -i .ssh/aws_id_rsa root@ec2–54–81–87–78.compute-1.amazonaws.com
The list of all the instances (both stopped and running) that we use are:
puppet node_aws list
To destroy a running instance :
puppet node_aws terminate ec2–54–81–87–78.compute-1.amazonaws.com
We can specify the AWS region where instances are created with the — region option (the default value is us-east-1).
AWS provisioning and configuration with resource types
Since Puppet 3.8, the recommended way of managing AWS resources with Puppet is by using the official AWS module (https://github.com/puppetlabs/ puppetlabs-aws), this can manage AWS services to configure cloud infrastructure.
To access AWS services, Puppet needs the SDK and the credentials. To install the SDK, run:
gem install aws-sdk-core retries
You may need to execute a different gem depending on your installation and how you are going to run AWS management commands. For Puppet enterprise and starting on Puppet 4, gem is in /opt/puppetlabs/puppet/bin/gem,; if the code is going to be executed from a puppetserver, gem has to be invoked as /opt/ puppetlabs/bin/puppetserver gem, and the server needs to be reinstalled to see the new gems.
Credentials have to be set as environment variables:
export AWS_ACCESS_KEY_ID=your_access_key_id
export AWS_SECRET_ACCESS_KEY=your_secret_access_key
export AWS_REGION=region
Otherwise, in a file at ~/.aws/credentials:
[default]
aws_access_key_id = your_access_key_id
aws_secret_access_key = your_secret_access_key
aws_region = region
Finally, install the Puppet module itself with the following command:
puppet module install puppetlabs-aws
Once installed and with the configured credentials, it can be used as any other resource, directly from code or using the command line. As an example, a new instance can be launched with this Puppet code:
ec2_instance { ‘instance-name’:
ensure => present,,
image_id => ‘ami-123456’,
instance_type => ‘t1.micro’,
key_name => ‘key-name’,
}
Managing CloudFormation
We have seen how Puppet functionalities can be extended with modules, either by providing resource types that enrich the language or additional faces that add actions to the application.
The Puppet Labs’ Cloud Formation module provides one of these extra faces, https://github.com/puppetlabs/puppetlabs-cloudformation. It adds the puppet cloudformation subcommand, which can be used to deploy a whole Puppet Enterprise stack via Amazon’s Cloud Formation service.
Besides installing a Master based on Puppet Enterprise, the module configures various AWS resources (security groups, IAM users, and ec2 instances) and Puppet specific components (modules, dashboard groups, and agents).
