How to get started with Razor and Puppet – Part 5

Please read the comments, updates there!
Razor has gotten a CLI and API update, making some of the commands shown here not work correctly. If you come across something that doesn’t work, please look at the Razor wiki for help regarding that command.

Welcome to the next part of this series on “How to get started with Razor and Puppet”, where I’ll try to guide you through some of the configuration settings that are needed to get an even better automated flow of your Razor and Puppet deployments and installations.

First off, let’s go through what we have so far:

  1. A working DNS and DHCP service
  2. A working Razor installation that deploys Ubuntu 12.04 LTS onto your virtual nodes with no manual tasks needed
  3. A Puppet Master that receives those nodes as a final task of the deployment
  4. A Puppet Master that just can’t wait to get started installing cool stuff on our nodes

So let’s be nice to our Puppet Master and get it going with a few services and settings, ok?

First, let’s make sure you have a configuration file for Puppet, it should be in located at /etc/puppet/puppet.conf. If there is no file there, you can copy this into it (this is the default config from the packages we installed in Part 1).

[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
templatedir=$confdir/templates
[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

Verify that your Fully Qualified Domain Name as set in your DNS is in /etc/hosts on the Puppet Master:

192.168.72.130  puppet.purevirtual.lab  puppet.localdomain puppet

Now, as you saw in Part 4 we used the hostname “puppet” for our broker, and that should work fine as we already have an SSL cert for that hostname. However, as you will have more and more nodes to be connected to the Puppet Master, it might be a good idea to create an SSL cert that contains it’s FQDN as well (especially if you go outside your local net). To create this we’ll edit the puppet.conf file and add these two lines under [master]:

certname=puppet.purevirtual.lab
dns_alt_names=puppet,puppet.localdomain

Restart the service (service puppetmaster restart), and this will not only give us a certificate name of puppet.purevirtual.lab, but the certificate is also valid for hostnames puppet and puppet.localdomain. This means that you can now change the Broker Name in Part 4 to “puppet.purevirtual.lab” which will give a much cleaner setup. You can verify that the certificate was created by looking after a file by the name of “puppet.purevirtual.lab.pem” in /var/lib/puppet/ssl/certs, and validate the certificate by using the following command where you’ll also find the alternative names:

root@puppet:~# openssl x509 -text -in /var/lib/puppet/ssl/certs/puppet.purevirtual.lab.pem
<snip>
        Subject: CN=puppet.purevirtual.lab
<snip>
            X509v3 Subject Alternative Name:
                DNS:puppet, DNS:puppet.localdomain, DNS:puppet.purevirtual.lab
<snip>

Ok, perfect, now we have a valid certificate configuration for all our names of the Puppet Master. But what about the puppet node’s certificates? You might remember we had to manually sign them on the Puppet Master when they were handed off to the Broker in the last steps of deployment in Part 4. This actually made our Razor deployment stop at “broker_wait”, and it never went further after that. Everything was deployed correctly but you needed to restart the puppet agent service on the nodes after the manual certificate signing. Manual labour for signing certificates? We can’t stand for that! Let’s automate it!

Edit the /etc/puppet/puppet.conf file on the Puppet Master to allow for a function called “autosign” where we can sign client certificates automatically:

[master]
...
autosign = /etc/puppet/autosign.conf

Automatically signing certificates might be seen as a security risk, but that all depends on how you set it up. You can verify hostnames or domainnames before signing anything, you can let your firewall only let in trusted sources, or you could sign every node that’s connecting to the Puppet Master. Create a file called /etc/puppet/autosign.conf and put a * in there for now, meaning every node that asks for it will be signed and handed a configuration. Insecure, yes, but easy for us during this tutorial. If you wanted to secure it a bit by just signing those with hostnames in your domain, add “*.your.domain” instead. After that’s done, restart the puppetmaster service (service puppetmaster restart), boot up a new VM to be deployed (how many have you deployed now during these tutorials? Add a comment!) and watch for the following in your Razor log:

root@puppet:~# razor active_model logview
<snip>
postinstall=>os_complete     os_boot               n/a                            03:39:54  2 sec    9.9 min   10rX4LiuAMsqG0WydnHyaW
os_complete=>broker_success  broker_agent_handoff  n/a                            03:40:44  50 sec   10.8 min  10rX4LiuAMsqG0WydnHyaW

Broker Success! Now that’s awesome, we have an automated OS deployment and Broker handoff by Razor, and automatic certificate signing and configuration by Puppet. Sweet!

Now let’s see if we can do some magic on the Puppet node that we’ve installed. How about we install a service? One of the most important services in an automated datacenter is the NTP service that makes sure that all nodes have the same time. This will help us when troubleshooting logs and is also a necessity when communicating securely, like between the Puppet Master and nodes.

If you run the command “date” on your newly node you’ll see the following:

root@node8:~# date
Wed Jul 11 04:13:48 PDT 2012

PDT stands for Pacific Daylight Time. I’m located in Sweden which uses Central European Time (CET), so that’s not good. Let’s install timezone and NTP modules into our Puppet Master and do a few configuration changes.

root@puppet:~# puppet module install saz-timezone
Preparing to install into /etc/puppet/modules ...
Downloading from http://forge.puppetlabs.com ...
Installing -- do not interrupt ...
/etc/puppet/modules
âââ saz-timezone (v1.1.0)
root@puppet:~# puppet module install puppetlabs-ntp
Preparing to install into /etc/puppet/modules ...
Downloading from http://forge.puppetlabs.com ...
Installing -- do not interrupt ...
/etc/puppet/modules
âââ puppetlabs-ntp (v0.0.4)

So we have these module available, but they won’t be installed on any of the nodes that we have. To get them installed we need to define the module that each node will need. This is done through the /etc/puppet/manifests/site.pp file (not created by default, you need to create your own). In this file you can add everything on each nodes configuration and each function that they should perform. This means that the file can get quite large and might be hard to handle after a while. So let’s divide it up into smaller, easier to manage pieces. Add this into your site.pp file:

# site.pp
# define nodes
import "nodes.pp"

This will make it possible for us to handle the nodes in another file. You could have different node files for different purposes, for instance one file with your webservers and their functions (web.pp) and one with all your MySQL servers and their functions (mysql.pp). For now we’ll just use one, nodes.pp. In this file, add the following:

node default {    
}

This default node definition will make sure that all nodes not specificallydefined will still have the base timezone configuration applied to them. This base configuration could be a number of things (including packages that you always want on your nodes), but we’ll keep it at setting timezone and installing NTP for now.

If you want to add a specific setting to a specific node, it’ll look like this:

node specialnode.purevirtual.lab {
    something special goes in here
}

We’ll look at defining specific nodes later. First, let’s have a look at what our default configuration for all non-specific nodes will look like, including timezone and NTP settings. Once again, add this into your /etc/puppet/manifests/nodes.pp file:

node default {    
    class { 'timezone':
        timezone => 'CET',
    }
    include ntp

}

This will install the NTP service per the NTP module’s defaults (in this case adding servers from the NTP timeserver pool for us) and set the timezone to CET.

Either run the Puppet agent (command: puppet agent –test –verbose) manually on your node, or wait until the next update run is done. If run manually (stop the agent first if it’s running already), you should see this:

root@node8:~# puppet agent --server puppet --test --verbose
info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb
info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rb
info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb
info: Caching catalog for 21507ee0ad71012f4556000c293a4170
info: Applying configuration version '1342014613'

notice: /Stage[main]/Ntp/Package[ntp]/ensure: ensure changed 'purged' to 'present'
info: /Stage[main]/Ntp/Package[ntp]: Scheduling refresh of Service[ntp]
notice: /Stage[main]/Ntp/File[/etc/ntp.conf]/content:
--- /etc/ntp.conf       2012-06-05 22:12:18.000000000 +0200
+++ /tmp/puppet-file20120711-16752-6g25cp-0 2012-07-11 15:52:30.137454641 +0200
@@ -11,18 +11,19 @@
 filegen peerstats file peerstats type day enable
 filegen clockstats file clockstats type day enable

-# Specify one or more NTP servers.
-# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
-# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
-# more information.
-server 0.ubuntu.pool.ntp.org
-server 1.ubuntu.pool.ntp.org
-server 2.ubuntu.pool.ntp.org
-server 3.ubuntu.pool.ntp.org
+# You do need to talk to an NTP server or two (or three).
+#server ntp.your-provider.example
-# Use Ubuntu's ntp server as a fallback.
-server ntp.ubuntu.com
+# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
+# pick a different set every time it starts up.  Please consider joining the
+# pool: <http://www.pool.ntp.org/join.html>
+
+# Managed by puppet class { "ntp": servers => [ ... ] }
+server 0.debian.pool.ntp.org iburst
+server 1.debian.pool.ntp.org iburst
+server 2.debian.pool.ntp.org iburst
+server 3.debian.pool.ntp.org iburst
 # Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
 # details.  The web page http://support.ntp.org/bin/view/Support/AccessRestrictions
info: FileBucket adding {md5}32280703a4ba7aa1148c48895097ed07
info: /Stage[main]/Ntp/File[/etc/ntp.conf]: Filebucketed /etc/ntp.conf to puppet with sum 32280703a4ba7aa1148c48895097ed07
notice: /Stage[main]/Ntp/File[/etc/ntp.conf]/content: content changed '{md5}32280703a4ba7aa1148c48895097ed07' to '{md5}6e3461437c627101cf53e634abc62400'
info: /Stage[main]/Ntp/File[/etc/ntp.conf]: Scheduling refresh of Service[ntp]
notice: /Stage[main]/Ntp/Service[ntp]: Triggered 'refresh' from 2 events
notice: Finished catalog run in 31.12 seconds

Then run the “date” command to very that the timezone is set correctly:

root@node8:~# date
Wed Jul 11 15:38:08 CEST 2012

Voila! You now have a fully configured Razor installation with a Puppet Broker handoff, automatic signing of certificates, and all your new nodes will automatically have NTP installed and the correct timezone set. Fully automated, only thing left to you is to press the old ON-button on your servers 🙂

I’ve had comments on what people would like to see Puppetized next, so far OpenStack and CloudFoundry is on the list. What would you like to see Puppetized?

Oh, and here’s some good links to resources for your further studying:
http://narrabilis.com/mybook/puppet
http://finninday.net/wiki/index.php/Zero_to_puppet_in_one_day
And the book Pro Puppet by James Turnbull and Jeffrey McCune

Advertisements

About Jonas Rosland

Open Source Community Manager at {code} by Dell EMC
This entry was posted in Automation, EMC, Installation, IT Transformation, Puppet, Razor, VMware. Bookmark the permalink.

18 Responses to How to get started with Razor and Puppet – Part 5

  1. Swapnil Mankar says:

    Finally broker_success!!!! Its amazing to see that happening seamlessly. The NTP service got deployed without any manual work!

    And well, I must have deployed new VMs atleast 75 times using these tutorials so far 🙂

    I am eagerly awaiting to see the other services puppetized like openstack, apache etc.

    • jonas says:

      That’s great news! Thanks for going through all the tutorials, I hope you had fun doing it as well 🙂 75 VMs? That’s quite a lot, I like it!
      I think the next post will be about puppetizing Apache and some services bound on top of that, but that’ll have to wait until I’m back from vacation. So enjoy the summer and keep an eye out for new posts!

  2. Ashrith says:

    hey jonas, thanks for the great tutorial. I am king of stuck @ puppet broker hand off. For some reason razor was unable to do the puppet handoff, I found out this in the log(project_razor.log):
    E, [2012-09-30T12:37:19.678824 #24440] ERROR — ProjectRazor::BrokerPlugin::Puppet#compile_template: /opt/razor/lib/project_razor/broker/puppet/agent_install.erb

    Also, from the ‘active_model logview’
    os_complete=>broker_fail broker_agent_handoff n/a 12:37:23 4 sec 5.5 min 4R8hnHpRyO7psBNO6LDu9J

    I check for the host machine being deployed, there was no ip issue, nor dns issue. Eeverything looks fine to me. Any help is appreciated!

    • Jonas Rosland says:

      Hi Ashrit,

      Did the node have proper internet access? Can you verify that puppet was installed? Otherwise it won’t be able to do a puppet installation which is part of the puppet broker handoff.

      • Ashrith says:

        The node was able to ping razor node(which has pupper master installed) as well as has internet access. Also, puppet agent was not at all installed on the provisioned node.

      • Jonas Rosland says:

        Looks like there was an error executing the post-install script then. Are you able to install puppet manually via “sudo apt-get install puppet” on the node?

      • Ashrith says:

        hey jonas, i even tried using centos model and the razor was not even starting the process after kickstart. But the node was able to ping razor server as well as internet. Razor model_active logview revealed the following output:
        init mk_call n/a 13:39:55 0 sec 0 sec uqkRz44Un42n8SeCNeiC2
        init mk_call n/a 13:52:10 0 sec 0 sec 6HDcOa60W4MEzbXZcBoaXS
        init boot_call Starting Redhat model install 13:52:39 29 sec 29 sec 6HDcOa60W4MEzbXZcBoaXS
        init kickstart_file Replied with kickstart file 13:53:39 60.0 sec 1.5 min 6HDcOa60W4MEzbXZcBoaXS
        init=>postinstall kickstart_end Acknowledged kickstart end 13:58:04 4.4 min 5.9 min 6HDcOa60W4MEzbXZcBoaXS

      • Jonas Rosland says:

        Did you add a puppet broker to the CentOS model?

      • Ashrith says:

        Yes, I did. Here is the output from razor policy:
        $razor policy 5s

        UUID => 5sHsEuUSBL7u8ckBpdVGg6
        Line Number => 0
        Label => centos
        Enabled => true
        Template => linux_deploy
        Description => Policy for deploying a Linux-based operating system.
        Tags => [vmware_vm]
        Model Label => install_centos
        Broker Target => puppetmaster
        Currently Bound => 1
        Maximum Bound => 0
        Bound Counter => 1

        also, razor broker:
        $razor broker 2
        Name => puppetmaster
        Description => Puppet
        Plugin => puppet
        Servers => [razor.test.com]
        UUID => 2TXKWIsPJ18Iyks5npQtu

      • Jonas Rosland says:

        If you run
        grep postinstall /opt/razor/log/razor_daemon.log
        Can you see anything like this:
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall inject ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall added_puppet_hosts_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall set_hostname_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall apt_update_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall apt_upgrade_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall apt_install_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall send_ips 192.168.72.102 ‘{}’
        /opt/razor/bin/razor -w policy callback 7IPtzMfgLvfJFsk7p86bTc postinstall boot ‘{}’

      • Ashrith says:

        yes, here is the output:
        $grep postinstall /opt/razor/log/razor_daemon.log
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall inject ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall set_hostname_ok ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall sources_fix ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall apt_update_ok ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall apt_upgrade_ok ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall apt_install_fail ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall send_ips ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall boot ‘{}’
        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall final ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall inject ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall set_hostname_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall sources_fix ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall apt_update_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall apt_upgrade_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall apt_install_ok ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall send_ips 192.168.1.195 ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall boot ‘{}’
        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall final ‘{}’

      • Jonas Rosland says:

        I see you have two different results in the apt_install part of the post-install process:

        /opt/razor/bin/razor -w policy callback dEAhAByrz33pYn4yta97a postinstall apt_install_fail ‘{}’

        /opt/razor/bin/razor -w policy callback 3tio8nCNKP3fOP7nOZRGjC postinstall apt_install_ok ‘{}’

        So the last node seems to have gotten puppet installed ok. Send me an email at “firstname dot lastname” at EMC dot com, and let’s see if we can work it out for you.

      • Ashrith says:

        does this has something to do with razor version and mk version?

  3. Shannon says:

    When will you talk about specific node definitions? I know how to define specific nodes in the site.pp but am trying to figure out the best way of finding the node definitions that Razor passed to Puppet so I get the names right. I am doing this for OpenStack deployment and will have specific node definitions for each role.

    Good blog series.

  4. Dean Chris says:

    Pleasant Write-up, Thanks

  5. Rusheel Jain says:

    nice tutorials 😀
    …..
    i wanted to know that while installing OS via razor, is it possible to pass answers via answer file during model creation instead of typing each answer there.

    Thanks

    • Jonas Rosland says:

      Thanks!

      You can create your own models instead of using the standard ones, and in each model you can create the standardized answers that you want. I would recommend you to look into that, and I’ll probably do a blog post about it next week.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s