How to get started with Razor and Puppet – Part 4

Please read the comments, updates there!
Razor has gotten a CLI and API update, making some of the commands shown here not work correctly. If you come across something that doesn’t work, please look at the Razor wiki for help regarding that command.

Welcome to Part 4 in this ongoing series on “How to get started with Razor and Puppet”. If you’ve followed all the steps outlined earlier, you should now have a fully functional Razor installation going that will deploy Ubuntu 12.04 in an automated fashion:

  1. You boot up a VM
  2. The VM boots a Micro Kernel over the network and reports its hardware configuration as Node Attributes back to Razor
  3. Razor then organizes these Attributes into Tags
  4. A Policy will be chosen by using the Tags that are set on the node
  5. The Policy will kick off the Ubuntu 12.04 installation

With this scenario you can then happily create server after server and fill your datacenter with Ubuntu installations. But what should you do after that? Creating automated OS installations is definitely fun, but you probably want to actually run something on the deployed servers as well. Here’s where Puppet comes in.

Puppet is a great tool that can be used for managing the installed packages and configurations on a multitude of server installations. You have a Ubuntu server that needs  Apache+PHP? Easily achieved. You want a Red Hat server to include an installation of MySQL? Can do!

So with Razor as an OS deployment tool, and Puppet as an application and configuration management system, you’re pretty much set and can let your developers and software administrators loose on the systems to deploy services to their heart’s content.

For this to be possible we need to install a Puppet agent on the nodes after they’ve been deployed. This could be done manually (it’s very easy as the puppet-package is in most distrubutions repos), but as we’re trying to automate as much as possible in this tutorial we will add the Puppet agent as a part of the OS deployment and do a handoff to the Puppet Master that we’ve already installed, where we will then in the next blog posts create some configurations for the nodes. Sound like fun? Let’s get started!

Let’s create a new Policy that looks like our old one, but this time include the Puppet handoff. So what did our old one look like?

root@puppet:~# razor policy
Policies
#  Enabled   Label                      Tags                        Model Label    #/Max  Counter          UUID
0  true     precise  [cpus_2,memsize_1GiB,vmware_vm,onlyonedisk]  install_precise  1/-    1        Yf6unMzUdX68qdzpi9sFU

We see we have one policy, what did it look like?

root@puppet:~# razor policy Yf6unMzUdX68qdzpi9sFU
 UUID =>  Yf6unMzUdX68qdzpi9sFU
 Line Number =>  0
 Label =>  precise
 Enabled =>  true
 Template =>  linux_deploy
 Description =>  Policy for deploying a Linux-based operating system.
 Tags =>  [cpus_2, memsize_1GiB, vmware_vm, onlyonedisk]
 Model Label =>  install_precise
 Broker Target =>  none
 Currently Bound =>  1
 Maximum Bound =>  0
 Bound Counter =>  1

You can see that the Broker Target is set to none, that’s what we want to change. Add a Broker, point it to our Puppet Master server called “puppet” call it “puppetmaster”, and use the broker plugin “puppet”:

root@puppet:~# razor broker add plugin=puppet name=puppetmaster description="Puppet Master" servers=puppet
 Name =>  puppetmaster
 Description =>  Puppet Master
 Plugin =>  puppet
 Servers =>  [puppet]
 UUID =>  7FJsrrsojTcZ164KR3AI6q

Update our Policy to use the Broker we created:

root@puppet:~# razor policy update Yf6unMzUdX68qdzpi9sFU --broker-uuid 7FJsrrsojTcZ164KR3AI6q

If you need to update your Tags in your Policy (or something else, like the label for instance), you can do that as the same time like this (here I am changing cpus_2 to cpus_1):

root@puppet:~# razor policy update Yf6unMzUdX68qdzpi9sFU --broker-uuid 7FJsrrsojTcZ164KR3AI6q --tags cpus_1,memsize_1GiB,vmware_vm,onlyonedisk
 UUID =>  Yf6unMzUdX68qdzpi9sFU
 Line Number =>  0
 Label =>  precise
 Enabled =>  true
 Template =>  linux_deploy
 Description =>  Policy for deploying a Linux-based operating system.
 Tags =>  [cpus_1, memsize_1GiB, vmware_vm, onlyonedisk]
 Model Label =>  install_precise
 Broker Target =>  puppetmaster
 Currently Bound =>  0
 Maximum Bound =>  0
 Bound Counter =>  1

If you followed the tutorial during Part 1, you should have a Puppet Master up and running already on this server I call “puppet”, so you should be ready to start up a VM, see if Ubuntu is installed properly and Puppet is installed and the agent is pointing to the correct server. However, I would highly suggest you do some changes before this, and this is due to two reasons. One, we want to update the DHCP/DNS function to accomodate for proper hostnames and domain lookups, and two, there might be a small bug in your Razor installation if you haven’t updated it for a few days (todays date is July 9th 2012) that messes with your hostname setup for your nodes.

Let’s update the dnsmasq.pp that you created earlier, and add a few options that will be useful for you as a DNS admin 🙂

    dnsmasq::conf { 'another-config':
        ensure  => present,
        content => "dhcp-range=192.168.72.100,192.168.72.150,12hndhcp-boot=pxelinux.0ndhcp-option=3,192.168.72.2ndhcp-option=6,192.168.72.130ndomain=purevirtual.labnexpand-hostsndhcp-host=puppet,192.168.72.130nserver=8.8.8.8",
    }

As you can see the DHCP range is still the same, we’re still booting with pxelinux.0 and we still have the same gateway, but we’ve changed some other stuff. First, we changed the default DNS server from Google’s (8.8.8.8) to our own DNS server address, we’ve added a domain called purevirtual.lab, we’re adding that domain to every node connecting (expand-hosts), and we’ve added a static definition for our puppet server with it’s IP address. Finally, for our DNS server to actually be able to lookup stuff on it’s own, we’ve added an upstream DNS server, this time Google’s. Also, comment out the “127.0.1.1” line in /etc/hosts so our DNS doesn’t answer with that IP address when asked for the server “puppet”. Run the config by executing the following:

puppet apply dnsmasq.pp

Done! Let’s fix the bugs in Razor while we’re at it. Make sure that the following is commented out or removed in the file /opt/razor/lib/project_razor/model/ubuntu_precise.rb:

 #      attr_accessor :hostname

Also make sure that the following is edited into the file /opt/razor/lib/project_razor/model/ubuntu/precise/os_boot.rb as it won’t set the hostname properly otherwise:

#!/bin/bash

hostname <%= hostname %>
echo <%= hostname %> > /etc/hostname
sed -i '/127.0.1.1/{x;/^$/d;x}' /etc/hosts
echo 127.0.1.1 <%= hostname %>.localdomain <%= hostname %> >> /etc/hosts

Now we’re ready for some AWESOME Razor and Puppet magic. Let’s boot up a VM!

We can see that the VM is attaching to our policy:

root@puppet:~# razor node
Discovered Nodes
         UUID           Last Checkin  Status                                 Tags
2GYlQeQPF6acdUH2aOrSAy  7.7 min       B       [onlyonedisk,IntelCorporation,memsize_1GiB,cpus_1,vmware_vm,nics_1]
root@puppet:~# razor policy
Policies
#  Enabled   Label                      Tags                        Model Label    #/Max  Counter          UUID
0  true     precise  [cpus_1,memsize_1GiB,vmware_vm,onlyonedisk]  install_precise  1/-    4        Yf6unMzUdX68qdzpi9sFU
root@puppet:~# razor policy Yf6unMzUdX68qdzpi9sFU
 UUID =>  Yf6unMzUdX68qdzpi9sFU
 Line Number =>  0
 Label =>  precise
 Enabled =>  true
 Template =>  linux_deploy
 Description =>  Policy for deploying a Linux-based operating system.
 Tags =>  [cpus_1, memsize_1GiB, vmware_vm, onlyonedisk]
 Model Label =>  install_precise
 Broker Target =>  puppetmaster
 Currently Bound =>  1
 Maximum Bound =>  0
 Bound Counter =>  4

For a nice logview of what’s going on, you can use the following command:

root@puppet:~# razor active_model logview
All Active Model Logs:
         State                   Action                    Result                Time     Last     Total             Node
init                      mk_call               n/a                            07:29:41  0 sec    0 sec     2GYlQeQPF6acdUH2aOrSAy
init                      boot_call             Starting Ubuntu model install  07:30:02  21 sec   21 sec    2GYlQeQPF6acdUH2aOrSAy
init                      preseed_file          Replied with preseed file      07:30:26  24 sec   45 sec    2GYlQeQPF6acdUH2aOrSAy
init=>preinstall          preseed_start         Acknowledged preseed read      07:30:27  1 sec    46 sec    2GYlQeQPF6acdUH2aOrSAy
preinstall=>postinstall   preseed_end           Acknowledged preseed end       07:36:15  5.8 min  6.6 min   2GYlQeQPF6acdUH2aOrSAy
postinstall               postinstall_inject    n/a                            07:36:16  1 sec    6.6 min   2GYlQeQPF6acdUH2aOrSAy
postinstall               set_hostname_ok       Replied with os boot script    07:36:31  15 sec   6.8 min   2GYlQeQPF6acdUH2aOrSAy
postinstall               sources_fix           n/a                            07:36:32  1 sec    6.8 min   2GYlQeQPF6acdUH2aOrSAy
postinstall               apt_update_ok         n/a                            07:37:23  51 sec   7.7 min   2GYlQeQPF6acdUH2aOrSAy
postinstall               apt_upgrade_ok        n/a                            07:38:50  1.4 min  9.2 min   2GYlQeQPF6acdUH2aOrSAy
postinstall               apt_install_ok        n/a                            07:48:02  9.2 min  18.4 min  2GYlQeQPF6acdUH2aOrSAy
postinstall=>os_complete  os_boot               n/a                            07:48:03  1 sec    18.4 min  2GYlQeQPF6acdUH2aOrSAy
os_complete=>broker_wait  broker_agent_handoff  n/a                            07:48:34  31 sec   18.9 min  2GYlQeQPF6acdUH2aOrSAy

Now watch as it’s being installed, and when it’s done, and when you see the “broker_agent_handoff “, run the following command on your Puppet Master server to see the certificate request that has been sent to the master from the new node:

root@puppet:~# puppet cert list
  4a7d7450ac00012f43b4000c293a4170 (05:79:D3:FA:89:A7:FD:DF:DF:66:FD:4F:CB:FA:D2:01)

Now you might think “Ok, certificates, AWESOME, I’ll have to deal with that mess as well?”. Just relax, it’s not as bad as you think, it’s there to enable secure communications between the master and its nodes. This means you can have secure Puppet communications over any network, even the Internet (think public servers in a hosting facility for instance). Hopefully that changed your thinking into “Ah, secure communications, that’s great, so now what?”. Let’s sign that certificate request from our new node so it can communicate properly with our master. The name that is shown is a unique identifier for your new machine, and we can reuse that identifier when we’ll start with our configurations of the nodes 🙂

root@puppet:~# puppet cert sign 4a7d7450ac00012f43b4000c293a4170
notice: Signed certificate request for 4a7d7450ac00012f43b4000c293a4170
notice: Removing file Puppet::SSL::CertificateRequest 4a7d7450ac00012f43b4000c293a4170 at '/etc/puppet/ssl/ca/requests/4a7d7450ac00012f43b4000c293a4170.pem'

If you run the command “puppet cert list” again you’ll just see an empty response. that’s because that command only gives you a list of nodes you need to sign certificates for. If you want to list all the nodes you’ve signed certificates for, do this:

root@puppet:~# puppet cert list --all
+ 4a7d7450ac00012f43b4000c293a4170 (F3:29:36:E2:66:27:D6:87:39:54:E8:8F:3C:36:19:82)
+ puppet                           (C6:C9:F4:0C:B7:8D:2A:41:FC:78:69:CC:B6:A3:D7:DC)
+ puppet.localdomain               (74:D5:56:EB:0B:75:48:C4:69:FB:79:C7:7B:D0:D4:EE) (alt names: DNS:puppet, DNS:puppet.localdomain)

If there’s a specific server that isn’t in use anymore, or it’s being reinstalled, or you just hate it and want to remove it from puppet, do “puppet cert revoke <ID>” to disable the communications between the master and node, and then to clean up the stored certs from that specific node you can do “puppet cert clean <ID>”. Don’t do this with the new node though, you’ve only just started with it!

Ok, I think we’re done for today. Let’s see what you’ve learnt:

  1. Modified our DHCP/DNS setup to incorporate some important features for a fully functional setups
  2. Created a Broker using the Puppet plugin for Razor to handoff the node to
  3. Created a Policy that uses the Broker as a last action during the OS installation
  4. Seen the broker handoff being done by getting a certificate request from our new puppetized (yes, that’s now a real word) node
  5. Signed the node’s puppet certificate

This will probably be the last blog post detailing the setup of Razor, and we will look more into running Puppet and Getting Stuff Done in the next posts. Please comment on what services you’d like to see puppetized (like NTP, Apache, MySQL and so on), and Enjoy!

Advertisements

About Jonas Rosland

Open Source Community Manager at {code} by Dell EMC
This entry was posted in Automation, EMC, Installation, IT Transformation, Puppet, Razor, VMware. Bookmark the permalink.

6 Responses to How to get started with Razor and Puppet – Part 4

  1. Swapnil Mankar says:

    Had a trivial question here, can we include the ip address of the puppetmaster directly while declaring the ‘servers’ field in the broker?

    Eg:
    Name => puppetmaster
    Description => Puppet Master
    Plugin => puppet
    Servers => [192.168.42.112]
    UUID => 2F4vHXom3EJOEJ0uPTQ9tO

    • jonas says:

      You can use the IP address if the certificate that the host answer with has the IP set as a cert name. In this tutorial we haven’t messed around with the cert name settings, so it will not work with an IP, which is why I added the hostname (in my case “puppet”) as I know my server will give that as a proper cert name and it will work afterwards. I’ll probably do a post on how to change your cert settings later on, but for now, if you’ve followed the tutorial then no, the IP won’t work 🙂

  2. Swapnil Mankar says:

    Thanks Jonas. This is awesome!! I followed the tutorial and provided the hostname for the broker server. Currently the active policy state is “broker_wait”. I am assuming we need to do “something” so that the puppetmaster takes over and starts setting up various services.

    I would like to see something like the openstack platform being puppetized 🙂 I think puppet already has the capability to do that. Am wondering how it would work in this case.

    • jonas says:

      Thanks for the uplifting comment, glad you like it!
      If you’re stuck at “broker_wait”, that’s ok for now. Just sign the puppet agent cert, login to the newly deployed node and run “nohup puppet agent”. That should restart the agent and you should be all set, even if the Razor status won’t change.
      This is most likely due to the fact that we didn’t set our Puppet Master config to autosign the puppet agent cert, something that the current Razor implementation prefers. The installation is complete otherwise though, so you should be all set if you just run the “puppet agent” command. I’ll update the blog post with more info on this later.

      Openstack you say? Interesting. It will not be the first blog post (starting with something considerably smaller), but great tip and I promise I’ll look into it 🙂

  3. Michael Lozano says:

    A question and a couple of notes:

    Does the puppetmaster have a DNS server as part of the package, or do I need to set up a DNS server on my own with appropriate domain/IP settings?

    I noticed in the updated dnsmasq.pp in this post, you left out the ‘/’ symbol in front of the ‘ndhcp-option’ variables. Not sure if this is important.

    Also, you set your DNS server to 192.168.72.130, and your DHCP range to 192.168.72.100-150. This means at some point, the DHCP server is going to push out the IP address of your DNS server to some unsuspecting VM. 🙂

  4. Allan says:

    I need some assistance. Why is my broker_handoff failing?

    [root@razor ~]# razor active_model logview
    All Active Model Logs:
    State Action Result Time Last Total Node
    init mk_call n/a 05:18:37 0 sec 0 sec dSDX4CT1O0E3JIMRFTzzQ
    init boot_call Starting Ubuntu model install 05:18:54 17 sec 17 sec dSDX4CT1O0E3JIMRFTzzQ
    init preseed_file Replied with preseed file 05:19:19 25 sec 42 sec dSDX4CT1O0E3JIMRFTzzQ
    init=>preinstall preseed_start Acknowledged preseed read 05:19:20 1 sec 43 sec dSDX4CT1O0E3JIMRFTzzQ
    preinstall=>postinstall preseed_end Acknowledged preseed end 05:23:47 4.5 min 5.2 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall postinstall_inject n/a 05:23:48 1 sec 5.2 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall set_hostname_ok Replied with os boot script 05:24:06 18 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall sources_fix n/a 05:24:07 1 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall apt_update_ok n/a 05:24:08 1 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall apt_upgrade_ok n/a 05:24:08 0 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall apt_install_fail n/a 05:24:08 0 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall os_boot n/a 05:24:09 1 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    postinstall=>os_complete os_final Replied with os complete script 05:24:10 1 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ
    os_complete=>broker_fail broker_agent_handoff n/a 05:24:10 0 sec 5.5 min dSDX4CT1O0E3JIMRFTzzQ

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s