VMware View Failover Automation – Solved!

Last week I spent some time talking to a partner, ByteLife, about a solution that they’ve created for a customer. The customer needed an automated failover solution for their VMware View environment and this is the story on how they solved it.

VMware-View-Diagram-large-edited

First some background to what the customer saw as their problem. Customers using VMware View sometimes face issues trying to handle fault tolerance in case of site disaster. Even though VMware has a solution for failing over virtual machines to a secondary site, if you are lucky enough to have one, it does not support virtual desktop infrastructures (see here for the Compability Matrix, no SRM support for View).

As a result, VMware View is often installed either separately in both datacenters, guaranteeing that at least half of the desktops would survive the site failure, or even as a single environment that could break down in whole during a large outage.

The customers, whose business and workers usually don’t like losing access to their applications and/or desktops during a site failure, can choose a more complex setup and use specific manual failover tasks during the site failure. The good thing is that it is possible using solutions such as this from VMware or this from EMC. On the other hand – during a site failover, IT personnel already have a tremendous load and pressure to bring the site or the services back online – any additional service to worry about just adds to the unnecessary complexity of the crisis. Having an automated failover that can be initiated by few clicks in the remaining datacenter will free up the IT staff’s time, when they need it the most.

So for this specific customer, ByteLife has developed a solution called “VMware View Failover Automation” with the following key functionalities:

  • Failover desktop pools and virtual machines in case of site crash
  • Migrate desktop pools and virtual machines during maintenance, tests, and rebalancing the load between sites or as failback after disaster
  • Restore storage synchronization between datacenters after the outage
  • Integration with vSphere WebClient

But wait, there’s more!

For this, all you need is vCenter Orchestrator, no SRM. Yes, you read that right, no SRM. What’s even cooler is that you can actually use this for several sites, you’re not limited to just two sites! Imagine that, being able to failover any VMware View site, without SRM, within minutes.

Failover of the VMware View environment takes only minutes, depending on the number as well as nature of desktops and the components that are failed over. It’s been proven that the first users can restart their work in new site in less than 5 minutes after the failover is initiated, which I find pretty amazing compared to the other solutions I’ve seen on this subject. So how this work, what makes it so fast?

View Pools

Looking at the picture above, let’s assume your current Linked clone pools is called  *-A-Pool and your pool that’ll be used in a failover scenario is called *-A-Pool-Recovery. The pools are exactly the same, uses the same VM as base image, and some VMs are already pre-provisioned. So when failing over, all that’s done is registering the users to the *-A-Pool-Recovery pool, removing them from *-A-Pool and then they can reconnect. Same desktop Pool ID, same everything, so it’s fully transparent to the users. Some other settings are automated as well, like maximum amount of desktops per pool. All pools are enabled all the time, to make sure it’s possible to do changes and things like recompose on all pools to have a consistent image version across the entire environment. All automated, and seeing it live is really impressive.

But what about the manual pools? Well, they’re handled a bit differently. In case of a failure, the vCO workflow shuts down all manual VMs (if they are still reachable and running), removed from vCenter inventory, the datastores dismounted, replication flow of the datastore is then switched and the now primary datastore is attached to the secondary site, the VMs readded to the inventory, powered on, then the manual pool is modified in the AD LDS database to be moved from A to B. And of course, all user assignments are preserved. All automated, frickin awesome IMHO!

VMware View vCenter Orchestrator Failover Workflow

As this is based on vCO workflows, there’s no hardcoded input on pools or available sites, everything is collected using the Status Report, Migration, Failover and Restore Synchronization workflows. The vCO workflows only lists the pools and sites that actually have entitlements and are active, everything else is hidden meaning you can focus on getting your stuff up and running quickly instead of having to trawl through all the possible environments that *might* be used.

So, this can be used for failover, but also planned migration of VMs from one site to another if you want to balance the workload between sites for instance.

Another cool feature that came up during the discussion is that you could actually use this for recomposing large environments with very little downtime. Let’s say you’re currently using *-A-Pool as in our previous example, you could recompose the virtual desktops in *-A-Pool-Recovery, and just migrate your users over there. Instead of recomposing all existing VMs, you’d move your users to already recomposed images with fresh patches and everything installed, how cool is that?!

I found it very refreshing to see a totally new take on the failover methods for VMware View environments, and I’m certain it would benefit your environment.

And lastly, some technical info:
The solution is based on VMware vCenter Orchestrator workflows. The current version of VMware View Failover Automation is supported with VMware View 5.1 and up; and EMC VNX with MirrorView. The network latency between two sites must not exceed 5 ms.

Contact info for the solution:
Alar Kuuda (Project Manager) – alar.kuuda@bytelife.com
+372 5097873

Posted in Automation, EMC, SRM, vCenter, vCenter Orchestrator, View, VMware, VNX, vSphere5 | Tagged , , | 2 Comments

EMC World 2013 – Best one yet?

HeroImage2013

I’ve spent the last week in Vegas, meeting incredible people and learning a lot at this year’s amazing EMC World. Amazing, just like every year, but always a bit different. Time for an EMC World 2013 wrap-up don’t you think?

Let’s start with some interesting numbers:

For the event, 93 trucks holding 604,335 lbs arrived with stuff. All that stuff filled 2.25 Million sq ft space, which also included 3.4 miles network cables. The 15,000 attendees booked 42,618 hotel nights, drank 174,480 cups of coffee, walked on average 5,523 steps or 2.3 miles a day, looked at 8,768 PowerPoint slides and listened to the keynote sessions through a 190,000 Watt PA system. There were over 500 breakout sessions covering everything that EMC and it’s partners can deliver, and at the same time there were 31 Hands-On Labs where all attendees could have a closer look at all the cool tech that was announced throughout the week.

EMC World 2013 Hands-On Labs

And of course, some social media badge hacking was done as well :)

EMC World 2013 Badge Hack

During the first keynote sessions, Joe Tucci welcomed and thanked us all, customers, partners, employees, and then laid out the strategy for EMC going forward.

This strategy is very focused on enabling customers to run their own Software-Defined DataCenter (SDDC) by leveraging the intelligence built into today’s and tomorrow’s smarter hardware, abstracting the management to easier create services that consume virtual resources.

Leverage Intelligence

One of the biggest announcements at EMC World this year was definitely around Software-Defined Storage and a product called ViPR that makes it all possible. It abstracts the heterogenous management for different storage functions like block, file and object, and ties it all together into a single API with integrations to VMware, Microsoft and OpenStack (and yes, a GUI is of course available too). We’ve already done two blog posts on ViPR here and here so please go there for more information on it.

Software-Defined Storage

Another interesting “announcement” was the view of EMC’s four brands. At the top level is EMC², with EMC II, VMware, RSA and Pivotal as separate brands that are free to execute their own missions, but still are strategically aligned. I think it’s great to see that this strategy is working well for all four brands, and that it’s still the strategy going forward.

EMCs four brands with photos

As you might have noticed EMC has a new member of the family, Pivotal. Announced in late April, Pivotal’s mission is to take customers on their journey for a new platform for their Big Data, Fast Data and New Apps with full Cloud Independence. The last part is very crucial, as Pivotal wants to make sure customers can use any Cloud available or already in use, be it a VMware Private Cloud, AWS, OpenStack or something else. Pivotal is owned by EMC, VMware and GE, where GE sees Pivotal as a crucial part for their telemetry data collection of all their future products like jet engines, trains, PACS and so on. Now THAT is really cool!

Pivotal One Fabric Ingredients

In parallel to EMC World there was also a conference for SEs, including partner SEs. All in all, we were approximately 3000 people at the SE Conference, where 500+ were from partners! During this part of the conference, SEs from all EMCs partners were invited to join us in separate keynotes where they got the opportunity to participate in full transparency discussions with our top-level management, and technical breakouts (some of them NDA). More info on what you could see this year can be found at Chad Sakac’s blog post here. If you’re an EMC/VMware/RSA/Pivotal partner, you should definitely be there next year!

Outside the keynotes and breakout sessions there were also a large Solutions Pavilion, where the superhero theme was very prominent (I LOVE superheroes, so I thought it was really cool :) ). Some of the figures you could find there were Captain Scaleout (Isilon Hero!) and the X-Men, and there were a lot of games/contests where you could win everything from iPads to Syncplicity-branded sneakers. A great number of our partners were there to show off their solutions and how they integrate with the four brands on EMC, very cool to see that it wasn’t all focused on the storage part of EMC but rather the SDDC.

Jonas Rosland and X-Men

Another area where a lot of interesting things happened were over at the (HUGE!!!) EMC Square Social Media space. EMC TV were livestreaming interviews with a ton of interesting people, there were performances done by magicians and gymnasts, places where you could get your photo taken as a “DataCenter Hero” and more.

EMC Square Social Space

One part where I spent a lot of time as well was in the Blogger’s Space, where there were a dedicated space for all the EMC Elects (fancy!). The EMC Elects that attended EMC World were also invited for a tour of the SuperNAP in Las Vegas, the world largest and most powerful datacenter, so if you needed another reason why to get yourself listed as an EMC Elect, there you go :) Want to enlist as an EMC Elect? Add yourself or nominate someone you think should be on this list here.
So, what else happens in the Blogger’s Space during events like this? Well, mostly extremely interesting discussions with focused people, and to mention a few I’d like to point out @CommsNinja, @mjbrender, @colinmcnamara and @VirtualChappy. There are more but I won’t make this into an #FF post :)

Jonas Rosland DataCenter Hero
Other stuff that happened in the EMC Square Social Media space were several recordings of #EngineersUnplugged, and of course a ton of whiteboarding between customers, partners, bloggers and employees. It was pure awesome :)

To end this great EMC World 2013, we had Bruno Mars rockin’ it for a special private performance for all of us on Wednesday night, and MAN that was a great show. The whole band really filled the room and everyone in it with a ton of energy, and everyone really pitched in with singing, dancing and jumping around :)

Bruno Mars at EMC World 2013

Lastly, I’d like to extend a thank you to all the attending customers, partners and employees, you made it all an awesome experience for me and my colleagues and friends. So THANK YOU!

EMC World Thank You

Posted in Automation, Big Data, EMC, EMC World, IT Transformation, News, Pivotal, Social Media, VMware | Tagged , , , | Leave a comment

EMC ViPR give your SDDC superpowers

vipr_small_logoIts no news that IT operations are transforming with the help of software. VMware talk about the Software Defined Datacenter and how everything is becoming Software Defined. What do this mean in a storage context and how does EMC ViPR integrate with the SDDC to further extend on its capabilities. One of the questions being talked about a lot today is about storage being turned into software based appliances. With EMC ViPR we can deliver the best of both worlds and adapt as the market transform.

When adding physical storage to your SDDC today there are different questions that need to be answered before the storage resource end up being consumed by your SDDC. A lot of the questions need interaction with several different teams which in the end mean you lose valuable time before the ordered resources can be utilized.

manual_storage_provisioning_process

If we break down the process into storage related tasks one quickly understand that the provisioning task is not as easy as it may sound like. The administrator of the SDDC order storage resources and then have to answer a few questions sent back from the storage team and architects. When that´s done several different configuration tasks need to happen in different physcial hardware, switches, array etc.

What_make_storage_so_complex

A substantial part of this interaction between different teams and personnel can be automated with EMC ViPR meaning operations resources is relieved from repetitive manual tasks instead automated by software pushing speed of delivery as well as quality up substantially while at the same time standardizing storage operations, reporting and more.

With the ViPR vCenter Orchestrator plugin we can deliver full storage lifecycle management into your own SDDC. Instead of opening a storage provisiong request, waiting, answering questions, waiting ,likely answering more questions and waiting we can integrate the storage provisioning process directly into vCenter WebUI with approval processes and automatic provisioning.

Below is an example of how this could look like from a VM admin perspective when in the need to add additional datastore to the cluster named “homecluster”.

Step 1.

Mylab_Add_physcial_datastore_step1_masked

We click on “Add physical datastore” and follow the vCO workflow wizard.

Step 2.

Mylab_Add_physcial_datastore_step2

Here we can choose storage type, FC, NFS and decide on size of datastore and name.

Step 3.

Mylab_Add_physcial_datastore_step3

Next we choose which virtual storage array and pool to consume resources from.

Step 4.

Click finish and wait for your ordered storage resource to appear in your vSphere cluster. Depending on if there is an approval process attached in viPR to this resource it may take shorter or longer before your can start to consume the new resource.

Below is how this workflow look like. I will add another post later covering the details behind the workflow as well as how additional functionality can be added that extend your SDDC functionality further.

ViPR_vCO_workflow

Posted in Automation, vCenter, vCenter Orchestrator, vSphere5 | Tagged , | 1 Comment

Today, ViPR is Bo(u)rn(e) – Virtualize Everything, Compromise Nothing

Today is the first day of EMC World here in Las Vegas, and just a few moments ago during the keynote a new product was announced called EMC ViPR. Some of you might have heard about a project called Bourne (one of our worst kept secrets apparently), and this project now has not only a proper name but also a really cool logo :)


ViPR

So just what is this ViPR thing? I’ll explain in just a bit, but let’s start by looking at the current state of the Software-Defined Data Center (SDDC) functionality across the board first. The past decade and a half has seen virtualization technology transform applications, servers and networks into software abstractions that enable data center and IT managers to build adaptive and agile data centers. The rise of the SDDC promises to build on the progress of virtualization by completely abstracting every component of the data center from its underlying hardware so that IT can truly deliver IT resources as customizable, on-demand services. This is the transformative potential. However, the reality is that storage hasn’t really transformed into an easily managed entity for a truly virtual data center. Unlike applications, servers and networking, storage and its valuable data is still too often tied to proprietary hardware. And that sucks.

Storage hardware and operating systems still vary much more than current server, client or network platforms. Storage platforms are incredibly diverse – even different arrays from the same vendor can feature different operating systems, proprietary APIs and unique feature sets. Every new IT endeavor might require a new storage array – be it block, file or object-based – optimized for that purpose. Out of necessity, storage administrators have become storage managers who spend most of their time managing arrays rather than optimizing information storage for the business. If enterprises and service providers are going to break from this pattern and be part of the evolution to a SDDC they need to fundamentally rethink storage.

And this is where ViPR comes in, to disrupt the status quo. EMC ViPR brings the same virtualization benefits enjoyed by the compute and network elements of SDDC to storage. EMC ViPR is a revolutionary approach to storage automation and management that transforms existing and new heterogeneous physical storage environments into a simple, extensible and open virtual storage platform. The value proposition of the SDDC and cloud computing – easily consumed IT services, simple API access, and single management view – is now finally available for storage. This also means that your management of storage, no matter if you’re using VMware, Hyper-V or OpenStack will look the same, behave the same and provide the same functionality.

Software-Defined Storage integreates with VMware, Hyper-V and OpenStack

At it’s core, EMC ViPR is a storage virtualization software platform that abstracts storage from physical arrays – whether file, block or object-based – into a pool of virtual shared storage resources enabling an easy storage consumption model across physical arrays and the delivery of applications and innovative data services. ViPR abstracts the storage control path from the underlying hardware arrays so that access and management of multi-vendor storage infrastructures can be centrally executed in software.

Screen Shot 2013-05-06 at 4.53.11 PM

Of course, this is not just for EMC storage. I really hope you weren’t thinking that, as one of the coolest parts about ViPR is that it makes a multi-vendor storage environment look like one, big virtual array. ViPR uses software adapters that connect to the underlying arrays, and also exposes the APIs so any vendor, partner or customer can build new adapters to add new arrays. This creates an extensible “plug and play” storage environment that can automatically connect to, discover and map arrays, hosts and SAN fabrics.

Add support for new arrays

And, because ViPR is software-defined, it can easily extend to support non-EMC arrays and integrate with cloud stacks. ViPR is the first truly open software-defined storage platform. Through de-facto industry standard APIs including Amazon S3, OpenStack Swift and EMC Atmos, ViPR frees data and applications from storage dependencies and enables IT to meet new workloads and use cases with existing infrastructure.

Support for existing standards

So to sum it up, ViPR is:

  • Simple – providing easy storage provisioning, delivery and management across all arrays
  • Extensible – maintaining the unique capabilities of underlying storage arrays (EMC and third-party) making it possible to seamlessly migrate application data across private, public and hybrid cloud environments
  • Open – Everyone’s free to participate in the ViPR community, to help us and you to deliver the best services possible to your business

Stay tuned for more info on this awesome solution in the near future :)

Posted in Uncategorized | Leave a comment

Going to VMworld 2013? Purevirtual´s session covering vCO, Razor and Software Defined Storage could be for you

This years voting for sessions at VMworld 2013 have just begun and as lots of other people, Purevirtual also have a session proposal. Together with products from VMware, vCO and vSphere we will will walk you through how vCO can be used to integrate Software Defined Storage and Razor (Razor should not be new to you, there are several articles covering Razor on our blog) to add very cool enhancements to your Software Defined Datacenter.

our vmworld session abstract

Here´s how to vote for sessions at VMworld. First you need a VMworld account. Don´t have one? No problem. It´s easy to register an account (for free) at http://www.vmworld.com. When your account is ready go to http://www.vmworld.com/cfp.jspa to vote for you favorite sessions you like to see at VMworld this year.  Hint ;) Session 5472 is ours.

Our VMworld 2013 session

Thank you for your Support and hope to see you at VMworld 2013

Posted in Razor, VMworld | Tagged , , , | Leave a comment

Summary of Cisco Live UK 2013 through the eyes of a vSpecialist

Hats off to Cisco for a great event last week, very well executed!

I was lucky enough to be a part of this year’s event, and was there in the role of an “Expert Roamer” which meant I should pick up and hopefully be answer a lot of the questions our customers and partners had. The questions ranged from “Which one is better, FCoE or iSCSI?” to “So, I see EMC is much more than just a storage company, can you tell me what you can do for me in the data center space?”. Our booth was packed throughout the course of the three days, and even more so during the sessions that we were running every 15 minutes. More on those in a bit.

Cisco Live was all in all a great event, with 4000+ attendees and a lot of partners showing off solutions, services and products at their booths. I talked to several of them, and most of them were on the same page. The data center is transforming, and automation will play a big part in that during the coming years.

During the keynote, Padmasree Warrior, Chief Technology and Strategy Officer with Cisco, talked about what the future holds and also showcased several interesting product demos. What I found most interesting was how similar EMC and Cisco is in their messaging. At EMC, we’re always trying to show the best way of handling information. Storing it, making sure you can trust it, and enable information exchange throughout private, public and hybrid clouds. Cisco does the same but with another viewing point, namely communication.

With more than 50,000,000,000 devices thought to be connected through the Internet in 2020, technologies like smarter, more efficient wireless and IPv6 is becoming increasingly important. And I’m not talking about just your toasters being wirelessly connected either, but everything from manufacturing to healthcare. We’re talking about the Internet of Everything (#IoE).

With this Internet of Everything, communication will not just focus on person to person, but also thing to thing. And for all things to be able to talk to each other, we need more people with other skill sets than we have today. Therefore it was very interesting to see that this was acknowledged by Cisco, where they showed info from a Forbes article that contained information on 10 jobs that didn’t exist 10 years ago.

New skills will need to be developed in the IT area, just as it has been in many other professions. Think you are the best network technician if you just understand how to connect a server to a network? Think again. Link here.

Of course, more than just connectivity and skills was presented during the keynote. Of major importance was and is of course the move to Cloud. Or “The Cloud” as Cisco names it. This of course doesn’t mean Public Cloud for all of you, but rather a mix between running some services internally and some externally, with proper, secure and fast information exchange between them.

Together with the Internet of Everything and Cloud was also the consumerization of IT, namely the rising user expectations and mobile lifestyle. I myself consider my iPhone to be an Internet connected device with multiple functions rather than a phone. I’d say the phone “app” is the one least used, and has been so for quite a while.

Seems like I’m not alone out there thinking the same, as could be seen at Cisco’s huge Social Media lounge, where tweets and photos were livestreamed throughout the entire event. It’s easier to send out an important message to hundreds of followers or friends on Twitter and Facebook than to call or text them all.

Aside from the really interesting keynote, there was a lot of cool things being showed at the booths as well. EMC, VCE and VMware had a huge presence, with two large booths for EMC and VCE and a smaller one for VMware. During the last day of Cisco Live we had over a hundred people in our EMC booth at one time, and the same over at the VCE booth! It’s was crazy and immensely fun :)

Cisco has made 8 acquisitions from June until now, all in the software space, extending the strategy Cisco has around the software defined data center. After asking on Twitter, the main thing people wanted to hear more about during the keynote was SDN and the programmability of networking. One thing I found really cool was the onePK which allows you to programmatically manage your entire network environment. And, what’s even better, you can manage it with Puppet. You can be sure of it becoming a blog post in the near future :)

To conclude this and make it a proper wrap up, I’d like to show you a collage of pictures from the event, to show you a bit more on how the feeling is to be there in the flesh. Enjoy :)

Posted in Automation, Big Data, Cisco, Converged Infrastructure, EMC, IT Transformation, Puppet, VCE, VMware, VPLEX, VSPEX | 1 Comment

Going to Cisco Live in London this week? Don’t miss out on all the cool stuff!

IMG_5152
If you’re one of the lucky ~4000 people who’ll attend Cisco Live in London this week, I hope you’ll enjoy it to the fullest. Make sure you go to interesting sessions, attend cool vendor demos in the exhibition center, and network with your peers/idols/fans.

IMG_5151

EMC is a Platinum Sponsor for this weeks event, and as such we a have HUGE booth right by the entrance to the exhibition hall with 8 different areas fully stocked with experts wanting to talk to you:

Workstation 1 & 2: Cloud transforms IT Infrastructure
Workstation 3: Cloud transforms ITaaS
Workstation 4: Big Data transforms Business
Workstation 5: Trust Transforms Cloud
Workstation 6: Cloud Services
Workstation 7: Get connected with EMC (Social media workstation)
Workstation 8: EMC Digital Collateral Kiosk

To make it easy for you to keep track of all the cool stuff that we do at Cisco Live, EMC have a Community site up at https://community.emc.com/community/events/cisco_live, please have a look there for content, discussions and latest news.

Also, we’ll have presentations on how you can Transform IT+Business+Yourself running throughout the day with drawings for t-shirts and Apple TVs (awesome device IMHO).

IMG_5148

Btw, ever wondered what a Vblock or a VSPEX actually looks line? Not only do we have real hardware on the floor, we also have an interactive screen where you can easily move, remove, add, change and zoom in on all the hardware that makes up those converged infrastructures. Come touch it yourself!

We also have a bunch of great speaking slots, make sure you don’t miss out on the ones you’re really interested in:

EMC CONFERENCE SESSION:

Date: Jan. 30 – 16:30-17:30
Title: EMC & Cisco: Transforming mission-critical business continuity
Speaker: Oliver Shorey (EMC UK  @shorestor)
Cisco Live Session # BRKSPS-2503

EMC CASE STUDY:

Date: Jan. 31 – 11:30-12:30
Title: VSPEX transforms IT
Speaker: Gil Shneorson, EMC  & Richard Flanders, MTI Technology

LIVE WEBCAST:

Date: Jan. 30 -  15:00-16:00
Title: Transform your Data Center with EMC VSPEX and Cisco Technology
Speaker: Josh Mello  (@joshmello)
Product: VSPEX
Registration page: http://ow.ly/gRhbf

At the Cisco stand:

Date: Jan. 31 – 10:45-11:00
Title: Transform your Data Center with EMC VSPEX and Cisco Technology
Speaker: Josh Mello  (@joshmello)
Product: VSPEX

At the VCE stand:

Dates: Jan 29 at 11:30 and Jan 30 at 10:30
Title: EMC Backup and Recovery
Speaker: Mark Galpin (@markgalpin)

At the LSI stand:

Title: EMC, LSI and Cisco’s combine to deliver best-of- breed solution for server flash caching
Speaker: Simon Stevens  (@EMCFlashHeart)
At the EMC Booth we’ll also have Social Meetups during the event, dates, times and subject for these are:

Jan. 29:  Josh Mello – VSPEX (12:00-12:30)
Jan. 30: Simon Stevens – EMC Flash Strategy  (12:00- 12:30) and     Gary Owen (VMware) – Why your VDI Platform choice matters  (14:00-14:30)
Jan. 31: Roberto Barbero (VMware) – Delivering The Efficiencies Of Virtualisation To Business Critical Applications  (14:00-14:30)

Hope to see you on the show floor!

Posted in Automation, Backup, Big Data, Cisco, Converged Infrastructure, EMC, IT Transformation, Unified Storage, VCE, VSPEX | Leave a comment

Managing Razor provisioning with Puppet – Part 2

puppet plus razor is good

So we went through Puppet configuration of deploying a functional Razor installation in the first part here.

Now let’s look into how we can get Puppet to manage our Razor environment, namely models, images, policies and brokers. For this we’re gonna start off with configurations for deploying the Debian Wheezy distribution and then go on to Ubuntu Precise.

Open up your Puppet node configuration (site.pp or similar) for your Razor deployment and add the following within your “node puppet” definition that we defined in Part 1:

rz_image { "debian_wheezy":
  ensure => present,
  type => 'os',
  version => 'beta4',
  source => "http://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/current/images/netboot/mini.iso",
}

Run “puppet agent -t” and you should see the following output:

Info: Applying configuration version '1355082848'
 Downloading rz_image from http://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/current/images/netboot/mini.iso ...
 /Stage[main]//Node[puppet]/Rz_image[debian_wheezy]/ensure: created

Ok, so something got applied, now let’s run the following to verify that it worked:

root@puppet:~# razor image
UUID => 2RdsxhDxOvnNii39AY3h56
Type => OS Install
ISO Filename => mini.iso
Path => /opt/razor/image/os/2RdsxhDxOvnNii39AY3h56
Status => Valid
OS Name => debian_wheezy
OS Version => beta4

Look at that! Automatic ISO download and image injection into Razor. Let’s keep going and add a model for Debian Wheezy as well:

rz_model { 'install_wheezy':
  ensure => present,
  description => 'Debian Wheezy',
  image => 'debian_wheezy',
  metadata => {'domainname' => 'purevirtual.lab', 'hostname_prefix' => 'debian-', 'root_password' => 'password'},
  template => 'debian_wheezy',
}

Run your Puppet agent again and this is what you should see:

Info: Applying configuration version '1355083415'
 /Stage[main]//Node[puppet]/Rz_model[install_wheezy]/ensure: created

Let’s verify that as well:

root@puppet:~# razor model
 Models
 Label Template Description UUID
 install_wheezy linux_deploy Debian Wheezy Model 4EBOggCnQG90P4qBUNT4zI

Now we’re getting somewhere!

Let’s define our broker (meaning what we’ll hand off to after OS provisioning is done, here we’ll use our Puppet installation) before we go further:

rz_broker { 'puppet_broker':
  ensure => present,
  plugin => 'puppet',
  servers => [ 'puppet.purevirtual.lab' ]
}

Apply the broker configuration using “puppet agent -t”:

Info: Applying configuration version '1355084448'
/Stage[main]//Node[puppet]/Rz_broker[puppet_broker]/ensure: created

After applying it, let’s verify it:

root@puppet:~# razor broker
Broker Targets:
Name Description Plugin Servers UUID Version
puppet_broker puppet_broker puppet [puppet.purevirtual.lab] 6jvJa0SsZYt0vUEQzusb8y Default

Let’s create a Razor tag as well, shall we? Just to make sure that only the VMs with the correct hardware requirements gets installed with Debian Wheezy. Let’s call it.. hmm.. “halfagigofram”!

rz_tag { "halfagigofram":
  tag_label => "halfagigofram",
  tag_matcher => [ {
    'key' => 'mk_hw_mem_size',
    'compare' => 'equal',
    'value' => "512MiB",
  } ],
}

Apply it…

Info: Applying configuration version '1355084532'
 /Stage[main]//Node[puppet]/Rz_tag[halfagigofram]/ensure: created

And verify it!

root@puppet:~# razor tag
 Tag Rules
 Name Tags UUID Matchers [count]
 halfagigofram halfagigofram UHIdK6YlezPUnfevHtebY 1

Now let’s create a new Razor policy using everything we’ve done so far:

rz_policy { 'wheezy_policy':
  ensure => present,
  broker => 'puppet_broker',
  model => 'install_wheezy',
  enabled => 'true',
  tags => ['halfagigofram'],
  template => 'linux_deploy',
  maximum => 10,
}

Last apply for today:

Info: Applying configuration version '1355084748'
 /Stage[main]//Node[puppet]/Rz_policy[wheezy_policy]/ensure: created

Verify the last part as well:

root@puppet:~# razor policy
Policies
# Enabled Label Tags Model Label #/Max Counter UUID
2 true wheezy_policy [halfagigofram] install_wheezy 1/10 1 4eAaj0CxKdQNtYD3SmDxk8

Let’s try it out! Boot up a VM with half a gig of ram (512MB for those who can’t process words) and see the magic happen:

root@puppet:~# razor node
 Discovered Nodes
 UUID Last Checkin Status Tags
 7JHPaRlWkrmph5w8jdbvBw 3 sec A [halfagigofram,onlyonedisk,IntelCorporation,memsize_512MiB,cpus_1,vmware_vm,nics_1]

DONE! You now have a proper Puppet configuration for a Razor deployment of Debian Wheezy which will automatically get connected to Puppet after OS provisioning is complete, to do more interesting further application deployments. If you wonder if your node really got #puppetized, just do the following:

On the newly installed node:

cat /etc/puppet/puppet.conf
<snip>
certname = <LONG UUID>
</snip>

On the Puppet server run:

puppet cert list --all

And verify that you actually see the certname UUID there as well with a “+”-sign in front of it, meaning it’s signed and ready to rock!

Ok, so some of you might wonder how a Ubuntu variant of this would look like (as I was using Ubuntu is all my previous examples). Well, it would look something along these lines:

rz_image { "ubuntu_precise":
  ensure  => present,
  type    => 'os', 
  version => '12.04.1',
  source  => "http://ftp.sunet.se/pub/os/Linux/distributions/ubuntu/ubuntu-cd/12.04.1/ubuntu-12.04.1-server-amd64.iso",
}

rz_model { 'install_ubuntu':
  ensure => present,
  description => 'Ubuntu Precise',
  image => 'ubuntu_precise',
  metadata => {'
    domainname' => 'purevirtual.lab', 'hostname_prefix' => 'ubuntu-', 'root_password' => 'password'},
  template => 'ubuntu_precise',
}

rz_policy { 'precise_policy':
  ensure  => present,
  broker  => 'puppet_broker',
  model   => 'install_ubuntu',
  enabled => 'true',
  tags    => ['memsize_1GiB'],
  template => 'linux_deploy',
  maximum => 10,
}

Run “puppet agent -t” once again to get everything set up:

Info: Applying configuration version '1355164222'
 Downloading rz_image from http://ftp.sunet.se/pub/os/Linux/distributions/ubuntu/ubuntu-cd/12.04.1/ubuntu-12.04.1-server-amd64.iso ...
 /Stage[main]//Node[puppet]/Rz_image[ubuntu_precise]/ensure: created
 /Stage[main]//Node[puppet]/Rz_model[install_ubuntu_precise]/ensure: created
 /Stage[main]//Node[puppet]/Rz_policy[ubuntu_precise_policy]/ensure: created

At last, verify it by checking “razor image”, “razor model” and “razor policy”. Now you have not just one but TWO operatingsystems ready to be deployed, well done you!

Please let me know in the comments how it worked out for you and what distributions you’ve deployed using this method. Enjoy :)

References:
http://wiki.debian.org/OpenStackRazorHowto
https://github.com/puppetlabs/Razor/wiki
https://github.com/puppetlabs/puppetlabs-razor/wiki

Posted in Automation, EMC, How to, Installation, IT Transformation, Puppet, Razor | Tagged | 5 Comments

Managing Razor provisioning with Puppet – Part 1

puppet plus razor is good

If you’ve gone through the tutorials on how to get started with Razor and Puppet that I’ve posted earlier (see here for the first part of five) you should have a Razor and Puppet installation already up and going. For this tutorial series I’d like to show you how to manage your Razor configurations with Puppet, making it less dependent on commandline and more easily manageable.

First, let’s look at installing Razor through Puppet. A manual variant of doing this was already covered in the first part of the Razor and Puppet howto, but as we’re moving into managing everything automatically through Puppet let’s go through a cleaner and more repeatable way of doing it. For this to work you’ll already need to have a working Puppet installation up and running (explained in my previous howtos, but basically what you need is Puppet installed with autosigning of certificates enabled (unsafe for production environments but fine for testing)). I am assuming you are root or using sudo in front of every command.

First, install the Razor module:

puppet module install puppetlabs-razor

Then open up your /etc/puppet/manifests/nodes.pp (or any other file where you’re managing your node config, like site.pp) and add this into it:

node puppet {
  class { 'sudo':
    config_file_replace => false,
  }
  include razor
}

Of course, make sure that the nodename (here called “puppet”) matches your Razor server name. Now run the puppet agent (command: “puppet agent -t”) on the machine you’ve defined above and watch the magic happen as Razor get’s automagically installed! This way, when you’ll update the Razor module as a new version gets released everything will be updated on your Razor node as well. Sweet!

Another thing we did previously was to install DNSmasq for DHCP and DNS functionality,  and we managed it manually through a single configuration file. For more of the specific settings in the DNSmasq config please see this post. Let’s clean that up a bit as well, and use the following settings to get DNSmasq up and running properly using a proper Puppet configuration:

puppet module install saz-dnsmasq

Then add the following into your node definition:

node puppet {
 class { 'sudo':
 config_file_replace => false,
 }
 include razor

 dnsmasq::conf { 'another-config':
   ensure => present,
   content => "dhcp-range=192.168.72.100,192.168.72.150,12h\ndhcp-boot=pxelinux.0\ndhcp-option=3,192.168.72.2\ndhcp-option=6,192.168.72.130\ndomain=purevirtual.lab\nexpand-hosts\ndhcp-host=puppet,192.168.72.130\nserver=8.8.8.8\n",
 }
}

Run “puppet agent -t” again on your Razor server and watch DNSmasq getting installed and ready to use for all your deployment needs.

So, instead of doing it all manually you now have a complete Puppet node configuration for your present and current Razor installation needs, awesome :)

Next post will cover defining images, models, policies and brokers using Puppet.

Posted in Automation, EMC, How to, Installation, IT Transformation, Puppet, Razor | Tagged , , , , | Leave a comment

Exploring vCO and REST APIs

Lets talk about vCenter Orchestrator and how to use it to drive other products or solutions that have a rest based API. During my journey learning about vCO I had to communicate with an external REST API. Specifically an element orchestrator for storage process automation.

First of all we need a common understanding of how vCO can be used to talk to REST APIs. At the very first you need to install the rest plugin. Next you add a so called “REST host” by runnning a predefined workflow that is automatically made available by the rest plugin. In the third step you define “REST Operation(s)”. A rest operation define how to call a REST API and with what parameters. this operation can later be reused in your own workflows. The good thing doing it this way is you get a lot of help along the way as you have to work your way through different wizards. For my use case it was not a great solution hence why I´m writing this article.

first of all you need to run the “Add a REST host” workflow.

Add all parameters asked for in the wizard:

Name = logical name you will later use in javascript
URL = Base URL to REST API host

Choose authentication mechanism. Our example host uses basic authentication.

Our example uses a shared user for all operations done where this host definition is used

Next step is to add a REST operation. This is an operation we can call from our own developed workflows if needed. I won’t go into much details of how to do this but instead link to another blog post that explain how this works. See, http://blogs.vmware.com/orchestrator/2011/07/browsing-the-netflix-catalog-using-the-vco-http-rest-plug-in.html
referenced blog post is created with an older version of vCO. Although the concepts are the same in vcO v5.1.

Lets get started with the details. One thing you should do too, is to let vCO manage your REST host credentials. I don’t want to have username and passwords in clear text within my workflows which is why I decided to use vCO builtin repository of REST Hosts. Doing it this way mean you don’t have to worry much about security as much as you can call upon a host from your workflow, without knowing it´s username and password. Lets have a look at how it is done in JavaScript. The code example below is part of a “Scriptable Task” with a few input parameters. Namely “restHost” and “size”, “restHost” is created as a workflow attribute with a predefined value that contain the logical name of the REST host we added earlier (mytesthost) and “size” is an input parameter.


var restOp = new RESTOperation("Order NFS Storage from ISM"); // Define the name of the new operation
restOp.method = "POST"; // How is data sent to the api when called. POST mean we send data to the api.
restOp.urlTemplate = "/api/services/43?volumeName={exportName}&size={size}&rwHosts={RWHosts}"; // Define how to call api. Three parameters get sent when we call the api.
restOp.defaultContentType = "application/json";
var exportName = "vCO_" + datastoreName; // Set name of NFS export
var RWHosts = "192.168.100.21"; // Which hosts should have R/W access to export
var inParamtersValues = [exportName, size, RWHosts]; // Add input parameters and variables to array
operation = restHost.addOperation(restOp); // Add defined operation to rest host defined in "restHost" attribute
var content = ""; // Our api do not need any payload send to it as all interaction is done through different URLs
var request = operation.createRequest(inParamtersValues, content); // Create request to api

Now you know how to use a predefined REST Host in your workflows directly from javascript while defining your own operation that can be executed against that host. The code snippet below will execute the defined REST Operation above and return data from the REST Host and store it in a variable named “response”.


var response = request.execute(); // Execute request and store returndata in "response"
/ Code below is for debugging purposes and will output data in Orchestrator client when run. Output is not persistent.
System.log("Returned status code: " + response.statusCode);
var contentAsString = response.contentAsString; // Convert response to plain string
System.log("Content as string: " + contentAsString);

At this point we have executed our operation and we got a response back. If this was production grade code we would have error checking in the code before we process the returned data. For now we consider all responses to be ok and being of type json. Code snippet below show you how to process our returned json data and extract a variable named “orderId”.


var jsonResponse = JSON.parse(contentAsString);
ismOrderId = jsonResponse.orderId; // Extract the orderId variable form the response
System.log ("orderId = " + ismOrderId);

By now you should have a basic understanding of how to interact with REST APIs through JavaScript in vCO. As you have realized already this article do not tell how to build a complete workflow that communicate with a REST API. It was not my intention to do so either. I welcome all comments and interaction as I´m also learning about vCO at the same time.

Posted in Automation, Unified Storage, vCenter Orchestrator | 2 Comments