My attempts to marry DSC and vSphere have been going on for nearly a year* now. I showed some of my attempts and intermediate results at VMworld 2015, in two sessions at the PowerShell + DevOps Global Summit and recently during a session at the 24th VMUGBE+. But now I’m finally going public with the vSphereDSC module.
Since WMF 5 has been made available in preview, and still is in RTM at the moment I’m writing this, there have been constant changes to the way I was writing the DSC resources for vSphere. Since the February 2016 WMF 5 release, I now have a (somewhat) stable, working class-based solution. At least, that is what my initial tests seem to indicate.
This intro for my vSphereDSC series, will lay out the playing field. I’ll explain the concept I’m using, show some of the issues I encountered and explain the layout of the vSphereDSC Resource module.
* “Wisely and slow; they stumble that run fast”, Romeo and Juliet, Act II, Scene III
To start you’ll need some knowledge on what DSC (Desired State Configuration) is all about of course. This series of posts is not going to do that!
But don’t despair, there are many free, online resources available that explain way better than I ever could, how DSC works. To name but a few:
- On MSDN Windows PowerShell Desired State Configuration Overview
- On Simple Talk the 3-part series PowerShell Desired State Configuration
- The MVA Course Getting Started with PowerShell Desired State Configuration (DSC)
- On PowerShell.Org the The DSC Book
- Many DSC articles in the PowerShell Magazine are definitely worth reading, as are all of their articles btw.
Also check out the numerous blog posts and session recordings that have been made available. Google/Bing are your friends!
Not free, but there are also many DSC books available, and most of the major course vendors have one or more DSC Courses in their catalog nowadays.
I would also like to give a shout out to Brandon Olin, who is developing a DSC Resource module named POSHOrigin_vSphere. He uses a different angle to configure a vSphere environment through DSC. At the moment he has an impressive VM resource in his module.
The vEngine Concept
The LCM (Local Configuration Manager) is the DSC component that takes care of the “make it so” part. On regular intervals, or when you tell it to, it compares the Configuration, it has or has received, with the actual situation on the host.
If there are differences, and if the LCM was configured to correct this, the LCM will “correct” the drift. It will “Make it so!”
The first decision I had to make, was where are we going to run this LCM that will interact with a vSphere environment. An obvious choice would be to install it on the vCenter server, but there are some strong counter arguments for that.
- The vCenter Server, with the introduction of the VCSA (vCenter Appliance), is not necessarily running a Windows OS
- The vSphereDSC resource module should be able to configure any vSphere Server, that includes a stand-alone ESXi host
- For support and performance reasons we would probably not want to install additional “features” on a vSphere Server
- Changes to a vSphere Server (upgrades, migrations…) should not impact the vSphere DSC configuration and resources
The obvious choice, for me, was to introduce a “helper” server, the vEngine!
The LCM that configures/corrects the vSphere environment, will run on this vEngine Server. The vEngine Server only needs PowerShell and the PowerCLI modules/pssnapins. And it will “talk” with the vSphere Server via PowerCLI.
The workflow for this setup will look something like this
(1) The vSphere administrator makes sure he has the required DSC Resource modules on his workstation. This includes the vSphereDSC module. These modules can be retrieved from public or private repositories.
(2) The vSphere administrator creates a Configuration for his vSphere environment.
(3) The Configuration is pushed to the DSC Pull Server. The Configuration is stored there as a MOF file.
(4) The LCM on the vEngine will query the Pull Server at regular intervals to check if there is a new Configuration present. If there is, it will check if all the required resources for the new Configuration are present. If not, it will pull these DSC Resources from the Pull Server.
If there is no new Configuration, the LCM will use it’s local Current Configuration to “test” against the vSphere environment. If there is a drift, and if the LCM is configured to correct this, it will apply the Configuration on the vSphere environment.
During my work the vSphereDSC module I did encounter several “issues”. Some where caused by actual bugs that hopefully will be fixed when WMF 5 is released. Others were/are in fact caused by some of the design decisions in WMF 5. Remember that WMF 5 as we can currently use, is not yet the official release version.
A short summary of the issues (and the effect they had on the vSphereDSC module):
- The original idea to use Disconnected Sessions suffered from occasional crashes in the WMI that is running the LCM. After talking with the PowerShell Team this appears to be a know issue, but the fix is not yet present in the public WMF 5 build. So Disconnected Sessions are “On hold” for now.
- Class-based DSC resources, at the moment, do not support multiple files. That means all the vSphereDSC resources will be in one file. And that one file is configured in the RootModule parameter in the Manifest. Besides a less cleaner organisation, this didn’t have a big impact.
- The LCM resets the runspace each time a class method is invoked. As a consequence, there is no passing of information between successive invocations of class methods. This also caused issues with the PowerCLI Modules and PSSnapins. To fix this I moved all PowerCLI lines to a “helper” file outside the class methods. This way the number of times we need to load the PowerCLI Modules and PSSnapins is also limited, and it improves the runtime of the class methods.
- A PSSnapin that is loaded inside a module is only visible inside that module. This caused some problems when I was still struggling with the previous issue. For now, the external, PowerCLI Helper functions are in one .ps1 file that is dot-sourced from within the class methods. I will need to revisit this, and check if I can package the Helper functions in a module.
- Class-based DSC Resources do not support inheritance. I was hoping to use the same object hierarchy that is present in vSphere (for example ExtensibleManagedObject – ManagedEntity – VirtualMachine), but that had to be abandoned.
As you have gathered from the Issues paragraph, there have been some changes to my original plan. A quick summary of what I have and how I organised it.
In fact I try to follow the layout as described PowerShell\DSCResources as close as possible. The following is a quick treeview of part of my private repository. It shows the general layout, more details will follow in the posts that will discuss individual resources.
$env: psmodulepath (folder)
|- vSphereDSC (folder)
|- Tests (folder)
|- Integration (folder)
|- Unit (folder)
|- Tools (folder)
|- Examples (folder)
A question I regularly get when presenting the vSphereDSC module, is why would you use DSC to configure your vSphere environment ? They are already automating their vSphere environment through PowerCLI, why would they need to change that ?
I used to sometimes reply with “Why do people climb mountains ?”. But that was of course intended as an icebreaker. There are, in my opinion, enough reason to switch to configuring your vSphere environment through DSC.
Because I know each one of you will encounter the same question(s) when you propose DSC to configure your vSphere environment, I decided to start listing the arguments and some potential use cases in here. And my list is obviously far from complete, please contribute, and send me arguments and use cases you see.
- “Pets vs cattle”. Independent who coined the analogy first, this concept should be one of the major arguments you use in a DSC vSphere discussion. Scale out instead of scale up!
- “Scale * Complexity -gt Skill”. This comes from a TechEd 2013 presentation (MDC-B302) by Jeffrey Snover and Kenneth Hansen. Declarative, autonomous and idempotent deployment, configuration and conformance of standards-based managed elements.
- “Change is the primary cause of outage”. Comes from the same TechEd presentation as the previous bullet. Automation, in this case through DSC, will allow a continuous service availability, a continuous release cycle and continuous deployment.
- Separation of Configuration and Environment data. You would only need to source control your master Configuration logic, and the Environment data individually.
This concludes the vSphereDSC Intro. The scene has been set
What’s next ?
In the next posts in this series, I will start expanding into the individual resources that are present in the vSphereDSC module.