The vSphereDSC module contains a set of DSC resources to can be used to configure a vSphere environment. These DSC resources can be used against any vSphere Server, beit a vCenter or an ESXi node. On the condition of course that the selected resource is supported on the vSphere Server.

For “users” of the vSphereDSC resources, the post will show how to automate keeping the module up to date and how to manage the life cycle of the Configuration files that are build on the vSphereDSC resources.

For those of you that want to contribute to the development of the vSphereDSC module and it’s resources, this post will also show how you can automate the testing phase. In a first instance through a number of PowerShell scripts, in a later phase through the use of a build server.

The vSphereDSC module

The vSphereDSC is a module that contains a number of DSC resources, that you can use to configure your vSphere environment. To use the vSphereDSC resources, you will need to install the module on your systems.

The first place where you will need the module, is the station where you create and compile your DSC Configuration files.

The other place depends on which enacting method you select to use. With a Push setup, you will have to make sure the DSC Resource module is available on the target node. With a Pull setup, you will have to make the DSC Resource module available on the Pull server.

The following schema, which I shamelessly copied from Introducing PowerShell Desired State Configuration (DSC), shows a high-lelvel view of both methods.

There are some good overviews available on these two DSC methods. To name but a few:

• Getting Started with PowerShell Desired State Configuration (DSC), an MVA course with Jeffrey Snover and Jason Helmick
  
• Enacting configurations, in the Windows PowerShell Desired State Configuration Overview
• Pushing Configurations and Configuring a Pull Server, both in The DSC Book
• How to set up your first PowerShell DSC pull server from Adam Bertram

In my development and test environment, I use a Pull Server setup. Most of the scripts and procedures that are provided, are intended for a Pull Server setup. But the actions required for using a Push setup, are mostly the same, and simpler (fewer steps).

And one last remark, before we actually start doing things, I use the February 2016 WMF 5 release for creating the vSphereDSC classes.

Configuration files

In the vSphereDSC module, you will find a subfolder Configurations, in which there are several sample Configuration files. Let’s have a closer look at one of these files.

enum Ensure {
   Absent
   Present
}

$tgtName = 'vEng.local.lab'
$configName = 'vmw'
 
Configuration $configName
{
  param(
  [System.Management.Automation.PSCredential]$Credential
  )

  Import-DscResource -ModuleName vSphereDSC

  Node $AllNodes.NodeName
  {
    $number = 0
    foreach($datacenter in $Node.Datacenters)
    {
      $number++
      $dcName = "Datacenter$number"
      VmwDatacenter $dcName
      {
        Name = $datacenter.DatacenterName
        Path = $datacenter.Path
        Ensure = $datacenter.Ensure
        vServer = $Allnodes.Server
        vCredential = $Allnodes.Credential
      }
    }
  }
}

#region VCSA Account
$vcUser = '<your VCSA user>'
$vcPswd = '<your VCSA user password>'
$sVcCred = @{
  TypeName = 'System.Management.Automation.PSCredential'
  ArgumentList = $vcUser,(ConvertTo-SecureString -String $vcPswd -AsPlainText -Force)
}
$vcCred = New-Object @sVcCred
#endregion

$ConfigData = @{   
  AllNodes = @(
    @{
      NodeName = '*'
      Server = 'vcsa.local.lab'   
      Credential = $vcCred
      PSDscAllowPlainTextPassword=$true
      PSDscAllowDomainUser = $true
    },
    @{
      NodeName = $configName
      Datacenters = @(
        @{
          DatacenterName = 'DC1'
          Path = '/'
          Ensure = [Ensure]::Present
        }
      )
    }
  )  
} 

. "$(Split-Path $MyInvocation.MyCommand.Path)\..\..\Tools\Get-TargetGuid.ps1"
$guid = Get-TargetGuid -TargetName $tgtName

Invoke-Expression  "$($configName) -ConfigurationData `$configData -OutputPath '.\DSC'"

$pullShare = '\\pull\DSCService\Configuration\'
$mof = ".\DSC\$($configName).mof"
$tgtMof = "$pullshare\$guid.mof"

Copy-Item -Path $mof -Destination $tgtMof
New-DSCChecksum $tgtMof -Force

# For testing with Start-DscCOnfiguration
Copy-Item -Path $mof -Destination ".\DSC\$($tgtName.Split('.')[0]).mof"

Annotations

Line 1-4: Enumerations that are used in the vSphereDSC Resources that are used in this Configuration file

Line 6: The hostname of the vEngine server

Line 7: The name we give to the Configuration, which will also be the name of the generated MOF file

Line 9-34,46-66: We separate the “What” and “Where” in the Configuration. The “What” is in this Configuration the way we define a vSphere Folder. The $ConfigData hash table, is the “Where”. See Separating “What” from “Where” in PowerShell DSC for some more information on this subject.

Line 36-44: In the current version of the vSphereDSC module, credentials still need to be in clear text. I plan to change that later on.

Line 68-69: The Servers that are managed by the Pull Server, need to be addressed by their unique GUID. I store those in a file, and use a short script to retrieve the GUID from that file.

Line 71: We “run” the Configuration. Note how we pass the Where via the ConfigurationData parameter. The resulting MOF file will be stored in a subfolder, named DSC, in the current folder.

Line 77: We copy the MOF file to the Pull server. During the copy, the MOF will be renamed to <GUID>.mof

Line 78: The Pull server needs to distinguish between successive versions of a Configuration file. For that reason we calculate a checksum file.

Line 81: We are going to use Start-DscConfiguration to test the Configuration, for that reason we rename the MOF file to the shortname of the vEngine Server. More on that later.

Apply the Configuration

Now that we have the MOF file, in the correct format, in place on the Pull Server, we can wait till the LCM on the vEngine Server “pulls” the Pull Server again. But we can also speed that up, by forcing the DSC Configuration to run. In the Tools folder, there is a sample script to do just that. The file is named Test-DscConfiguration.ps1.

$tgtName = 'vEng.local.lab'

$configName = $tgtName.Split('.')[0]

Start-DscConfiguration -ComputerName $configName -Wait -Verbose -Force -Path .\DSC

The $tgtName variable is the name of the vEngine Server. We extract the first qualifier from the FQDN. And that was the name we gave to the MOF file in the last line of the previous script (veng.mof). The Start-DscConfiguration cmdlet will look for a MOF file in the location indicated by Path, and it will look for a MOF file with the name that is passed on the ComputerName parameter.

When all goes according to plan, you will see the Verbose messages scrolling through on the PS console. Have a look at this sample run in a short demo I recorded for the 24th VMUGBe+.

Development

During the development of the vSphereDSC module, I will regularly need to test the code I have written. In the previous section I described how you “force” your Configuration to run immediately. In this section, I’m going to describe what needs to be done to get the vSphereDSC module, in the correct format, on the Pull Server.

Scripted

This is the least advanced way of getting the vSphereDSC module to the Pull Server. It’s all done through a script. The script is provided as file vSphereDSC-Module-To-Pull-Server.ps1 in the Tools folder.

$moduleName = 'vSphereDSC'
$sourceFolder = "$($env:userprofile)\OneDrive\Github"
$pullSrvName = 'pull.local.lab'

$srcPath = "$($sourceFolder)\$($moduleName)"
$pullSrv = "\\$($pullSrvName)\DscService\Modules"

$localPath = $env:PSModulePath.split(';') | where{$_ -like "$($env:userprofile)*"}

# Increment Version Build
$modVersion = Test-ModuleManifest -Path "$($srcPath)\$($moduleName).psd1" | Select -ExpandProperty Version
$newVersion = [Version]::new($modVersion.Major,$modVersion.Minor,$modVersion.Build + 1,$modVersion.Revision)
Update-ModuleManifest -Path "$($srcPath)\$($moduleName).psd1" -ModuleVersion $newVersion

# Copy new build to Pull server
$compressedModuleName = "$($srcPath)\..\$($moduleName)_$($newVersion.Major).$($newVersion.Minor).$($newVersion.Build).$($newVersion.Revision).zip"
Compress-Archive -Path "$($srcPath)\*" -DestinationPath $compressedModuleName
Copy-item -Path "$($srcPath)\..\$($moduleName)_$($newVersion.ToString()).zip" -Destination $pullSrv
New-DscChecksum -Path "$($pullSrv)\$($moduleName)*.zip"

# Copy new build to local Modules folder
if(Test-Path -Path "$localPath\$moduleName")
{
    Get-ChildItem -Path "$localPath\$moduleName" -Recurse | Remove-Item -Recurse -Force -Confirm:$false -ErrorAction SilentlyContinue
    Remove-Item -Path "$localPath\$moduleName" -Recurse -Force -Confirm:$false -ErrorAction SilentlyContinue
}
Copy-Item -Path $srcPath -Destination $localPath -Recurse

Annotations

Line 1-3: Specify the name of the module, the location of the module and the name of the Pull Server.

Line 5-6: Construct the source and destination paths

Line 8: To have the latest version of the vSphereDSC module on my station, I will also copy the latest version here.

Line 10-13: For each successive run of this script I increment the Build property of the Version by 1.

Line 16-19: The module needs to be stored on the Pull Server, zipped and with the Version in the filename. And again we need to create a checksum for the module file.

Line 22-27: Copy the new version locally

Once this is done, you start compiling and testing Configurations with the new version of the vSphereDSC module. See also the previous

Build Server

The automation of these steps will be automated. Documentation to follow !

With this information I hope it will be easy to start using and contributing to the vSphereDSC module !

Enjoy !

A vSphere Folder is a resource which can exist rather independently in an existing vSphere environment. You can easily create some test Folders to get the hang and feel of the vSphereDSC module and play with DSC Configurations based on this vSphereDSC resource.

Lay of the Land

In the Intro of this series I already mentioned some of the issues I encountered, and how these had impact on the vSphereDSC module setup. It is my intention to gradually build up the vSphereDSC module, meaning that I will keep adding DSC Resources in future posts, but I will also be providing Unit and Integration tests (with Pester) and automate the module’s build process with Appveyor.

The posts will be primarily based on a DSC Pull Server set up. This because that corresponds with what I use in my test environment, but also because I consider that architecture as the “automated” version of a DSC setup. That’s the one that you will be using in a real environment later on. This doesn’t mean that you can not test resources and Configurations without having to wait for the LCM interval to expire, as I will show later in this post.

The vSphereDSC module uses class-based DSC resources, that means you need to have WMF 5 to use the module. All my tests have been done with the WMF 5 February 2016 build on the Pull and vEngine servers, so you will need to install that version as a minimum. On the Pull server and the vEngine server, which are Windows 2012 R2 servers, I have build 5.0.10586.117.

On the Windows 10 station, that I use to write the Configuration files, I have 5.0.10586.122. Note that WMF 5 on Windows 10 is an official release, not a preview.

As a side note, there is a Wiki page (maintained by Jan Egil Ring) that lists the PowerShell builds.

Until now I have been doing my development of the vSphereDSC module from a private BitBucket repository. But as of this post, the documented parts of the vSphereDSC module will be available on Github, and from the PSGallery.

The Layout

As I already described in the Intro, the file structure of the vSphereDSC module is quite straightforward. Some design decisions were dictated by the issues that I encountered. The structure on Github now looks like this

$env: psmodulepath (folder)
    |- vSphereDSC (folder)
        |- vSphereDSC.psm1 
        |- vSphereDSC.psd1
        |- vSphereDSCHelper.ps1
        |- Tools (folder)
            |- Get-TargetGuid.ps1
            |- Test-DscConfiguration.ps1
            |- vEng-LCM-Configuration-Guid.ps1
            |- vSphereDSC-Module-To-Pull-Server.ps1
        |- Examples (folder)
            |- PowerCLI
                |- pcli01-PowerCLI.ps1
            |- VmwFolder
                |- cf01-vCenter-Folder.ps1
                |- cf02-vCenter-Folder.ps1
                |- cf03-vCenter-Folder.ps1

The actual code for the vSphereDSC module is located in the vSphereDSC.psm1 and vSphereDSCHelper.ps1 files.

The reason why the Helper function are in a separate file and why this is not a nested module were already explained in the Intro. In short, we need to keep the PowerCLI cmdlets outside the class methods, and due to PowerCLI PSSnapin we can not use a module. As a bypass, the vSphereDSCHelper.ps1 file is dot-sourced in the vSphereDSC.psm1 file. This looks something like this

. "$($PSScriptRoot)\vSphereDSCHelper.ps1"

The Code

If we look at the VmwFolder class, we can distinguish three distinct areas. Note that I use the Region directive to organise my code. This also allows me to show/hide the individual parts.

Class Properties

The Class Properties are just that, the properties that will be present in the VmwFolder class. But it is also the location where you specify the attributes of these properties. The most important one being the Key attribute. In fact all properties marked with the Key attribute will be taken together to form the “key” for a specific instance of the VmwFolder class. This “key” shall uniquely identify each instance of the VmwFolder class.

In the case of VmwFolder, the properties Path and Name are not sufficient to uniquely identify a VmwFolder instance. The reason is that we can have “Yellow” and “Blue” folders with the same name, who appear to be in the same location. The reason they appear to be in the same location is due to the hidden folders that exist in a vSphere environment.

Take for example the following situation where we seem to have a folder with the name Folder2, located under the Datacenter Homelab. From the Web Client we see the following. Is this one object or are these two different objects ?

The old C# Client, where the “Yellow” and “Blue” foldernames originated, seems to say it are two different objects, provided we take the colour as a distinghuising characteristic.

The answer, these are two different objects. And it is all due to the “hidden” folders that we could be confused. In a Datacenter object, there are pointers to the Hosts and Clusters view and the VM and Templates view. These pointers go to the hidden folders “host” and “vm” which are the respective parents for these two Folder objects that have the same name.

And that is why the Type property is a Key property for the VmwFolder class.

DSC Methods

With this I mean the three methods that DSC requires to be present in each DSC class-based resource: Test, Set and Get.

As a general rule, I try to keep the code in these methods limited to the actual function of these three methods. For example, in the Test method, the code first checks of the folder exists (line 76), then depending on the desired state ($this.Ensure), the code returns $true or $false.

Note how on line 72, the vSphereDSCHelper.ps1 file is dot-sourced. This file contains all the code where we use PowerCLI cmdlets. In this particular case we needed to that, because in line 74 we call the Helper function Connect-VmwVIServer, which uses PowerCLI cmdlets.

Class Helper Methods

In this region we code all the methods that contain the more advanced code. While the Test, Set and Get methods were limited to their basic functionality logic, these Class Helper methods contain the “deeper” code. As an example, let’s have a look at the TestVmwFolder method.

In lines 106-111 we handle the logic that composes that path to the node, in this case a Folder, that we are looking for. This path is composed of the Path and the Name property. We need to code the special case where the node could be located in the RootFolder.

When the node is found, we need to check if the type corresponds with the Type of the Folder we are looking for (lines 114-115)

It’s this kind of logic that we want to keep out of the standard Test, Set and Get methods.

Note that there are quite a bit of Write-Verbose lines in the code. This use of the Write-Verbose cmdlet is indispensable when developing DSC resources in my opinion.

External Helper Functions

These are all the functions that use PowerCLI. As already explained, we can’t use PowerCLI inside the class methods. The solution I have chosen is to keep these functions outside of the class in a separate .ps1 file.

Most of the functions in there are rather straight-forward PowerCLI-based functions. Some of these were in fact already discussed in some other posts on this blog. See Universal PowerCLI Loader, Get-InventoryPlus – Inventory of all vSphere objects and vSphere Object by Path.

Verbosity

Perhaps an interesting point to make in relation to the Write-Verbose I made earlier. We do want to see “verbose” output from the class methods and the Helper functions, but we don’t really want to see “verbose” output from all the PowerCLI cmdlets.

Instead of suppressing the verbosity on each individual cmdlet where we don’t want it, I opted for using the $PSDefaultParameterValues at the in the module.

That way I avoid the many lines that would scroll by during the loading of the PowerCLI modules and PSSnapin.

Tools

In the Tools folder I will include some scripts that I use during the development/test phase of the vSphereDSC module. The purpose and the proper use of these scripts will be explained in a separate post on the Principles of Operations.

Examples

PowerCLI

To start working with the vSphereDSC module we need to deploy PowerCLI on the vEngine Server. And obviously we are going to do that with a DSC Configuration file. The sample file is named pcli01-PowerCLI.ps1.

The Configuration I provide to push PowerCLI to the vEngine server, is similar to what Vinith Menon posted in his Automate PowerCLI Install across your Windows Environment using PowerShell DSC post. Note that the Configuration file uses the Start-DscConfiguration cmdlet to immediately start the installation of PowerCLI on the vEngine server.

If all goes well you should see “PowerCLI installed” in the verbose output.

VmwFolder

In the vSPhereDSC module a number of Configuration sample files are included.

Note that in this stage you will still need to provide the credentials of the vCenter account in clear text !

The samples are rather straight-forward. In the Principles of Operations I will show how you can organise smooth testing of your Configurations.

This concludes the introduction of the VmwFolder resource.

Since this is the first actual vSphereDSC resource that is made public, I would love to hear your feedback. Use the Issues option on the Github repository to report feedback, bug requests, feature request…

Enjoy!

Since WMF 5 has been made available in preview, and still is in RTM at the moment I’m writing this, there have been constant changes to the way I was writing the DSC resources for vSphere. Since the February 2016 WMF 5 release, I now have a (somewhat) stable, working class-based solution. At least, that is what my initial tests seem to indicate.

This intro for my vSphereDSC series, will lay out the playing field. I’ll explain the concept I’m using, show some of the issues I encountered and explain the layout of the vSphereDSC Resource module.

_{* “Wisely and slow; they stumble that run fast”, Romeo and Juliet, Act II, Scene III}

Intro

To start you’ll need some knowledge on what DSC (Desired State Configuration) is all about of course. This series of posts is not going to do that!

But don’t despair, there are many free, online resources available that explain way better than I ever could, how DSC works. To name but a few:

• On MSDN Windows PowerShell Desired State Configuration Overview
• On Simple Talk the 3-part series PowerShell Desired State Configuration
• The MVA Course Getting Started with PowerShell Desired State Configuration (DSC)
• On PowerShell.Org the The DSC Book
• Many DSC articles in the PowerShell Magazine are definitely worth reading, as are all of their articles btw.

Also check out the numerous blog posts and session recordings that have been made available. Google/Bing are your friends!

Not free, but there are also many DSC books available, and most of the major course vendors have one or more DSC Courses in their catalog nowadays.

I would also like to give a shout out to Brandon Olin, who is developing a DSC Resource module named POSHOrigin_vSphere. He uses a different angle to configure a vSphere environment through DSC. At the moment he has an impressive VM resource in his module.

The vEngine Concept

The LCM (Local Configuration Manager) is the DSC component that takes care of the “make it so” part. On regular intervals, or when you tell it to, it compares the Configuration, it has or has received, with the actual situation on the host.

If there are differences, and if the LCM was configured to correct this, the LCM will “correct” the drift. It will “Make it so!”

The first decision I had to make, was where are we going to run this LCM that will interact with a vSphere environment. An obvious choice would be to install it on the vCenter server, but there are some strong counter arguments for that.

• The vCenter Server, with the introduction of the VCSA (vCenter Appliance), is not necessarily running a Windows OS
• The vSphereDSC resource module should be able to configure any vSphere Server, that includes a stand-alone ESXi host
• For support and performance reasons we would probably not want to install additional “features” on a vSphere Server
• Changes to a vSphere Server (upgrades, migrations…) should not impact the vSphere DSC configuration and resources

The obvious choice, for me, was to introduce a “helper” server, the vEngine!

The LCM that configures/corrects the vSphere environment, will run on this vEngine Server. The vEngine Server only needs PowerShell and the PowerCLI modules/pssnapins. And it will “talk” with the vSphere Server via PowerCLI.

The workflow for this setup will look something like this

(1) The vSphere administrator makes sure he has the required DSC Resource modules on his workstation. This includes the vSphereDSC module. These modules can be retrieved from public or private repositories.

(2) The vSphere administrator creates a Configuration for his vSphere environment.

(3) The Configuration is pushed to the DSC Pull Server. The Configuration is stored there as a MOF file.

(4) The LCM on the vEngine will query the Pull Server at regular intervals to check if there is a new Configuration present. If there is, it will check if all the required resources for the new Configuration are present. If not, it will pull these DSC Resources from the Pull Server.

If there is no new Configuration, the LCM will use it’s local Current Configuration to “test” against the vSphere environment. If there is a drift, and if the LCM is configured to correct this, it will apply the Configuration on the vSphere environment.

Issues

During my work the vSphereDSC module I did encounter several “issues”. Some where caused by actual bugs that hopefully will be fixed when WMF 5 is released. Others were/are in fact caused by some of the design decisions in WMF 5. Remember that WMF 5 as we can currently use, is not yet the official release version.

A short summary of the issues (and the effect they had on the vSphereDSC module):

• The original idea to use Disconnected Sessions suffered from occasional crashes in the WMI that is running the LCM. After talking with the PowerShell Team this appears to be a know issue, but the fix is not yet present in the public WMF 5 build. So Disconnected Sessions are “On hold” for now.
• Class-based DSC resources, at the moment, do not support multiple files. That means all the vSphereDSC resources will be in one file. And that one file is configured in the RootModule parameter in the Manifest. Besides a less cleaner organisation, this didn’t have a big impact.
• The LCM resets the runspace each time a class method is invoked. As a consequence, there is no passing of information between successive invocations of class methods. This also caused issues with the PowerCLI Modules and PSSnapins. To fix this I moved all PowerCLI lines to a “helper” file outside the class methods. This way the number of  times we need to load the PowerCLI Modules and PSSnapins is also limited, and it improves the runtime of the class methods.
• A PSSnapin that is loaded inside a module is only visible inside that module. This caused some problems when I was still struggling with the previous issue. For now, the external, PowerCLI Helper functions are in one .ps1 file  that is dot-sourced from within the class methods. I will need to revisit this, and check if I can package the Helper functions in a module.
• Class-based DSC Resources do not support inheritance. I was hoping to use the same object hierarchy that is present in vSphere (for example ExtensibleManagedObject – ManagedEntity – VirtualMachine), but that had to be abandoned.

Module Layout

As you have gathered from the Issues paragraph, there have been some changes to my original plan. A quick summary of what I have and how I organised it.

In fact I try to follow the layout as described PowerShell\DSCResources as close as possible. The following is a quick treeview of part of my private repository. It shows the general layout, more details will follow in the posts that will discuss individual resources.

$env: psmodulepath (folder)
    |- vSphereDSC (folder)
        |- vSphereDSC.psm1 
        |- vSphereDSC.psd1
        |- vSphereDSCHelper.ps1
        |- Tests (folder)
            |- Integration (folder)
            |- Unit (folder)
        |- Tools (folder)
            |- Get-TargetGuid.ps1
            |- Test-DscConfiguration.ps1
            |- vEng-LCM-Configuration-Guid.ps1
            |- vSphereDSC-Module-To-Pull-Server.ps1
        |- Examples (folder)
            |- vmwFolder
                |- cf01-vCenter-Folder.ps1
                |- cf02-vCenter-Folder.ps1
            |- vmwDatacenter
                |- cd01-vCenter-Datacenter.ps1
            |- vmwVMHost
            |- vmwCluster

Use Cases

A question I regularly get when presenting the vSphereDSC module, is why would you use DSC to configure your vSphere environment ? They are already automating their vSphere environment through PowerCLI, why would they need to change that ?

I used to sometimes reply with “Why do people climb mountains ?”. But that was of course intended as an icebreaker. There are, in my opinion, enough reason to switch to configuring your vSphere environment through DSC.

Because I know each one of you will encounter the same question(s) when you propose DSC to configure your vSphere environment, I decided to start listing the arguments and some potential use cases in here. And my list is obviously far from complete, please contribute, and send me arguments and use cases you see.

• “Pets vs cattle”. Independent who coined the analogy first, this concept should be one of the major arguments you use in a DSC vSphere discussion. Scale out instead of scale up!
• “Scale * Complexity -gt Skill”. This comes from a TechEd 2013 presentation (MDC-B302) by  Jeffrey Snover and Kenneth Hansen. Declarative, autonomous and idempotent deployment, configuration and conformance of standards-based managed elements.
• “Change is the primary cause of outage”. Comes from the same TechEd presentation as the previous bullet. Automation, in this case through DSC, will allow a continuous service availability, a continuous release cycle and continuous deployment.
• Separation of Configuration and Environment data. You would only need to source control your master Configuration logic, and the Environment data individually.

This concludes the vSphereDSC Intro. The scene has been set
What’s next ?

In the next posts in this series, I will start expanding into the individual resources that are present in the vSphereDSC module.