DSCR for VMware and you!

On December 13th 2018 the PowerCLI Team provided us with an early end-of-year present. The Desired State Configuration Resources (DSCR) for VMware are published, and they are open sourced!
If you missed the announcement, hold what you are doing, and go read the VMware PowerCLI blog post Getting Started with Desired State Configuration Resources for VMware right now!

The next question on your mind is probably “How can I contribute?“. Well, with the correct tools and some VMware PowerCLI knowledge, it turns out that this is not too difficult. What follows is my first attempt at contributing to the Desired State Configuration Resources for VMware.

Repositories, tools and clones

For most of you what follows might probably state the obvious, but I included all the steps for completeness.

The Repository

The DSCR for VMware are published in a GitHub repository. So you will need access to GitHub. Create yourself a GitHub account, see Signing up for a new GitHub account.

Once you are on GitHub, go to the DSCR for VMware repository. Since you don’t want to be messing with that code directly, create a fork of the repository. For instructions on that see Fork a Repo.

That completes the repository part. You now have a copy of the  DSCR for VMware repository under your own GitHub account.

Tools

I understand that you are eager to add your own code to the DSCR for VMware module, but there are a few other preparatory steps you will have to take before you can actually start coding.

Determine on which station you are going to do your coding. On this station you will have to install some tools and some PowerShell modules.

Git commands

Since you will be working with a GitHub repository, you will need the Git commands. You can download the installation file from the Git – Downloads page.

Git GUI (optional)

This is not really needed, but it can make your life a lot easier. Instead of entering all the Git commands from a prompt, you can go for a Git GUI. There are several of these available, each with its own pro and cons.

Personally I prefer to use GitKraken

Editor

Your choice of editor can be whatever you prefer to work with PowerShell code. My editor of choice is currently Visual Studio Code with the PowerShell Extension.

Modules

To contribute to the DSCR for VMware module, and follow the Coding guidelines, you will need to install the following PowerShell modules on your station.

A “work” copy

We are nearly there.

To start coding, you will need to clone your copy of the repository (remember the fork earlier on) to your station.

With a Git GUI this is a simple process, provided you have set up the connection to your GitHub account beforehand.

  1. Select the Clone option
  2. Select your GitHub connection
  3. Select the target Folder for the clone
  4. Specify the name of the clone folder
  5. Clone the repository

Since we don’t want to mess with our copy of the master, it is good practice to create a branch, and do our development work on that branch.

To create a branch, right click on the Master entry on the left side and then enter the name of your branch in the text box. In the example I create a branch named dev-lucd.

Once the branch is created, it becomes the active path. Meaning that when I open the local files, I’ll be working in the dev-lucd branch. And while I’m doing my coding, the master branch stays unchanged. That is one of the reasons we use repositories.

In summary, and to show how easy it is.

Coding

The Coding Guidelines are required reading!

Intro

Now let’s get coding, but first some basic information for first time users, or as a refresher for the others.

Editor

One of the nice features of the Visual Studio Code editor, is that you can open a folder. This way all files in this folder will be shown in the Explorer.

Selecting a file in the Explorer will open that file in an Editor window.

Git Integration

A nice and useful feature of the Visual Studio Code editor is that it offers builtin Git integration. From within the editor you have access to the most common the Git commands.

Visual Studio Code has many keyboard shortcuts. After a while you will know the important ones by heart. To get you started, here are shortcuts to some handy cheat sheets.

Windows

Linux

macOS

Resource Organisation

Due to current limitations, all DSC resources are contained in the module’s single .psm1 file, namely VMware.vSphereDSC.psm1.

The DSC resources that are exported, are specified in the module’s .psd1 file (VMware.vSphereDSC.psd1) under the DscResourcesToExport entry.

There is a third file, named VMware.vSphereDSC.Helper.psm1, which contains a number of “helper” functions.

The DSC resources are defined as PowerShell classes. This allows the important feature of class inheritance.

A DSC resource, defined as a class, needs to have at least three methods Get, Set and Test. You can find more information in Writing a custom DSC resource with PowerShell classes.

Our First DSC Resource

Background

As a proof of concept and to document the workflow, this post will document how we can add a DSC Resource for the Issues message and the MoTD message on ESXi nodes and on a VCSA.

The theory behind these two features, motd and issue, is quite straightforward, and there are ample blog posts around the subject. Have for example a look at Wiliam Lam’s Easily manage ESXi & VCSA SSH login banner & MOTD in vSphere 6.0 post.

It all comes down to setting or clearing the two advanced settings on the target ESXi node ir VCSA.

There is one caveat with these advanced settings, how do differentiate between clearing an entry and not passing a value in a parameter. One solution would be to make the property that defines the motd and issue, a mandatory property. But this brings other disadvantages, like obliging you to add this property to each configuration file.

I decided to go for a separate, also not mandatory, property that explicetely asks for clearing the content of the motd and issue setting. You will see how this was implemented later on in this post.

Class setup

From the Coding Guidelines we know how we have to set up our classes for the new resources.

The resource for the ESXi based resource inherits from VMHostBaseDSC and BaseDSC

The resource for the VCSA based resource inherits from BaseDSC.

As an example, new DSC resources for ESXi and VCSA would start like this. Note that this just a skeleton, you will most probably have to add properties and methods.

With the class inheritance, your class will have a number of inherited properties and methods. The following screenshot shows what you get from those two base classes.

Two examples

Expand an Existing Resource

The first release of the DSCR for VMware module already contains a resource named vCenterSettings. Let’s try to add the motd and issue entries to this class. As a first step, I add the properties I will need to control the configuration of the motd and issue settings.

This is only a partial extract of the vCenterSettings class properties. Also note how we added the paths to the advanced settings in two hidden properties ($MotdSettingName and $IssueSettingName).

Next we will have to update the existing methods already available in the class, in such a way that they can handle the two new advanced settings we are adding. We don not have to touch the three required methods Set, Test and Get, that every DSC resource class must have, since they are already written in a general way without any specific dependencies on the Advanced Settings.

One such method we have to adapt is the ShouldUpdatevCenterSettings method. The logic in this method is quite straightforward.

  • Get all the Advanced Settings
  • With the Name of the setting (see the hidden properties from earlier), the method retrieves current Value for the Advanced Setting
  • With the ShouldUpdateSettingValue method the requested value is compared with the actual value. This method returns a result Boolean with the outcome of that comparison
  • For the Motd and Issue settings, the MotdClear and IssueClear comes into play. As remarked earlier, this allows us to clear the content of advanced setting without the need to make it a Mandatory Property in the class
  • If any advanced setting needs to be changed, the method returns $true, otherwise it returns $false

The methods that actually change the Value of an Advanced Setting, take care of the MotdClear and IssueClear properties. In fact we use that fact that we can define separate methods depending on the number and type of parameters we pass to a method. So there is a separate method SetAdvancedSetting when we call the method with four parameters.

There are a number of other changes I had to make in the vCenterSettings class, but I leave it to the reader to further inspect those once my changes are merged in the repository.

A New Resource

There is currently no resource yet for Advanced Settings for ESXi nodes. In this new class I will try to create such a resource, and for starters only provide the motd and issue settings. It should be rather trivial to add other advanced settings later on.

Note how use two hidden properties ($IssueSettingName and $MotdSettingName) in the class to specify the path to both advanced settings.

The basic layout of the methods we need in the class are more or less copies of the methods we also found in the vCenterSettings class. The major difference is that I left out the ShouldUpdateSettingValue method, since it was not required for the motd and issue settings. The decision if a change is required is already made in the ShouldUpdateVMHostSettings method.

Debugging

So you have coded your new resource, or update an existing, and you want to try it out.

As a first step you would run your Pester Unit tests, and fix all the issues you encounter in there. There is a section later on on these Pester tests.

The next step would of course be that you are going to try and use your resource on a test environment. But, just like always happens to me, you might notice some errors and/or issues when you run your tests.

As I imagine that using and writing DSC resources might be new territory for some of you, I added this Debugging section. It is definitely not final, but just lists some of the handy tricks I discovered/learned while developing DSC resources.

How do you test these?

First, make sure that you have read Getting Started with Desired State Configuration Resources for VMware. The only comment I can make on that post is that for “Kyle and the beard” everything always works 🙂

For us mere mortals some handy tips.

  • Configure the LCM agent on the station you will test on to ignore configuration signing and to work in debug mode (that will force the LCM to reload the DSC module each time, which is handy when you are making changes to the module)
  • Use the Verbose switch when applying a configuration with Start-DscConfiguration. It provides you with a lot of information on what your DSC module is doing.
  • Write your own verbose messages in your DSC resource classes. With a simple Write-Verbose in your code, you can provide debugging information. I for example, add a message at the start of each method that shows the code has entered that method. You will find some examples in the verbose output above

Configurations

The ultimate goal of your DSC resource is of course to use it to configure and monitor parts of your vSphere environment.

As was shown, perhaps implicitly, in Getting Started with Desired State Configuration Resources for VMware the commonly accepted way to apply DSC configurations, is to separate configuration from environment data. See also Separating configuration and environment data.

In practice this comes down to having code, that is static, unless you add properties to your DSC resource class, and code that contains the actual configuration data. These two types of data can be combined in one .ps1 file, or they can be kept in two separate .ps1 files.

For our newly created DSC resource, we would have a configuration file, something like this. Note how this file works with parameters, and that nothing would need to be changed to this file, while we change the configuration itself, the so-called environmental data.

The environmental data is kept in a separate file. Something like this for example.

As you notice, we call the other file (VMHostSettings_Config.ps1). This script will generate the MOF file, and will apply the configuration with Start-DscConfiguration. Notice how I added the Verbose switch to have a better view on what is happening. Once your DSC resource is sufficiently tested and considered stable, you might remove the Verbose switch.

Since the proof is in the pudding, this is what a SSH session looks like after the configuration was applied.

For the vCenterSettings resource we can apply our configuration in the same way. First our configuration part.

Note that I left out the parameters for the settings that were already available in this vCenterSettings resource.

And then the environmental part.

And the result after applying the configuration.

Pester Tests

Unit Testing

As specified in the Coding Guidelines, you have to provide Pester Unit test scripts for the resources you add to the module. Unit tests test how the code flows through your methods.

From my own experience, all this Pester code looks daunting and complex at first, but at some point you ‘make the click‘, and from then on it all looks obvious and not too hard at all.

Let’s take a small extract from the Pester Unit tests I wrote for the VMHostSettings class. There is some structure and logic in such tests.

  • Your Unit test has to cover the Set, Test and Get functionality of your resource. Each of these is covered in one Describe block.
  • As not to suffer from relics from test 1 while doing test 2, you can reset your environment. This is defined in a AfterEach block
  • We group our tests in Context blocks. Such a Context block generally consists of a number of tests that start from the same situation. That could for example be, all tests that run with no parameters passed. The text that you specify on such a Context block, will appear on the Pester output.
  • Since you don’t want to run these Unit tests against a live environment, you are going to mock the cmdlets used in your methods.
  • In a mock definition you specify which cmdlet you are mocking and what this mock shall return to your tests. Most of the time this is minimal information, and just enough that the methods you are testing can proceed.
  • These mock definitions are provided in a BeforeAll block, meaning that these mock definitions are done before any actual tests.
  • Since we are testing a class, we have to create a minimal representation of such a class for the tests. This is done with the New-Object cmdlet, and the resulting object is stored in the variable $resource.
  • Now we can define the actual tests. These are defined in It blocks. Add a meaningful description of the test, this is the text you will see appearing, hopefully in green, when the Unit test are run.
  • Since we are in the Set testing part, remember the Describe from earlier on, we have to call the Set method on the class with $resource.Set()
  • A series of tests will verify that all cmdlets that you expect to be called are called. This is defined with the Assert-MockCalled. There are multiple options on that command, including how many times you expect the (mocked) cmdlet to be called.

The DSCR for VMware provides a script to run all the Unit tests in one run, but sometimes you might have the need to run a specific Unit test. That can be done by just running that single test.

Go to Tests folder, and start the test.

If all goes well, you should see something like this.

Integration Testing

Integration tests verify that your code actually does what it is supposed to do, by running it against an actual environment. There is no mocking involved, when your code is supposed to create a VM, the Integration test will verify that the VM is actually created.

All the Pester testing will eventually be incorporated in CI/CD pipeline. In that pipeline you will specify how the test environment for your Integration test can be reached. That is why you will notice that driving scripts for this testing requires (mandatory) some parameters like:

  • Server: which vSphere Server
  • Name: only for tests that run against a VMHost resource
  • User/Password: the credentials to connect to Server

For an Integration test you will need at least two files:

  • Pester script: this contains the tests
  • Configuration file(s): this contains the DSC configurations you are testing with

Pester script

This Pester script contains the actual Integration tests that will be executed. The filename is <DSC-resource-name>.Integration.Tests.ps1.

In that file, under the Describe block, you have a number of Context blocks. Each Context block describes tests for one or more of the properties available in the DSC resource.

In the Context blocks, you have a number of It blocks. In these It blocks you perform that actual tests with the DSC resource.

A minimal set of It blocks for a Context are:

  • Does the Configuration create a MOF and is it applied without any errors?
  • Does applying the Configuration have the expected result?
  • Does a Test with the Configuration show compliance (is the result $true)?

A sample Describe block looks like this

Configuration file

This Configuration file contains one or more DSC Configuration definitions that you are testing with the Pester script.

The filename follows this layout, <DSC-resource-name>_Config.ps1.

In the file there are one or more Configurations that allow you you test all aspects of your DSC resource. In the following example we test the motd and issue properties of the VMHostSettings resource.

Submit your Code

Once you have your new DSC resources ready, tested and Pester tested, you are ready to submit to the repository.

First, stage all the files that you changed on your station, to your repository. Make sure to clearly document what the changes are all about.

One requirement is that all your commits shall be signed. First you will need to set up global settings for your account and email.

And verify the settings with

For each commit you make to your repository, you can now “sign” it with

This will open an editor window with the commit content, including the signoff.

Once your repository has the changes and you made sure all your commits are signed, you are ready to create your Pull Request (PR) against the vmware/dscr-for-vmware repository. Make sure to follow the instructions in the CONTRIBUTING document.

Some important points:

  • the base is the VMware repository
  • open the PR against the dev branch
  • make sure to select the branch with your latest changes on your side (here dev-lucd)

Once your PR is submitted, it will be reviewed by the PowerCLI Team. When they have questions, or want you make changes, you can follow that up in the comments of your PR.

When the review is completed, the reviewer will approve the merge of your code. And you will see your code being merged in the dev branch.

Epilogue

I intend this to be a living post, in other words any new information or any new experiences I gather, I will add them here.
And of course, if you have questions or remarks, feel free use the Comments or my Contact Form.

Enjoy!

One Comment

    DSCR and Pester testing - LucD notes

    […] my previous post DSCR for VMware and you! I described the open source project that was started to develop DSC resources for vSphere (DSCR). […]

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.