Recently I had the pleasure of doing a guest post, called Finding your way in the PowerCLI Community, on the PowerCLI blog. The subject of the post was how to find community threads, that might hold an answer to your question.
Now this wouldn’t be a PowerShell/PowerCLI blog, if I didn’t try to automate the procedure. And with a serious amount of RegEx involved, I was able to create some working code. Here it is, my Find-VMTNPowerCLI function.
Warning: pure PowerShell, no PowerCLI content !
The script
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 |
function Find-VMTNPowerCLI{ <# .SYNOPSIS Search the VMTN PowerCLI related communities .DESCRIPTION The function automates the functionality that is available through the Search page .NOTES Author: Luc Dekens .PARAMETER Question The question as you would enter it on the VMTN Search page .PARAMETER Start The start date of the period in which you are looking for threads that fitt your search. The default is December 24th 2007. .PARAMETER Finish The start date of the period in which you are looking for threads that fitt your search. The default is now. .PARAMETER ReplyStart The start date of the period in which you are looking for replies that fitt your search. The default is December 24th 2007. .PARAMETER ReplyFinish The start date of the period in which you are looking for replies that fitt your search. The default is now. .PARAMETER Author Filter the returned threads to those started by one of the authors. .PARAMETER ReplyAuthor Filter the returned threads to those where the reply came from one of the authors. .PARAMETER PowerCLI Switch, search the PowerCLI Community. The default for this switch is $true. .PARAMETER Onyx Switch, search the Onyx Community. The default for this switch is $false. .PARAMETER UpdateManager Switch, search the UpdateManager Community. The default for this switch is $false. .PARAMETER PowerShellers Switch, search the PowerShellers group. The default for this switch is $false. .PARAMETER AutomationTools Switch, search the AutomationTools COmmunity. The default for this switch is $false. .PARAMETER Descending Switch, when true the returned threads will be in a descending Reply Date order. The default for this switch is $true. .EXAMPLE PS> Find-VMTNPowerCLI -Question 'svmotion thin' .EXAMPLE PS> Find-VMTNPowerCLI -Question '5.0.1' -PowerShellers .EXAMPLE PS> Find-VMTNPowerCLI -Question 'regex get-view' ` >> -Start (Get-Date).AddDays(-14) #> param( [CmdletBinding()] [parameter(Mandatory = $true)] [string]$Question, [datetime]$Start = (New-Object DateTime -ArgumentList 2007,12,24), [datetime]$Finish = (Get-Date), [datetime]$ReplyStart = $Start, [datetime]$ReplyFinish = $Finish, [string[]]$Author, [string[]]$ReplyAuthor, [switch]$PowerCLI = $true, [switch]$Onyx, [switch]$UpdateManager, [switch]$PowerShellers, [switch]$AutomationTools, [switch]$Descending = $true ) function Get-VMTNSearch{ param( [string]$url, [switch]$Group ) $pst = [TimeZoneInfo]::FindSystemTimeZoneById("Pacific Standard Time") $t0 = Get-Date $t1 = [TimeZoneInfo]::ConvertTime($t0,$pst) $tdiff = $t0 - $t1 $regBodyText = [regex]'(?s)\[DocumentBodyStart.*?-->(?<bodyText>.*)<!--' $strAnswerAuthorGroup = 'jive-thread-reply jive-thread-reply-alt clearfix[\w\W]*?xxxxxxx[\w\W]*?<a href="/people/(?<user>.*)"' $strAnswerAuthorOther = '<a name="xxxxxxx[\w\W]*?<a href="/people/(?<user>.*)"' $regThreadAuthor = [regex]'jive-thread-post-body[\w\W]*?<a href="/people/(?<user>.*)"' $regThreadDate = [regex]'jive-thread-reply-date[\w\W]*?<span>(?<threaddate>.*?)</span>' $regReplies = [regex]'(?<replies>\d*) Replies' $web = New-Object System.Net.WebClient $answerFeed = [[xml]]$web.DownloadString($url) $answerFeed.rss.channel.item | Sort-Object -Property {[datetime]$_.pubDate} -Descending | Select @{N="ReplyDate";E={[datetime]$_.pubDate}}, Title, @{N="Text";E={ $bodyText = "" if($_.InnerText -match $regBodyText){ $bodyText = $matches['bodyText'] $bodyText = $bodyText -replace "</p>|<br/>","`n" $bodyText = $bodyText -replace '<blockquote class="jive-quote">|</blockquote>' $bodyText = $bodyText -replace "<.*?>" $bodyText = $bodyText -replace ">",">" $bodyText = $bodyText -replace " ","`t" } $bodyText}}, @{N="Reply";E={ Set-Variable -Name page -Value $web.DownloadString($_.Link) -Scope 1 $bookmark = $_.Link.Split('#')[1] if($group){ $regAnswerAuthor = [regex]$strAnswerAuthorGroup.Replace('xxxxxxx',$bookmark) } else{ $regAnswerAuthor = [regex]$strAnswerAuthorOther.Replace('xxxxxxx',$bookmark) } $user = "" if($page -match $regAnswerAuthor){ $user = $matches['user'] } [System.Web.HttpUtility]::UrlDecode($user) }}, @{N="Poster";E={ $user = "" if($page -match $regThreadAuthor){ $user = $matches['user'] } [System.Web.HttpUtility]::UrlDecode($user) }}, @{N="ThreadDate";E={ $threadDate = "" if($page -match $regThreadDate){ $threadDate = ([datetime]$matches['threaddate']).Add($tdiff) } $threadDate }}, @{N="Replies";E={ if($page -match $regReplies){ $matches['replies'] } else{0} }}, Link $web.Dispose() } $containerPowerCLI = 2530 $containerOnyx = 3498 $containerUpdateManager = 3482 $containerAutomationTools = 3102 $containerPowerShellers = 1013 $typePowerShellers = 700 $searchUrl = "https://communities.vmware.com/community/feeds/search" $questionText = "q=" + [System.Web.HttpUtility]::UrlEncode($Question) $peopleEnabled = "peopleEnabled=true" $containerType = "containerType=14" $container = "container=$containerPowerCLI" $dateRange = "dateRange=all" $numResults = "numResults=" + [int]::MaxValue $query = $questionText + "&" + $peopleEnabled + "&" + $containerType + "&" + $container + "&" + $dateRange + "&" + $numResults $url = $searchUrl + "?" + $query $result = @() if($PowerCLI){ $result += Get-VMTNSearch -Url $url } if($Onyx){ $url = $url -replace "(container=)(\d+)","`${1}$containerOnyx" $result += Get-VMTNSearch -Url $url } if($UpdateManager){ $url = $url -replace "(container=)(\d+)","`${1}$containerUpdateManager" $result += Get-VMTNSearch -Url $url } if($PowerShellers){ $url = $url -replace "(container=)(\d+)","`${1}$containerPowerShellers" $url = $url -replace "(containerType=)(\d+)","`${1}$typePowerShellers" $result += Get-VMTNSearch -Url $url -Group } $condition = '$_.ReplyDate -ge $ReplyStart -and $_.ReplyDate -le $ReplyFinish -and' $condition += ' $_.ThreadDate -ge $Start -and $_.ThreadDate -le $Finish' if($Author){ $condition += ' -and $Author -contains $_.Poster' } if($ReplyAuthor){ $condition += ' -and $ReplyAuthor -contains $_.Reply' } $sblock = $executioncontext.InvokeCommand.NewScriptBlock($condition) $result | where {&$sblock} | Sort-Object -Property ReplyDate -Descending:$Descending } |
Annotations
Line 63: The first thread in the PowerCLI Community dates from December 24th 2007. No reason to go further back in time 🙂
Line 77-149: An internal helper function. This is the real workhorse of the function, in here all the RegEx mgic takes place.
Line 80: The way Jive returns results for a community or a group is slightly different. Hence the Group switch, which will allow the function Get-VMTNSearch to act differently for the PowerShellers group.
Line 83-86: The ThreadData is returned in PST time (I suppose that is where the VMTN servers are located). These lines will calculate the time difference between the location where you run the script and PST time.
Line 88-93: The RegEx expressions that will be used in the function to extract the desired fields.
Line 96: The URL, which is passed from the calling function, contains the actual Search keywords. Here we download the results from the search and convert it to an XML list for further handling.
Line 97: The individual ‘items’ from the search result are retrieved and used to extract the data.
Line 103-108: The content of the reply post is interspersed with HTML code. The series of -Replace operators will convert and filter away the HTML tags.
Line 112: The function fetches the complete page for the thread. This required for extracting the properties that follow. The -Scope 1 parameter makes sure that the $page variable is available outside the calculated property code block.
Line 113-119: The RegEx string contains the bookmark for the specific reply we are looking at. That is why the script updates the RegEx string with the bookmark. The Reply field has a different layout depending if it comes from a Community or a Group.
Line 124,131: Some user names might contain blanks or other special characters, to avoid displaying the Javascript encoding for these special characters, the property value is decoded.
Line 136: The thread date is returned in PST time (see above). This converts the datetime value to your local time.
Line 151-155: All the VMTN Communities have a specific ID. This ID needs to be used in the search URL.
Line 156: The PowerShellers group has a specific type, different from the Community type.
Line 158: The basic Search page URL
Line 159: The search URL needs to have all special characters converted.
Line 160-164: The other parameters passed on the search URL
Line 164: On the Search page, the results are returned in batches of 10, 15 or 30. In the function we want all the results in one go, hence the maximum [int] value.
Line 171-186: The Get-VMTNSearch internal function is called for each of the selected communities.
Line 188-195: All the selected conditions are stored in a [string]
Line 196: The string with the conditions is converted in a code block
Line 197: The generated code block is used in the Where-clause to filter the results according to the passed parameters.
Sample usage
The function is quite simple to use. The Question parameter accepts the exact same search strings as you can enter on the Search page. See the VMTN Search Tips for the details.
A simple example that will look for the keyword ‘move-vmthin‘ in the PowerCLI Community.
1 2 3 4 |
$qtext = "move-vmthin" Find-VMTNPowerCLI -Question $qtext | Export-Csv .\search-results.csv -NoTypeInformation -UseCulture |
The resulting CSV file looks like this
Note that in the Text column you can find the complete content of the reply. This can make it quite easy to copy code to your favorite PowerShell editor.
The next example will look for the text ‘baseline groups remediate-inventory‘ only in the Update Manager PowerCLI Community.
1 2 3 4 |
$qtext = 'baseline groups remediate-inventory' Find-VMTNPowerCLI -Question $qtext -UpdateManager -PowerCLI:$false | Export-Csv D:\Tools\PowerShell\Scripts\search-results.csv -NoTypeInformation -UseCulture |
The result
With the search functionality, you could for example look for replies that contain functions. Searching for the words ‘function param‘ we would find quite a number of replies that contain code.
1 2 3 4 |
$qtext = "function param" Find-VMTNPowerCLI -Question $qtext | Export-Csv D:\Tools\PowerShell\Scripts\search-results.csv -NoTypeInformation -UseCulture |
The resulting CSV file contains 167 rows. Perhaps a bit too much. So let’s limit the replies to a specific time period. We only look for replies after September 1st 2011.
1 2 3 4 5 |
$qtext = "function param" $start = Get-Date -Year 2011 -Month 9 -Day 1 Find-VMTNPowerCLI -Question $qtext -ReplyStart $start | Export-Csv D:\Tools\PowerShell\Scripts\search-results.csv -NoTypeInformation -UseCulture |
Much better, now we find 23 replies, with recent code.
A word of warning
I’m pretty sure there are some threads around that will break the function. With so many threads and replies in these very active PowerCLI related communities and groups, there are bound to be some that I didn’t encounter during my tests and that will break the function.
Should you encounter something like that, feel free to post the parameters in the comments for this post and I will try to fix the function.
Enjoy !
Ammesiah
I really like the warning :p
Great job btw, like always ^^