This cmdlet returns a VMware.VimAutomation.Types.Host.VirtualPortGroup object, which contains a very useful property called Ports. In that property it lists all the guests that are connected to a port on the portgroup. That would be the solution to find unused portgroups, I thought. But while the vSphere Client also shows powered off guests that are connected, the Port property returned by the Get-VirtualPortgroup cmdlet doesn’t. 🙁
So the question at hand required a bit more scripting than I originally thought.