Hyper-V

Performance Monitoring Hyper-V Part 1 – Setup

April 2, 2012April 2, 2012 Jeff Miles Leave a comment

There are many sites out there that document how to monitor Hyper-V performance, but only a few of them have any detail on the actual setup and results of the monitoring. Perhaps this post (and the next one coming) will assist you, or perhaps it will only be of benefit for my personal reference documentation.

Its sad to admit, but until this month I hadn’t spent any large amount of time looking at my Hyper-V cluster to check for performance issues because I’ve been so busy. With the addition of new staff at work (invaluable!) I’ve had the chance to get caught up on actual system administration.

In addition to the necessity of doing this because it’s the right thing to do, there’s a new software implementation that is being considered, and I wanted to make sure I wasn’t over-selling our cluster capabilities.

Real-time active monitoring

H yper-V Windows Gadget: Made by Tore Lervik. I found this tool about 2 weeks after my initial implementation. Right now I’m only using it for active monitoring of the guest CPU performance, but it has many other features that make it worth downloading.

Hyper-V Mon.exe: This was discovered while reading a post at the excellent Hyper-V.nu by Peter Noorderijk. The actual post detailing this (and other monitoring which I’ll get to) is here. I really like this tool because it gives the actual CPU utilization of your hosts, in addition to other data.

Scheduled Monitoring and Analysis

A very useful tool that I discovered through the Hyper-V.Nu blog post is the “Performance Analysis of Logs (PAL)” tool. It can be found here on codeplex. This tool provides the ability to set a Performance Counter profile, export it to a Data Collector Set, and then import the results for analysis.
I won’t detail how to set up and use the tool in full here, as it’s been covered by Peter at the Hyper-V.nu link above, however there are a couple things to mention.

If you’re trying to use this with Hyper-V Server (as opposed to Windows Server with Hyper-V role) you’ll find that you can’t just run Performance Monitor to import that data collector set; instead you’ll need to use the Logman command.

But before you do that, you must modify your exported XML template in a text editor, because the Logman command is going to throw an error unless you don’t. When you open it up, look at line 5 & 6:

<Name>PAL_Microsoft_Hyper-V_R2_SP1</Name>
<DisplayName>@%systemroot%\system32\wdc.dll,#10026</DisplayName>
<Description>@%systemroot%\system32\wdc.dll,#10027</Description>

For some reason, logman doesn’t like the dynamic DisplayName and Description that are used by default. Change these to some static value, and save the xml file.

Next, copy the xml file to your Hyper-V host, and then remote into the host and run the following from the command line:

logman import Hyper-V_Monitor -xml "c:\Hyper-v_Counters.xml"

Then you can start the counters with:

logman start Hyper-V_Monitor

By default the results will be saved in C:\PerfLogs\System\Performance on your host. If you want to schedule the start and stop, you could use schtasks.exe to schedule the logman command.

Once you have the output from performance monitor, you can load it into the PAL tool as described at Hyper-V.nu, and view your results.

In part two, I’ll review my results and what I’ve found about them.

MD3220i firmware update

January 23, 2012January 23, 2012 Jeff Miles 2 Comments

There isn’t too much out on the interwebs about the MD3220i, so I thought I’d share my experience with updating the firmware.

Before the Christmas break Dell called and mentioned that a very important firmware update had been released, and it should be done as soon as possible. Due to scheduling, last week was when I scheduled a maintenance window to do it.

I was previously on firmware 10.70, so to fully update to 10.80 I need to use the bridge firmware to get to 10.75 first.

Luckily I realized that I needed to update the “PowerVault Modular Disk Storage Manager Client” before doing the firmware update. I uninstalled the previous version, and then reinstalled from this package:

ftp://ftp.dell.com/FOLDER88591M/1/DELL_MDSS_Consolidated_RDVD_3_0_0_18_A00_R314542.iso

However, installing from the setup.exe in that ISO, I had lost the actual “PowerVault Modular Disk Storage Manager Client”; no where to be found.

Eventually I figured out that from the install source, you also need to go into one of the sub directories, and run a different executable to get the MDSM Client.

Of course, I started drafting this post while I had that install source, and now I realize I must have deleted it so I can’t give the exact path of the executable that is needed. If I download the package again, I’ll make sure I update this post with the right path.

Once I had the MDSM updated, I updated to the bridge firmware first, and then the 10.8 firmware second. During this process the MDSM gave specific information about which controller it was updating, and all of my resources running from the MD3220i went uninterrupted.

Firmware updates are always a little stressful, even more so when the company SAN is the one undergoing an update; this one was smooth and painless which I was very glad for.

My Hyper-V cluster logs went wild though, reporting failures of the reach-ability of all hosts. Nothing to worry about since there’s dual controllers, but something to be aware of.

Hyper-V Cluster and Windows Server licensing

January 16, 2012January 16, 2012 Jeff Miles Leave a comment

Thanks to Aidan Finn and this blog post, I’ve recently discovered that how I planned the licensing for my Hyper-V cluster is incorrect.

With two Dell R410’s in a Hyper-V cluster, I purchased two licenses of Windows Server 2008 R2 Enterprise. I figured that as long as I didn’t run more than 4 VMs per host, I’d be good. In the case of a Live Migration or fail over, the VMs weren’t permanently assigned to the other host so it’d be alright. For future VMs, one off purchases of Server Standard would be made to allow for their use.

I really don’t understand why Microsoft doesn’t make it more obvious that Datacenter really is what’s necessary to run a Hyper-V cluster. Unless you’re running 4 VM’s or less across two hosts, Datacenter is the only way to go. Any more than that, and you’ll be doubling up on Enterprise licenses for each host, which by that point you might as well buy Datacenter. For a two host Hyper-V cluster this will cost between $12,000-15,000.

I’m assuming this is just as much of a problem for ESXi clusters too; apply a Windows license to each host even though it isn’t running Windows, so that during failover, the licensing is legal.

At least now I know that this is incorrect, so for 2012 fiscal two Windows Server Datacenter licenses will be purchased. Of course with Windows 8 around the corner, I’ll be getting these licenses with Software Assurance (assuming Windows 8 isn’t out yet).

Thinking about Datacenter has got me really excited actually, because of the additional Hyper-V features, future improvements to DFS and DFSR (which I’m anxiously awaiting news of) and updated OS for WDS, RDS and a bunch of other services.

Ubuntu on Hyper-V issues with Integration Services

March 11, 2011March 11, 2011 Jeff Miles 1 Comment

I’ve set up a new VM with Ubuntu 10.10 server, running on our Hyper-V 2008 R2 SP1 cluster.

I followed the excellent walkthrough by Ben Armstrong here to make sure I wasn’t missing anything.

However after the install completed, I immediatly I ran into problems. Following the instructions to enable the Integration Services, after a reboot there was still no network connectivity, and I started seeing strange errors during the boot process mentioning Call Trace errors.

As well, the CPU on the VM was running at 100% constantly.

After further reboots, the VM would lock up entirely and become unresponsive.

I restarted the process, but immediately after first login made a snapshot and started looking.

Running the “top” command, I found the process ksoftirqd taking 100% of the CPU. Looking into that brought me to this forum post:

http://ubuntuforums.org/showthread.php?t=1494797

It sounds very similar to my environment, as I’m using Dell R410 with Broadcom NIC’s.
Following those instructions, I was able to disable the integration components and add a legacy network adapter. Now the VM is running just fine.

Of course, I’m going to have to set up NTP now, and accept a bit of lower performance, but in this instance, thats fine.

If I ever do find out the source of this issue, I’ll edit this post.

Find Hyper-V VM GUID

March 8, 2011March 8, 2011 Jeff Miles 1 Comment

While creating and converting VM’s using SCVMM puts the configuration files and VHD’s in a nicely structured set of folders, if you create a VM through Hyper-V Manager or convert using disk2vhd, the files will be stored based on the GUID for the VM, which isn’t found anywhere in the GUI tools for Hyper-V.
Luckily I came across this blog post to which I owe full credit for the solution:

http://blogs.technet.com/b/m2/archive/2008/07/04/how-to-get-the-bios-guid-from-a-hyper-v-vm.aspx?ppud=4&wa=wsignin1.0#comments

Save the VBS script below as guid.vbs, copy to your Hyper-V host, and use psexec to execute it with this:

psexec \\hyperv cmd.exe

cscript c:\guid.vbs

You are using PSexec right?

Option Explicit
Dim WMIService
Dim KvpComponents
Dim VMList
Dim VMSettingList
Dim VM
Dim item
Dim setting
Dim component
 
'Get instance of 'virtualization' WMI service on the local computer
Set WMIService = GetObject("winmgmts:\\.\root\virtualization")
'Get all the MSVM_ComputerSystem object
Set VMList = WMIService.ExecQuery("SELECT * FROM Msvm_ComputerSystem")   
For Each VM In VMList  
 if VM.Caption = "Virtual Machine" then      
  WScript.Echo "========================================"      
  WScript.Echo "VM Name: " &amp; VM.ElementName      
  WScript.Echo "VM GUID: " &amp; VM.Name     
  WScript.Echo "VM State: " &amp; VM.EnabledState   
 
  ' Now get the BIOS GUID for this VM
  Set VMSettingList = WMIService.ExecQuery("SELECT * FROM Msvm_VirtualSystemSettingData")   
  For Each setting In VMSettingList
   Dim tempVMname
   tempVMName = "Microsoft:"  + VM.Name
   if setting.InstanceID = tempVMName then      
    WScript.Echo "VM BIOS GUID: " &amp; setting.BIOSGUID  
   end if
  Next
 end if
Next