Space Reclaimer for NetApp SnapDrive

SnapDrive for Windows 6.3, which was released last year, introduced support for VMDKs on NFS & VMFS Datastores

A couple of quick notes, you need Data ONTAP 7.3.4 or later to use block reclamation with SnapDrive for VMDKs.

You need to have VSC 2.0.1 or later installed, with the Backup and Recovery feature, and also SnapDrive (within the VM) must have network connectivity to the VSC (port 8043, by default) as well as Virtual Center.

Also, SnapDrive cannot create VMDKs for you, in the way it can create RDM LUNs. Instead, you have to create VMDKs the old fashioned way, but once they’re attached to the VM, SnapDrive will be able to manage them.

Okay, so I’ve got a VMDK (my C: Drive), which is in an NFS Datastore.

I copied 5GB worth of Data into the C: drive, then deleted it. This left my VMDK at 10GB in size.

So, Windows took up about 5GB, and the data (which is now deleted), was about 5GB – so let’s kick off space reclamation and see how much space I get back.

Right click on the VMDK, and select “Start Space Reclaimer”.

It will do a quick check, to see if it actually can reclaim any space for you.

The confirmation box reckons I get reclaim up to 3GB? Hmm, I was hoping for a bit more than that. Well, let’s run it anyway and see how well it does.

It’s worth noting the warning on here – while the VSC requires VMs to be shutdown in order to reclaim space, SnapDrive runs space reclamation while the VM is powered up – but, it will take a backseat to any actual I/O that’s going on, so you might want to run it in a low usage period.

 

So, I clicked okay, and it kicked off space reclamation – and it even gives me a nice little progress bar.

In my lab, it took about 3 minutes, and when it was done, it had shrunk my VMDK down to 5.6 GB.

 

So it was just being modest earlier when it said I could free up to 3GB!

In total, it has reclaimed 5.2GB – which is actually a little more than the data I copied in and deleted to start with!

NetApp SAN Boot with Windows

Thoughts and ideas concerning booting from SAN when we attempted this with our NetApp array.

1. SAN Boot is fully supported by MSFT.  The first thing that happened is that we were told that SAN boot is not supported and we could not get Microsoft support for this configuration.  It turns out that this is not correct.  SAN boot is fully support by Microsoft along with HW partners like NetApp.  This TechNet article fully outlines MSFT’s support for SAN Boot:  http://support.microsoft.com/kb/305547
2. Zoning is the #1 issue with SAN Boot on FC. In talking with NetApp support team (who were a HUGE help on this issue) the most common issue in SAN Boot from Fiber Channel is zoning.  Because zoning can be complex, this is the most likely cause of error.  We strongly recommend you check and then double-check your zoning before opening a support ticket.   In our case, the zoning for the server was correct, but we did make a zoning error on another server that we were able to correct on our own.
3. Windows MPIO support at install time is limited. Because WinPE is not MPIO aware, there can be strange results when deploying against a LUN that is visible via multiple paths.  Keep in mind that at install time, Windows boots to boot.wim which is running WinPE instead of a full Windows install.  After the bits are copied locally, windows reboots to complete the install and at this time Windows is actually running.  Because of this, NetApp support team recommends having only one path to the LUN at install time and then adding paths later once Windows is up and running and you can enable Windows MPIO.
4. AND YET…  MPIO is strongly recommended for SAN Boot.  Because a Windows host will blue screen if it’s boot lun dies, MPIO is strongly recommended for boot LUNs.  This is documented here:  http://technet.microsoft.com/en-us/library/cc786214(WS.10).aspx.  This can seem contradictory at first, but the bottom line is that MPIO is good, just add it later once Windows is up and running correctly.
5. Yes, but what about Sysprep?  It turns out that MPIO is not supported for Sysprep based deployments:  http://support.microsoft.com/kb/2504934.  So, again you need to configure MPIO post deployment when you are deploying against sysprep’d images.  In the case of NetApp, we strongly recommend using Sysprep boot LUNS which you can then clone for new machines.  This significantly shortens deployment time as opposed to doing a full Windows install for each new host.
6. It’s all about BIOS. Actually installing windows on a boot LUN does require that Windows Setup sees your target LUN as a valid install path.  This means that the server must report this drive as a valid install target or Setup will not let you select this disk.  For FC, you will need to enable the BIOS setting and select your target LUN in the HBA setup screen.  This process varies by vendor.  Then you need to make this disk your #1 boot target in your server’s BIOS.  Again, this process varies by manufacturer.  As noted above, you should only establish one path.  This includes dual port HBA’s.  Only configure one of the ports.
7. Where’s my disk? Once you do all the above correctly, Setup may still refuse to show you the disk.  This could be because the correct driver is not present on the install media.  One way to fix this is to inject drivers into your Boot.WIM and Install.WIM.  This process is required if you are using WDS but optional if you are hand building a server from DVD or ISO.  In our case, we were building a single master image that we were going to Sysprep so we simply inserted the media and added the drivers manually during setup.
8. OK, the disk is there, but I can’t install! One funny thing about Windows setup is that if you are installing from DVD, that DVD must be present to install (duh).  This is fine, unless you used the process above to insert the driver.  To do this, you need to remove the disk.  Then you get the drives and click install.  Windows will fail to install with a fairly obtuse error.  You need to remove the drivers DVD at this point and put the install DVD back in.  Seems obvious, but it took me a few minutes to figure out what was wrong the first time I tried it.

The Windows Host Utilities Installation and Setup Guide (http://now.netapp.com/NOW/knowledge/docs/hba/win/relwinhu53/pdfs/setup.pdf) has a very detailed description of Windows support for FC LUNs and there is a step by step process in this guide for configuring Boot LUNs.

Avoid Server 2008 VMtools and VUM Woes with PowerCLI

Last week after upgrading to vSphere 4.1U1 I noticed a lot of our guests did not have the proper VMtools installed. After a quick look I realized they were all Windows Server 2008 or 2008 R2 guests. The initial update for all of the guest was done using VUM, but the tools install was completely hung on all those systems.

Prepare to wait.

Apparently VUM triggers the “Interactive Services Dialog Detection” in Windows which looks like the message below.

Just login and click this on all your guest; you’ll be done by update 2.

Luckily there is an incredibly easy workaround. Using PowerCLI you can type 2 commands to update your VMtools install without triggering this nasty little message.

Get-VM | Update-Tools

If you don’t want the machines to reboot just add -NoReboot to the end of the Update-Tools command. Here is the syntax for pulling only Windows 2K8 guests in a cluster named QA.

Get-Cluster "QA" | Get-VM | where {$_.Guest -like "*Server 2008*"} | Update-Tools

Set NetApp NFS Export Permissions for vSphere NFS Mounts

One of the things missing from the NetApp VSC is the ability to set permissions on NFS exports when you add a host to an existing cluster.  If you have a lot of NFS datastores and don’t feel like setting permissions across NetApp arrays when you add a new host this should ease the pain.  Here are a few other use cases.

1. You change a VMkernel IP for NFS traffic on a host
2. You add a VMkernel IP for NFS traffic on a host
3. You add a new host to a cluster
4. You remove a host from a cluster

You’ll see removing host is a reason to run this script.  This is an important thing to note.  Running this script will replace existing NFS export permissions with those associated to the vCenter you run it against.  If you have any additional IP addresses assigned to the export they will get blown away by this script!  I also thought it would be cool to turn this into a form so I used PrimalForms to design a very simple front-end you can see below.

The DataONTAP PowerShell Toolkit 1.2 has support for networking, but we don’t have any systems running 7.3.3 or greater so I wasn’t able to make use of those cmdlets in this script.  Because of that I hard code the NetApp VIFs.  Additionally the way I parse the data is related to the length of the VIF used, and I have no support for VIFs of different lengths.  The VMkernel ports for NFS are found using a wildcard search for “NFS” in the port group name.

Don’t be intimidated by all this code, 99% of it was generated by PrimalForms in order to build the GUI.  Modify the 5 variables up front to add your NetApp VIFs and controller names.  You can make a simple batch file to call the script and run it with just a desktop icon to get a nice easy way to modify your NFS permissions on vSphere.  Thanks to @jasemccarty and @glnsize for help with finding the NFS mount in vSphere!

 

$array1VIF = "10.1.1.40", "10.1.1.41", "10.1.1.42", "10.1.1.43"
$array2VIF = "10.1.1.44", "10.1.1.45", "10.1.1.46", "10.1.1.47"

$array1Name = "netapp1"
$array2Name = "netapp2"

$vCenters = "server1", "server2"

$vifLength = $array1VIF[0].Length
$volStart = $vifLength + 9

#Generated Form Function
function GenerateForm {
##############################################################
# Code Generated By: SAPIEN Technologies PrimalForms
#(Community Edition) v1.0.8.0
# Generated On: 10/24/2010 9:34 PM
# Generated By: theselights.com
##############################################################

#region Import the Assemblies
[reflection.assembly]::loadwithpartialname("System.Windows.Forms") | Out-Null
[reflection.assembly]::loadwithpartialname("System.Drawing") | Out-Null
#endregion

#region Generated Form Objects
$form1 = New-Object System.Windows.Forms.Form
$cancelButton = New-Object System.Windows.Forms.Button
$okButton = New-Object System.Windows.Forms.Button
$groupBox1 = New-Object System.Windows.Forms.GroupBox
$vcenter = New-Object System.Windows.Forms.ComboBox
$groupBox2 = New-Object System.Windows.Forms.GroupBox
$nfsDatastores = New-Object System.Windows.Forms.ListBox
$InitialFormWindowState = New-Object System.Windows.Forms.FormWindowState
#endregion Generated Form Objects

#----------------------------------------------
#Generated Event Script Blocks
#----------------------------------------------
#Provide Custom Code for events specified in PrimalForms.
$handler_vcenter_DropDownClosed= 
{

Connect-VIServer $vcenter.SelectedItem

$nfsDS = get-datastore | where {$_.Type -eq "NFS"} | get-view | select Name,@{n="url";e={$_.summary.url}}
$nfsDS | % {$nfsDatastores.Items.Add($_.URL.substring($volStart)) | Out-Null }

}

$handler_vcenter_DropDown= 
{

$nfsDS | % {$nfsDatastores.Items.Remove($_.url.substring($volStart)) | Out-Null }
$nfsDatastores.Items.Remove("Select a Virtual Center to gather NFS mounts.")|Out-Null

}

$okButton_OnClick= 
{

$esxNFSIP = Get-VMHostNetworkAdapter -VMKernel | where {$_.PortGroupName -like "*NFS*"} | select IP -Unique
$esxNFSIP = $esxNFSIP | % {$_.IP}

Foreach ($ds in $nfsDS) {

 $nfsVIF = $ds.url.substring(8,$vifLength)
 $nfsMount = $ds.url.substring($volStart)
 $nfsName = $ds.name

 #//// Set permissions on source NFS exports

 $array1VIF | % { If ($_ -eq $nfsVIF) { $storageArray = $array1Name } }
 $array2VIF | % { If ($_ -eq $nfsVIF) { $storageArray = $array2Name } }

 Connect-NaController $storageArray

 Set-NaNfsExport $nfsMount -Persistent -ReadWrite $esxNFSIP -Root $esxNFSIP

 }

}

$cancelButton_OnClick= 
{

$form1.close()

}

$OnLoadForm_StateCorrection=
{#Correct the initial state of the form to prevent the .Net maximized form issue
 $form1.WindowState = $InitialFormWindowState
}

#----------------------------------------------
#region Generated Form Code
$form1.Text = "Set VMware NFS Permissions"
$form1.Name = "form1"
$form1.DataBindings.DefaultDataSourceUpdateMode = 0
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 344
$System_Drawing_Size.Height = 379
$form1.ClientSize = $System_Drawing_Size

$cancelButton.TabIndex = 5
$cancelButton.Name = "cancelButton"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 103
$System_Drawing_Size.Height = 23
$cancelButton.Size = $System_Drawing_Size
$cancelButton.UseVisualStyleBackColor = $True

$cancelButton.Text = "Cancel"

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 204
$System_Drawing_Point.Y = 328
$cancelButton.Location = $System_Drawing_Point
$cancelButton.DataBindings.DefaultDataSourceUpdateMode = 0
$cancelButton.add_Click($cancelButton_OnClick)

$form1.Controls.Add($cancelButton)

$okButton.TabIndex = 4
$okButton.Name = "okButton"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 103
$System_Drawing_Size.Height = 23
$okButton.Size = $System_Drawing_Size
$okButton.UseVisualStyleBackColor = $True

$okButton.Text = "Set Permissions"

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 45
$System_Drawing_Point.Y = 328
$okButton.Location = $System_Drawing_Point
$okButton.DataBindings.DefaultDataSourceUpdateMode = 0
$okButton.add_Click($okButton_OnClick)

$form1.Controls.Add($okButton)

$groupBox1.Name = "groupBox1"

$groupBox1.Text = "Virtual Center"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 265
$System_Drawing_Size.Height = 94
$groupBox1.Size = $System_Drawing_Size
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 42
$System_Drawing_Point.Y = 26
$groupBox1.Location = $System_Drawing_Point
$groupBox1.TabStop = $False
$groupBox1.TabIndex = 2
$groupBox1.DataBindings.DefaultDataSourceUpdateMode = 0

$form1.Controls.Add($groupBox1)
$vcenter.FormattingEnabled = $True
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 226
$System_Drawing_Size.Height = 21
$vcenter.Size = $System_Drawing_Size
$vcenter.DataBindings.DefaultDataSourceUpdateMode = 0
$vcenter.Name = "vcenter"
$vCenters | % {$vcenter.Items.Add($_) | out-null}
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 19
$System_Drawing_Point.Y = 35
$vcenter.Location = $System_Drawing_Point
$vcenter.TabIndex = 0
$vcenter.add_DropDownClosed($handler_vcenter_DropDownClosed)
$vcenter.add_DropDown($handler_vcenter_DropDown)

$groupBox1.Controls.Add($vcenter)

$groupBox2.Name = "groupBox2"

$groupBox2.Text = "NFS Mounts"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 262
$System_Drawing_Size.Height = 167
$groupBox2.Size = $System_Drawing_Size
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 45
$System_Drawing_Point.Y = 141
$groupBox2.Location = $System_Drawing_Point
$groupBox2.TabStop = $False
$groupBox2.TabIndex = 3
$groupBox2.DataBindings.DefaultDataSourceUpdateMode = 0

$form1.Controls.Add($groupBox2)
$nfsDatastores.FormattingEnabled = $True
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 226
$System_Drawing_Size.Height = 134
$nfsDatastores.Size = $System_Drawing_Size
$nfsDatastores.DataBindings.DefaultDataSourceUpdateMode = 0
$nfsDatastores.Items.Add("Select a Virtual Center to gather NFS mounts.")|Out-Null
$nfsDatastores.HorizontalScrollbar = $True
$nfsDatastores.Name = "nfsDatastores"
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 16
$System_Drawing_Point.Y = 24
$nfsDatastores.Location = $System_Drawing_Point
$nfsDatastores.TabIndex = 0

$groupBox2.Controls.Add($nfsDatastores)

#endregion Generated Form Code

#Save the initial state of the form
$InitialFormWindowState = $form1.WindowState
#Init the OnLoad event to correct the initial state of the form
$form1.add_Load($OnLoadForm_StateCorrection)
#Show the Form
$form1.ShowDialog()| Out-Null

} #End Function

#Call the Function
GenerateForm

Safely Virtualize Oracle on NetApp, VMware, and UCS

Virtualizing your Tier1 applications is one of the last hurdles on the way to a truly dynamic and flexible datacenter. Large Oracle databases almost always fall into that category. In the past a lot of the concern revolved around performance, but with faster hardware and support for larger and larger virtual machines this worry is starting to fade away. The lingering issue remains what is and what isn’t supported in a virtual environment from your software vendor?

Although Oracle has relaxed their stance on virtualization, they take the same approach that most do when it comes to support in virtual environments. Take for example the following excerpt from Oracle’s database support matrix: Oracle will provide support for issues that are known to occur on the native OS, or can be demonstrated not to be a result of running on the server virtualization software. Oracle may request that the problem be reproduced on the native hardware.

That last part is the killer for most companies. How could you quickly re-create a multi-terabyte database on physical hardware once it is virtualized if there is a problem? Luckily NetApp, VMware, and Cisco UCS provide a very elegant solution to address this issue. Let’s take a look at a simple diagram depicting a 10TB virtualized Oracle DB instance connected via 10GbE and utilizing Oracle’s Direct NFS client.

The guest OS has been virtualized and resides on the VMFS datastore, the vSphere host is booting from SAN, and the database is directly hosted and accessed on the NetApp array using NFS. Each data volume in the picture is connected using a different technology to illustrate protocol independence (outside of Oracle where NFS is used for simplicity of setup).

As you can see from the diagram the real challenge is re-creating that 10TB database in a way that is cost effective and fast. NetApp’s FlexClone technology allows the instant creation of zero space virtual copies. The process is similar to VMware’s linked clones, but NetApp does it with LUN’s or file data, and with no performance hit.

To build your safety net follow the steps below.

1. Create LUN on NetApp array
2. Create UCS Service Profile Template
3. Configure Service Profile Template and set to boot from LUN in step 1
4. Deploy Service Profile from template
5. Install same OS as virtualized instance (OEL 5.5 in this case)
6. Create FlexClone of Oracle files/volumes
7. Create exports and set permissions for newly created server
8. Configure OS with mount points designed for FlexCloned file/volume

At this point you have a full physical environment of that 10TB virtualized Oracle database. The diagram below shows what this looks like.

The next step is to clean this up since you don’t want this UCS blade occupied with the test environment.

1. Shut down the OS
2. Delete the Service Profile (not the template)
3. Delete the FlexClone(s)

Now in the event you have some nasty database issue, and Oracle tells you to reproduce the issue on physical hardware, you can listen on the phone as the support guys jaw hits the floor when you tell him to give you 5 minutes. The entire process can be scripted easily using the Data ONTAP and UCS PowerShell Toolkit, or using an orchestration tool of your choice.

Reserving a blade or two for this unlikely scenario may seem wasteful to some, but because of the flexibility of UCS you can quickly spin that hardware up into production for things like hardware maintenance without a performance hit or capacity on demand for your vSphere environment. With NetApp, VMware, and Cisco you can safely and efficiently take your company to a 100% virtualized private cloud environment.