StoreVirtual – VMFocus

HP ConvergedSystem 200-HC StoreVirtual System – Questions Answered

Posted on 3 June 201511 January 2021 by Craig

Background

HP released two offerings of the HP ConvergedSystem 200-HC StoreVirtual System last year. Essentially they have taken ESXi, HP StoreVirtual VSA, OneView for vCenter and automated the setup process using OneView Instant On.

Two models are available which are:

HP CS 240-HC StoreVirtual System, this has 4 nodes each with:
- 2 x Intel E5-2640v2 2.2GHz 8 Core Processor
- 128GB RAM
- 2GB Flash Backed HP Smart Array P430 Controller
- 2 x 10GbE Network Connectivity
- 1 x iLO4 Management
- 6 x SAS 1.2TB 10K SFF Hard Drives
- Around 11TB of usable capacity

HP CS 242-HC StoreVirtual System, this has 4 nodes each with:
- 2 x Intel E5-2648v2 2.2GHz 10 Core Processor
- 256GB RAM
- 2GB Flash Backed HP Smart Array P430 Controller
- 2 x 10GbE Network Connectivity
- 1 x iLO4 Management
- 4 x SAS 1.2TB 10K SFF Hard Drives
- 2 x 400GB Mainstream Endurance SSD
- Around 7.5TB of usable capacity

These are marketed with the ability to provision virtual machines within 30 minutes.

What Does Provision Virtual Machines Within 30 Minutes Really Mean?

To answer this question you need to understand what HP have saved you from doing, which is:

Installing ESXi across 4 x Hosts
Installing vCenter to a basic configuration
Installing HP StoreVitrual VSAE to a basic configuraiton across 4 x Hosts
Pre-installed Management VM running Windows Server 2012 Standard that has OneView for vCenter and CMC for StoreVirtual Management

So after completing the initial setup, you do have the ability to upload an ISO and start deploying an OS image.

What About The Stuff Which Marketing Don’t Mention? AKA Questions Answered?

Database

SQL Express is used as the database (local instance on Management VM). I have real concerns around the database if logging levels are increased to troubleshoot issues and/or the customer doesn’t perform an kind of database maintenance
- I’m waiting on confirmation from HP as whether you can migrate the SQL database instance to a full blown version

Host Profiles

Grey area, these can be used. However HP rather you stay with the base configuration of the nodes (much like the networking see below).

Licences

The solution is only supported using Enterprise or Enterprise Plus VMware licenses, with the preference being HP OEM.
Windows Server 2012 Standard is supplied as the Management VM. Initially, this runs from a local partition and is then Storage vMotioned onto the HP Converged Cluster. Windows licensing dictates that when a OS is moved across hosts using Standard Edition you cannot move the OS back for 90 days or you need to license each node for the potential number of VM’s that could be ran.
- HP have confirmed that you receive 2 x Windows Server 2012 Standard licenses and DRS Groups Manager rules are configured to only allow the Management VM to migrate between these two ESXi Hosts.

Management Server

You are able to upgrade the Management Server VM in terms of RAM, CPU and Disk Space and be supported.
You cannot add additional components to the Management Server VM and be supported e.g. VUM, vCenter SysLog Service
- I’m waiting on confirmation from HP around what is and isn’t supported, I would air on caution and not install anything extra

Networking

The 1GbE connections are not used apart from the initial configuration of the Management Server. My understanding is that these are not supported for any other use.
HP prefer you to stay with the standard network configuration, this causes me concern. 10GbE Network providing Management, iSCSI, Virtual Machine and vMotion traffic. How do you control vMotion bandwidth usage on a Standard vSwitch? You can’t a Distributed vSwitch is a much better option, but if you need to reconfigure a node, you will need to perform a vSS to vDS migration

Updates

You can upgrade individual components separately, however you must stay within the HP Storage SPOCK for the Converged System 200-HC StoreVirtual (Note a HP Passport login is required)

Versions

At the time of this post, the latest supported versions are as follows:

vSphere 5.5 U2 , no vSphere 6
vCenter 5.5 U2
HP StoreVirtual VSA 11.5 or 12.0
HP OneView for vCenter Storage/Server Modules 7.4.2 or 7.4.4
HP OneView Instant On 1.0 or 1.0.1
PowerCLI 5.8 R1

Final Thoughts

HP have put together a slick product which automates the initial installation of ESXi and gives you a basic configuration of vCenter. What it doesn’t give you is design to say that your workloads are going to be suitable on the environment and or a solution that meets a client requirements.

vSphere 5.x Space Reclamation On Thin Provisioned Disks

Posted on 30 March 201511 January 2021 by Craig

Space reclamation can be performed either on vSphere after a Storage vMotion has taken place or when files have been deleted from within a guest operating system.

With the release of LeftHand OS 12.0 as covered in my post ‘How To: HP StoreVirtual LeftHand OS 12.0 With T10 UNMAP‘, I thought it would be an idea to share the process of space reclamation within the guest operating system.

The reason for covering space reclamation within the guest operating system, is that I believe it’s the more common in business as usual operations. Space reclamation on vSphere and Windows is a two step process.

Zero the space in the guest operating system if you are running Windows Server 2008 R2 or below.
- UNMAP is enabled automatically as in Windows Server 2012 or above
- If VMDK is thin provisioned you might want to shrink it back down again
Zero the space on your VMFS file system

I’m going to run space reclamation on a Windows Server 2008 R2 on a virtual machine called DC01-CA01 and has the following storage characteristics:

Original Provisioned Space

Windows C: Drive – 24.9GB free space
Datastore – 95.47GB free space
Volume – 96.93GB consumed space
- 200GB Fully Provisioned with Adaptive Optimisation enabled

Next I’m going to drop two files onto the virtual machine which total 2.3GB in space. This changes the storage characteristics of DC01-CA01 to the following:

Increased Provisioned Space

Windows C: Drive – 22.6GB free space
- 2.3GB increase in space usage
Datastore – 93.18GB free space
- 2.29GB increase in space usage
Volume – 99.22GB consumed space
- 2.29GB increase in space usage

Sdelete

Next I have deleted the files from the C: Drive on DC01-CA01 and emptied the recycle bin. Followed by running sdeldete with the command parameters ‘sdelete.exe -z C:’ This takes a bit of time, so I’m going to make a cup of tea!

WARNING: Running Sdelete will increase the size of the thin provisioned disk to it’s maximum size. Make sure you have space to accommodate this on your volume(s).

VMKFSTools

Now sdelete has finished, we need to run vmkfstools on the datastore to shrink the thin provisioned VMDK back down to size. To do this the virtual machine needs to be powered off.

SSH into the ESXi Host and CD into the directory in which your virtual machine resides. In my case this is cd /vmfs/volumes/DC01-NODR01/DC01-CA01

Next run the command ls -lh *.vmdk which shows the space being used by the virtual disks. Currently stands at 40GB.

Next we want to get rid of the zero blocks in the MDK by issuing the command vmkfstools –punchzero DC01-CA01.vmdk

Now that’s done let’s check our provisioned space to see what is happening.

Interim Provisioned Space

Windows C: Drive – 24.9GB free space
- Back to the original size
Datastore – 95.82GB free space
- 0.35GB decrease from original size
Volume – 121.35GB consumed space
- 24.42GB increase from the original size!

So what’s going on then? Well Windows is aware that blocks have been deleted and passed this information onto the VMFS file system, which has decreased the VMDK size using the vmkfstools –punchzero command, however no one has told my HP StoreVirtual it can reclaim the space and allocate it back out again.

The final step is to issue the vmkfstools -y 90 command. More details about this command are covered in Jason Boche’s excellent blog post entitled ‘Storage: Starting Thin and Staying Thin with VAAI UNMAP‘ on this function.

Note: vmkfstools was deprecated in ESXi 5.1 and replaced with esxcli storage vmfs unmap -l datastorename See VMware KK2057513 for more details

WARNING: Running vmkfstools -y 90 will create a balloon file on your VMFS datastore. Make sure you have space to accommodate this on your datastore and that no operations will happen that could drastically increase the size of the datastore whilst the command is running

One final check of provisioned space now reveals the following:

Final Provisioned Space

Windows C: Drive – 24.9GB free space
- Back to the original size
Datastore – 95.81GB free space
- 0.34GB decrease from original size
Volume – 95.04GB consumed space
- 1.89GB decrease from the original size

Final Thought

Space reclamation has three different levels, guest operating system, VMFS file system and the storage system. Reclamation needs to be performed on each of these layers in turn so that the layer beneath knows it can reclaim the disk space and allocate it out accordingly.

The process of space reclamation isn’t straight forward and should be ran out of hours as each step will have an impact on the storage sub system especially if it’s ran concurrently across virtual machines and datastores.

My recommendation is to reclaim valuable disk space out of hours to avoid potential performance or capacity problems.

HP StoreVirtual Multi-Path Extension Module (MEM) for vSphere

Posted on 4 March 201511 January 2021 by Craig

With the release of LeftHand OS 12.0, HP have introduced a Multi-Path Extension Module to replace the previously recommended path selection policy being ‘Round Robin’ and Storage Array Type Plugin VMW_SATP_DEFAULT_AA

By default Round Robin would send 1,000 I/O down each path.

Note: That this can be altered as per my previous article entitled ‘How To Change Default IOP Limit‘ but should normally only be done under the guidance of the manufacturer.

The issue with this is how a volume is created within a StoreVirtual node. Let’s imagine we have two cluster nodes, A and B. When the first volume is created it is ‘owned by a master node’ in this case ‘node A’. When the second volume is created it is owned by ‘node B and so on, as shown below.

Volume 1 – Node A
Volume 2 – Node B
Volume 3 – Node A
Volume 4 – Node B

Using the recommended bonding method of ‘Adaptive Load Balancing’ SCSI read and write commands are issued on all NIC’s in the bond which can result in data being accessed from a non-authoritative node, which means a trip across the network to the authoritative node. I think a picture is in order!

This was rather inefficient and meant that random reads (which are served from disk) could be accessed from a non-authoritative node. This is where the StoreVirtual Multi-Path Extension Module (MEM) steps in. It has knowledge on where the data resides and ensures that all:

Read I/O’s are always serviced by the storage node that holds the authoritative data.
Write I/O’s are always serviced by the storage node that receives a mirror copy of the data ensuring data integrity. Remaining copies are then forwarded to the remaining storage nodes.

This results in the following data flow architecture.

The installation guide for HP StoreVirtual Multipathing Deployment Guide is fairly straight forward. The only issue I ran into was ‘could not find trusted signer’ when trying to install the vib.

This was resolved by adding –no-sig-check to the end of the command esxcli software vib install -v /HPMEM.vib –no-sig-check

Once installed, you will need to change the path selection policy for your datastores to HP_PSP_LH

How To: Rehost HP StoreVirtual Licenses

Posted on 1 March 201511 January 2021 by Craig

I’m not sure exactly when, but HP changed the licensing portal from ‘Poetic’ to a new portal named ‘My HP Licensing Portal’. All of the information looked exactly the same, however you could not rehost HP StoreVirtual Licenses.

The purpose of this blog post is to assist anyone who was in the same situation as me, scratching their head trying to figure it out!

Problem

You have an existing HP StoreVirtual license which ties the feature set to the first NIC MAC address on your StoreVirtual VSA. You have changed, upgraded or redeployed your VSA and you need to rehost the license onto the new MAC address.

Solution

Browse to myhplicensing.hp.com and login for your email address and password that you use for your portal account.

Note: If you are not sure what email address is tied to your account login to HP Licensing for Software and select Administration > My Profile which will show your email address.

Once logged into My HP Licensing select Rehost Licenses

Next select Rehost Licenses and click on the MAC Address you want to update

Select the tick box to confirm this is the license that you want rehosting and then click Rehost.

Select ‘Enter New Locking ID’ and enter the MAC Address of the first network adapter on your StoreVirtual. Then click Next

This bit takes a while but once done you will receive the license file which can be saved or emailed

How To: Map HP StoreVirtual Volumes to Datastores

Posted on 8 January 201511 January 2021 by Craig

Problem Statement

You have created numerous datastores on your HP StoreVirtual of the same size and presented these to your ESXi Hosts. However, you have since forgotten how the datastores map back to the volumes.

When you check the Runtime Name of your devices (Storage > Devices) to find out the LUN number, you see that each LUN has is ‘0’ as per the screenshot below.

This can be confirmed in HP StoreVirtual Centralised Management Console under Servers > Select Server > Volumes & Snapshots

Not very helpful at all!

Resolution

Each datastores has a unique iSCSI Target string which can be used to identify how they are mapped to volumes.

To find out what they are select the Datastore > Properties > Manage Paths

At the bottom we can see the Target, this shows tells us the following details:

DC02-MG01
- Denotes the Management Group the volume is in
39 is the hexadecimal representation of 27 which is the VMware NAA (thanks to Jonathan Reid for this information)
- Denotes the unique target identifier for the volume
DC01-DR01SRM
- Denotes the volume name on the HP StoreVitual

So we now know this datastore corresponds to the volume called DC01-DR01SRM in Management Group DC02-MG01.