Category: Server

Home / Category: Server

RHEL running on Hyper-V

This article will mainly focus on the preparatory steps and the requirements needed for installation of SANnav Management Portal 2.0.0. We will implement pre-requisites on the virtual machine, create the host, install the OS and configure some important services. As of December 2019, Broadcom supports SANnav on RHEL or CentOS only as bare metal installation or as virtual machine running on ESXi, however it should also be running on Hyper-V.

My configuration consists of a Microsoft Hyper-V environment using Failover Clustering on Windows 2012 R2 for HA purposes.
My guest host will use Red Hat Enterprise Linux version 7.6. Before starting, make sure to do the bookkeeping: reserve an IP address, write down the network mask and gateway. Also create the host name record in DNS.

Consider to read the SANnav Management Portal Guide for a better understanding of the requirements.

RHEL Virtual Machine in Hyper-V

Creating Virtual Machine

  1. Open Hyper-V Manager, right click on the physical host, select New, Virtual Machine…
  2. Specify name and location where you VM will reside
  3. Keep the defaults on generation of the virtual machine, I chose Generation 1 but you can choose Generation 2 according to your needs
  4. Assign at least 48 GB or RAM memory – this is the minimum RAM you need to select
  5. Configure a network card for your VM
  6. Create a new virtual hard disk and assign minimum 700GB of size
  7. On the Installation Options, keep the defaults – Install an operating system later
  8. Review the summar and click Finish to complete the wizard

You can accomplish the same on a quick way using Powershell. This time we’ll just skip Powershell classes and continue with the core.

At this point our VM container is created, but we still need to make some finetuning in order to be ‘compliant’ with SANnav. One of the requirements is that the VM has at least 2 sockets. In VMWare, we select the sockets during the creation of the VM. Microsoft doesn’t offer this option by default.

  1. Open Hyper-V Manager or Failover Cluster Manager if you have clustered your VM role, right click and select Settings
  2. Under Processors, select at least 16 processors
  3. Expand Processors and click on NUMA
  4. We will divide the Maximum number of processors by 2 (sockets). If you are planning to use 16 processors, type 8 and the VM will get 2 sockets.
    This is an important step, otherwise the setup of SANnav will not succeed.
  5. At this point we will attach the .ISO file of the RHEL package. Under SCSI Controller, click on DVD Drive and attach the .ISO file.
  6. Under Firmware, make sure to move DVD Drive up as the first boot device.

Installing RHEL

After we have configured the basic settings of our VM container, we will power on the VM and proceed with the installation of RHEL.

  1. Use the arrows to move up to Install Red Hat Enterprise Linux 7.6 and hit Enter to continue
  2. Select your preferred language.
  3. Under Localization, click on Date and Time and enter the locales.
  4. Optional: Enable Network Time and add your own time servers. Hit Done to close.
  5. Under Software, keep the defaults for Installation Source as it will automatically detect the attached .ISO file.
  6. Use Software Selection to select the preferres installation features. I will perform a Minimal Install with the following add-ons:
    Debugging Tools
    Security Tools
    System Administration Tools
  7. Under System, click on Destination and select I will configure partitioning.
  8. Create disk partitioning according to your needs. However make sure to follow the mimimum requirements of Broadcom for SANNav. For a single node installation of 3000 ports the minimum disk size is 600. Hit Done to continue.
    RHEL Partitions
    Partitions used in Red Hat for SANnav implementation

    For /boot /boot/efi and swap file, I’ve created separate partitions.
    For the / root I’m using LVM for sizing flexibility.

  9. Keep the default setting for KDUMP (enabled)
  10. Click on Network & Host name and turn on the Ethernet (eth0) card
  11. At the bottom enter the hostname and click Apply
  12. On the same screen click on Configure and enter the network settings. Disable IPv6 if not used
  13. Keep the defaults for Security Policy
  14. Hit Begin Installation to start the installer
  15. While installtion you’ll be prompted to create a root password and create a user
  16. Congrats! At this point you have a semi-compliant RHEL VM for SANnav Management Portal

 

Any suggestion or question? Leave a reply below, or feel free to contact us.

Make sure to subscribe to our mailinglist to get the latest. No spam. Promise!

Recently I came across an interesting application consistent backup issue of Hyper-V virtual machines in CommVault. The backup jobs would partly run or complete with one or more errors. According to the error log shown in CommVault, I would start by checking the free space on the Hyper-V host and within the virtual machines. However, this was not the case here as both machines had enough free space. Hence I started digging further into CommVault snapshots.

Error Code: [91:155]
Description: The system or provider has insufficient storage space. If possible delete any old or unnecessary persistent shadow copies and try again.
Source: <server name>, Process: vsbkp

Diagnosis

Checking CommVault logs under %Programfiles%\CommVault\Simpana\Log Files couldn’t shed light on the root cause. The next thing I had in mind was to check for any old checkpoints in Hyper-V Manager. An easy way to collect all the available checkpoints on a Hyper-V server is by using the following Powershell command:

Get-VMSnapshot -VMName *

Or even easier, to collect all checkpoint available on all cluster nodes you could use the following command.

Get-VM -ComputerName (Get-ClusterNode -Cluster %clustername%) | Get-VMSnapshot
Powershell command to get available snapshots on all cluster nodes
Powershell command to get available snapshots on all cluster nodes

From my experience with backups, I can say that CommVault sometimes can struggle taking backups of virtual machines that have checkpoints available.  You could remove unused checkpoints, however, it might happen in very rare situations.

Digging further into the Windows Logs of the Hyper-v host (Microsoft-Windows-Hyper-V-VMMS/Admin) I noticed a ‘sneaky and innocent’ error referring to a .avhdx file of a snapshot that does no longer exist.

The absolute path ‘C:\ClusterStorage\<CSVname>\<VMName><DiskGUID>.avhdx’ is valid for the “Hard Disk Image pool, but references a file that does not exist.

Event log showing the missing reference of the file
Event log showing the missing reference of the file

The missing file could be seen on the Hyper-V Manager, the virtual machine was still referring to inexisting CommVault snapshots in Hyper-V.

Hyper-V VM referring to a missing file
Hyper-V VM referring to a missing file

Following event ID’s from Windows Event Viewer could be interesting to look for: 10172, 32902

Solution

Restarting Hyper-V Virtual Machine Management (vvms) service will solve the problem. I did it using graphical interface, and a scary warning appeared asking me if I wanted to ‘Turn Off’ the service. Finally, it just halts the management service and does not affect the running VM’s.

Stop VMM service warning
Stop VMM service warning

The same operation can be accomplished using the following Powershell commands:

stop-service -name vmms
start-service -name vmms

Any suggestion or question? Leave a reply below, or feel free to contact us. Make sure to subscribe to our mailing list to get the latest.