ESXi Networking Part 1

One of the items that becomes apparent when using VMware is that you need to have a strong understanding of routing and switching.

This blog post is a bit self indulgent as I’m preparing for the VCP 5 exam, I thought it would be good for me to put together a few posts on the achitecture of the switches.

All of the switches within ESXi are software based and operate within the VMkernel.  They are called virtual switches (vSwitches) and are Layer 2 devices, which are capable of trunking and passing VLAN traffic.  A common myth is that vSwitches can trunk ports together using 802.1q. vSwitches do not use Spanning Tree Protocol as one vSwitch cannot be connected to another vSwitch.

Standard Switch (vSwitch)

These are created when we first install ESXi onto our server hardware.  By default this is called vSwitch0 and contains 120 visible Ports (actually holds 128 Ports, 8 are reserverd by the VMkernel), the first virtual machine ‘port group’ called VMNetwork and a Management Network which is used by the VMKernel.

Distributed Switch (dvSwitch)

These are standard switches which are logically grouped across all ESXi hosts who share a common distributed switch configuration. These are only available with Enterprise Plus licenses.

Port Groups

These reside within a vSwitch.  Port groups contain two different configurations:

– VMkernal Ports allow vMotion, Fault Tolerant Logging, iSCSI NAS, NFS traffic between ESXi hosts as well as allowing management of the ESXi host it resides on.

– VM Ports allow a virtual machines to access other virtual machines or network based resources.

The key thing to remember is that with Port Groups they must be named exactly the same across all ESXi hosts to allow traffic to flow.

Note, it is possible to have a vSwitch without any Port Groups, however this would be like having a physical switch without any physical ports!

Uplinks (pNIC)

An uplink if the physical network adapter that the vSwitch is connected too.  Without this the virtual machines that reside on the vSwitch would be isolated and unable to communicate with the rest of the network.

In the picture below we have a Standard Switch called vSwitch1 whose physical uplink (pNIC) is vmnic4.  It contains two different port groups, one for vMotion and Fault Tolerant Logging and the other for VM’s on VLAN29.

Even though we have two different port groups, it is important to remember that each port group is a boundary for communications, broadcasts and security policys.

ESXi 5 Host Isolation

What is a ‘host isolation’?

It’s the term that VMware use to define when an ESXi host is no longer able to communicate with specific IP address’s and therefore it is deemed to be isolated from the rest of the cluster.

By default the ESXi hosts default gateway (the VMkernal gateway) is used.  Depending on your infrastructure this is normally a Layer 3 switch, router or firewall.

Whats the problem with that you ask? Well what happens if you have an outage of your Layer 3 switch, firewall or router? Well vCentre will think that your ESXi hosts are isolated and depending on your ‘host isolation response’ perform one of the following actions:

The recommended action for vCentre 5 is to ‘leave powered on’.

We therefore need to provide more external devices for vCentre to communicate with before it invokes a host isolation response. To do this we go into the Cluster Settings > vSphere HA > Advanced Options.

We then add additional IP address’s that we went vCentre to communicate with in the following format:

das.isolationaddress1 10.0.0.1
das.isolationaddress2 192.168.1.1

We then end the range of IP address’s with ‘das.usedefaultisolationaddress’ ‘false’

What IP address’s would I recommend you use in a production environment?

– vMotion/FT switches
– SAN Controller Management IP address’s
– Layer 2 Switch
– Layer 3 Switch
– Firewall