I/O Virtualisation: green, flexible and virtual

These days just about every data centre anywhere in the world is somewhere along the road to virtualisation. In fact, I can't remember a job in the last two or three years where the company hiring me wasn't doing some form of server virtualisation.

Hypervisors such as VMware, Hyper-V and XenServer are driving an evolution the data centre. As a testament to this, the latest and greatest CPUs from chip makers like Intel and AMD are designed with virtualisation in mind. Chip based virtualisation technologies, such as Intel VT and AMD-V, provide hardware offloads and assists designed to work hand in hand with Hypervisors in driving this data centre evolution.

At its core though, this evolution is based on, and driven by, the following core principles – green, flexible, dynamic and virtual. Oh and cost.

However, despite Hypervisors blazing a trail, and CPU manufacturers now well and truly in the game, there still remains at least one set of components from the traditional server side lagging behind – the I/O components, in particular NICs and HBAs.

I/O technologies
Thankfully there are some very interesting technologies, including already shipping products that are starting to redress this imbalance. However, surprisingly few people are aware of them.

First up, the following two standards are about to have a major impact –

• SR-IOV (Single Root I/O Virtualisation)
• MR-IOV (Multi Root I/O Virtualisation)

Both of the above are PCI-SIG standards with the goal of bringing hardware based virtualisation to the I/O world.

I can't remember a job in the last two or three years where the company hiring me wasn't doing some form of virtualisation.  Nigel Poulton Freelance enterprise storage and new technology consultant

Conceptually, the two standards enable a single physical I/O adapter, such as a NIC, HBA or CNA (Converged Network Adapter), to be carved up into multiple virtual adapters.

E.g. A single SR-IOV capable NIC can be divided into multiple virtual NICs, referred to a vNICs. Each of these vNICs is presented to the servers PCIe tree as a discrete individual PCI function, and as a result looks and feels just like any normal individual physical adapter would. However, because we can get multiple vNICs from a single physical adapter, we can get higher utilisation, less cable sprawl, greater server density, lower power requirements due to fewer physical adapters in our servers, and greater flexibility.

Currently, SR-IOV and MR-IOV are relatively new and not widely available. They require SR-IOV capable I/O adapters as well as Hypervisor support, BIOS support and specific drivers. However, Citrix demoed SR-IOV capable NICs leveraging Intel VT-d on Dell servers performing live migrations at IDF in October 2009. So keep an eye out for these technologies becoming more and more mainstream.

Outside of the PCI-SIG standards, there are some already shipping products that implement the same functionality. Some of these include -

• Converged Network Adapters from the likes of Emulex and QLogic
• VirtenSys VIO switches
• Xsigo VP780 I/O Director

All of the above work to the same principle of creating multiple virtual adapters from a single physical adapter. One example being the Emulex OneConnect Universal CNA which is deployed as part of Virtual Fabric for IBM BladeCenter. This solution can create 8 vNICs all from a single dual-port 10Gbps adapter.

While the technologies discussed so far are great and a definite step in the right direction, the VirtenSys and Xsigo solutions take things a step further. They actually remove the I/O adapter form the server chassis, re-housing it in an external I/O Aggregator. Physical servers then connect to these I/O Aggregators via either PCIe cables (VirtenSys), or Infiniband (Xsigo).

This I/O Aggregator approach allows data centre staff even more flexibility – any vNIC, or vHBA, can be dynamically assigned to any connected host!

Need to add a NIC or HBA to a server? Simply create the vNIC or vHBA in the I/O Aggregator and present it to the server in question. Job done!

In the configurations that I have seen, the virtual adapters are dynamically recognised and picked up by the hosts – no reboots, definitely no need to crack the server open and no need to wait for new cables to be laid. Now that is quick!

There is little doubt that technologies discussed will have a huge impact in data centre design. We could even be about to witness the de facto home of the I/O adapter moved form the server chassis to an external I/O aggregator.

In summary, all of the above technologies enable –

• Cost and power savings – increased utilisation of physical resources results in purchasing and powering fewer physical resources
• Simplified cable management - reducing the number of cables from each server or each physical adapter
• Dynamic resource management – want to add a NIC or HBA to a server, simply create a new vNIC or vHBA from the already installed physical adapter.

The above benefits are bang in line with the core driving principles of data centre evolution that we discussed at the top of the article – green, flexible, dynamic, virtual and cost.

Tip: For a greener, more dynamic and flexible virtual data centre, take a serious look at the available and emerging I/O Virtualisation technologies, they are the future.

Nigel Poulton is a freelance enterprise storage and new technology consultant with a wealth of experience contracting for vendors as well as customers where he has successfully worked on some cutting edge projects and deployments. For further detail on any article Nigel writes, or for information regarding availability, Nigel can be contacted at nigel@storage-strategist.com.

04 Dec 2009