Binarymist

The views expressed in this post are my own and don’t reflect the views of my employer.

Recently I had the opportunity for work, to carry out some research on what’s in the market in regards to bare-metal hypervisors.

The following is the result of an in depth research and deployment project of the following bare-metal hyper-visors.
This will enable us to trial the hypervisors out for performance, ease of setup, ease of administration, and ease of use.

I’ve also looked at hardware costs, but first it needs to be decided which hypervisor we are going to go with.
As this would be a team decision, I thought the best way to go about this was to record some of my existing experience with further research into some of the product leaders offerings.

I haven’t used KVM before.
I knew it existed, but when I was last in the market comparing hypervisors, KVM was an infant.
Now it appears to have grown up and is comparable with it’s commercial rivals.
This pretty much sums up the KVM vs VMware battle
This pretty much sums up the Xen vs KVM battle

ESX(i)

I’ve used these extensively and am well aware of their pros and cons.
Supports iscsi.
I prefer not to have to pay for a product if there are FOS (Free & Open Source) offerings that get the job done just as well.
In looking at the likes of KVM and Xen, the cons of ESX/ESXi really stand out, not to mention the fact that KVM is completely free, more efficient and has a faster pace of growth.
With the free version, that’s ESXi, you get (as of version 5) 32GB vRAM, and that’s only because the community kicked up such a fuss about paying per CPU for a product that was originally free.
VMware keep changing the rules and pricing strategies when users go else where. I’d prefer not to pay at all.
I’m not going to spend time recording the pros and cons of VMware at this stage, as I think the other contenders have more to offer, and ask for less or nothing in return.
If we find that there are un-foreseen hurdles in the other products, we should look at ESXi as a backup.

Management

vSphere client (only runs on windows).
vSphere CLI (read-only, unless you pay for license)
Have very limited access to the hypervisor

Migration

• General
• Potential migration of KVM to VMware.
  Although this link says  the above won’t work, but has some other suggestions.

UPS

See my blog posts.

Citrix XenServer

XenServer support for iscsi

Xen is a type 1 bare-metal hypervisor. This means it runs as close to the hardware as possible.
To take full advantage of it’s speed, you have to run paravirtualised (modified OS’s).
Since most of our work at this stage would be on Windows, there would be no benefit here for us.
Runs in a small custom Linux system.
Intel VT-x or AMD-V is required to run full hardware virtualisation (HVM) rather than paravirtualised.

Licensing for XenServer Express

Be aware, Citrix can change their licensing structure at any time.
Features and current licensing model
XenServer Licensing FAQ
XenCenter can only connect to a single instance of XenServer at any one time.
XenServer currently free
XenCenter free
http://www.citrix.com/English/NE/news/news.asp?newsID=1687130

FAQ

Management

• States Microsoft System Center VMM can be used, which of course won’t be free.
• XenCenter
• Archipel

Migration

ESX(i) to XenServer

Seemed to have struggles (windows guest).
Seemed to be a little more successful (windows guest).

UPS

Integrating XenServer and APC PowerChute. Also see this.
Using apcupsd as KVM can.

Installation Stage

The getting started page. You can find the quick installation guide here.

The full installation guide.
The Administrators guide.

Download and install XenServer on your host.
Download and install XenCenter on your management box.

You’ll need the following details:

1. Hostname
2. Host IP and mask
3. Gateway
4. DNS Server
5. NTP Address

This was a very straight forward install.
I was expecting some trouble, but there wasn’t any.

KVM

KVM has support for iscsi.
Expected to run all production OS’s.
Why will KVM be the leader amongst hypervisors?

Interesting articles:

• http://www.linux-kvm.com/content/why-kvm-will-be-king
• http://avikivity.blogspot.com/2008/03/true-myths.html

Is completely free.
Considerably more resource efficient than the alternatives
There are no resource constraints. We pay for nothing and get an enterprise level product with a huge community.

KVM on Debian

Management

Web based KVM management offerings of which ProxMox VE seems to be the stand-out.
Many of these can also be used for Xen. Also see this.

ProxMoxVE

ProxMox is a commercial company.
ProxMox VE Looks Good.
From what I’ve seen, looks easier to setup than Archipel.
Proxmox VE is licensed under GPLv2 (Open source).
My understanding of the GPLv2 license, is that the suplier of the GPL’d software can decide to charge a fee for download at any time.
As far as I’m aware, Proxmox are within their rights to do so at any time.
Correct me if I’m wrong?
The ISO installer is packaged with Debian, although you can install on top of Debian.
Looks User friendly, has Web interface (multi platform). No installs required.
Support: incl free community and paid for. See here and here.
The wiki
Looks like what ever you can do on a Debian system, you can do on a ProxMox system.
See this link. Also includes ESXi comparisons.
Proxmox VE is free to use and open source.
Easy backups and restores.
Video tutorials here and here.

Archipel

Archipel Also looks good.
Free and Open Source, licensed under AGPL (which more specifically targets distributed applications).
Team of 6 voluntary developers. Lots of info here.
Supports all libvirt-supported virtualisation engines like KVM, Xen, VMware
The install on first appearance, looks more work than ProxMox.
Documentation, IRC channel (members are very helpful), etc.
The Archipel client is JavaScript, which is run locally.

Industry support

KVM is supported by major industry players such as…

1. IBM
2. Cisco
3. Intel
4. AMD
5. Redhat
6. Novell amongst others.

Migration

Looks like migration of guests from most platforms to KVM is covered.
VMware to Proxmox, XenServer to Proxmox.

UPS

Can be shutdown by an APC Smart-UPS
using the APCUPSD daemon This will shutdown immediately.
Or better, by using PCNS for Linux.
Using PCNS we can specify when to shutdown and all sorts of other things.

Installation Stage Archipel

Links found useful for the Debian setup

http://www.debian-tutorials.com/virtualization/kvm-virtualization-on-debian-squeeze-server

http://wiki.debian.org/KVM

http://wiki.libvirt.org/page/Networking#Bridged_networking_.28aka_.22shared_physical_device.22.29

http://wiki.kartbuilding.net/index.php/KVM_Setup_on_Debian_Squeeze

Setting up Debian

Download Debian Wheezy from here
Install it.
Give it a hostname. For example “vmhost” without the quotes.
When prompted, select the SSH Server option.
Update your package index and install the necessary packages.

As root, run:

apt-get update
apt-get install qemu-kvm libvirt-bin virtinst virt-top

virtinst is for virt-install tools etc.
qemu-kvm is the new name for the kvm package in squeeze
libvirt-bin is what will control kvm and start guests on boot etc.
virt-top is a ‘top’-like utility for virtualisation stats

Add user to groups

Add the currently logged in user that will be using the associated programmes.

usermod -a -G libvirt myusername
usermod -a -G kvm myusername

Then check that the user was added to the groups.

groups myusername

or

id myusername

or view all users in all groups

cat /etc/group | less

Setup networking

Your /etc/network/interfaces needs to have a similar section:
As root, run the following…

vi /etc/network/interfaces

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
   address 192.168.1.20
   netmask 255.255.255.0
   gateway 192.168.1.254
   broadcast 192.168.1.255

Now restart your interface:

ifdown eth0

ifup eth0

Check that the changes have taken affect:

ip addr show

Setup Bridged networking

You also need to set up a network bridge on our server.
Rather than use NAT based connectivity, we need bridge networking.

install the package bridge-utils.

apt-get install bridge-utils

Setting up Archipel

Links I found helpful:

• https://github.com/primalmotion/Archipel/wiki/Installation-manual
• http://blog.professional.geek.nz/post/5951118448/archipel-setup-walkthrough
• http://www.youtube.com/watch?v=KkoGBK2uqZQ

FAQ and supported browsers
https://github.com/primalmotion/Archipel/wiki/General%3A-FAQ &nbsp

Install ejabberd

apt-get install ejabberd

According to this, which is linked if you follow the install guide through,
we will need to update the path to the tls certificate.
Not sure where that is, but will have to find out.
the sample file contains the ejabberd configuration needed for Archipel.
It is not ready for production, so will need some modification. Yet to find out what.
Change all occurrences of FQDN to vmhost.mydomain.local and follow the other directions.

Once the ejabberd.cfg file is modified as suggested, download pscp.exe from here.
Put both the pscp.exe file and the ejabberd.cfg in the same folder (just to save typing paths and adding environment variables).
The help page is here if you get stuck.
Run a cmd prompt from the directory you have the 2 previous mentioned files within.
Then run:

pscp ejabberd.cfg myusername@192.168.1.20:ejabberd.cfg

Enter your password when prompted.
The file will be securely copied via SSH to your ~ dir.
You can’t copy directly to the /etc/ejabberd/ directory as you would need to be root of the destination machine.
Now go to the Debian box. cd into ~.
and move the config file to where it belongs.

su root

Enter your password when prompted.

mv ejabberd.cfg /etc/ejabberd/ejabberd.cfg

Then check that the move was successful.

Start the jabber server if it’s not already.
As root:

/etc/init.d/ejabberd start

Wait a few seconds and run:

/usr/sbin/ejabberdctl status

And you should get a result of running, with the version details.

You need to register a XMPP admin account (if you want archipel to work out of the box, just name it admin):

ejabberdctl register admin vmhost.mydomain.local MyCrazyPassWordHere

You should get something like:

User admin@vmhost.mydomain.local successfully registered.

Although I didn’t the last time because I wasn’t running as root.

Continue with the Archipel installation

The client is easy, just fetch and un-compress and your ready to go.

The agent, you will need to install qemu-utils if it’s not already.
It was for me.

As root, run:

apt-get install python-setuptools python-imaging python-numpy python-libvirt

python-libvert is Python bindings for the libvirt library which was already installed.

I also installed subversion:

apt-get install subversion

Now… as root, I chose to install the published packages on Pypi.
I ran:

easy_install archipel-agent

Post installation formalities

Finalise the installation:

archipel-initinstall

Follow the additional output instructions on the screen.

Now as root:

Create the pubsub nodes

archipel-tagnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-rolesnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-adminaccounts --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-vmparkingnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create

The last two commands were, introduced after beta 4, so they didn’t exist on the binary I installed.

You can now start the archipel agent.

/etc/init.d/archipel start

The logs are printed to /var/log/archipel/archipel.log

To be completely sure Archipel is up and your hypervisor is connected you can run:

ejabberdctl connected_users

If you choose to just dump the archipel client somewhere and browse to the index.html,
you will have to use Safari as the browser.
Alternatively, you can use Chrome,
but you need to pass the argument… –disable-web-security
Or the better way is to just uncompress the archive into a HTTP server directory,
and access it with your browser.
I’ve been told nginx works well with serving Archipel.
At this stage I just set the client up on IIS locally.
In saying that, I’m getting the index.html,
but I’m getting 404’s for Info.plist and main.j
I need to look into this.

Using Archipel

https://github.com/primalmotion/Archipel/wiki/User-manual

Once you have the page in your browser, enter the following details into the dialog.

Jabber ID: admin@vmhost.mydomain.local
Password: MyCrazyPassWordHere
BOSH service: http://vmhost.mydomain.local:5280/http-bind

If you can’t access vmhost, try navigating to http://vmhost.mydomain.local:5280/http-bind in your browser.

You should get something like the following:

If you don’t,
try pinging vmhost.mydomain.local.
If the IP works but the host.FQDN doesn’t, it’s a dns issue.
I checked the /etc/hosts file and it had the host name as expected.

127.0.1.1   vmhost.mydomain.local   vmhost

For some reason, the Debian box’s hostname wasn’t getting registered on the DNS server.
The way around this is to add the following entry to the hosts file of the machine you have your client running from.

192.168.1.20    vmhost.mydomain.local