The views expressed in this post are my own and don’t reflect the views of my employer.
Recently I had the opportunity for work, to carry out some research on what’s in the market in regards to bare-metal hypervisors.
The following is the result of an in depth research and deployment project of the following bare-metal hyper-visors.
This will enable us to trial the hypervisors out for performance, ease of setup, ease of administration, and ease of use.
I’ve also looked at hardware costs, but first it needs to be decided which hypervisor we are going to go with.
As this would be a team decision, I thought the best way to go about this was to record some of my existing experience with further research into some of the product leaders offerings.
I haven’t used KVM before.
I knew it existed, but when I was last in the market comparing hypervisors, KVM was an infant.
Now it appears to have grown up and is comparable with it’s commercial rivals.
This pretty much sums up the KVM vs VMware battle
This pretty much sums up the Xen vs KVM battle
I’ve used these extensively and am well aware of their pros and cons.
I prefer not to have to pay for a product if there are FOS (Free & Open Source) offerings that get the job done just as well.
In looking at the likes of KVM and Xen, the cons of ESX/ESXi really stand out, not to mention the fact that KVM is completely free, more efficient and has a faster pace of growth.
With the free version, that’s ESXi, you get (as of version 5) 32GB vRAM, and that’s only because the community kicked up such a fuss about paying per CPU for a product that was originally free.
VMware keep changing the rules and pricing strategies when users go else where. I’d prefer not to pay at all.
I’m not going to spend time recording the pros and cons of VMware at this stage, as I think the other contenders have more to offer, and ask for less or nothing in return.
If we find that there are un-foreseen hurdles in the other products, we should look at ESXi as a backup.
vSphere client (only runs on windows).
vSphere CLI (read-only, unless you pay for license)
Have very limited access to the hypervisor
- Potential migration of KVM to VMware.
Although this link says the above won’t work, but has some other suggestions.
See my blog posts.
XenServer support for iscsi
Xen is a type 1 bare-metal hypervisor. This means it runs as close to the hardware as possible.
To take full advantage of it’s speed, you have to run paravirtualised (modified OS’s).
Since most of our work at this stage would be on Windows, there would be no benefit here for us.
Runs in a small custom Linux system.
Intel VT-x or AMD-V is required to run full hardware virtualisation (HVM) rather than paravirtualised.
Licensing for XenServer Express
Be aware, Citrix can change their licensing structure at any time.
Features and current licensing model
XenServer Licensing FAQ
XenCenter can only connect to a single instance of XenServer at any one time.
XenServer currently free
- States Microsoft System Center VMM can be used, which of course won’t be free.
ESX(i) to XenServer
The getting started page. You can find the quick installation guide here.
Download and install XenServer on your host.
Download and install XenCenter on your management box.
You’ll need the following details:
- Host IP and mask
- DNS Server
- NTP Address
This was a very straight forward install.
I was expecting some trouble, but there wasn’t any.
Is completely free.
Considerably more resource efficient than the alternatives
There are no resource constraints. We pay for nothing and get an enterprise level product with a huge community.
KVM on Debian
ProxMox is a commercial company.
ProxMox VE Looks Good.
From what I’ve seen, looks easier to setup than Archipel.
Proxmox VE is licensed under GPLv2 (Open source).
My understanding of the GPLv2 license, is that the suplier of the GPL’d software can decide to charge a fee for download at any time.
As far as I’m aware, Proxmox are within their rights to do so at any time.
Correct me if I’m wrong?
The ISO installer is packaged with Debian, although you can install on top of Debian.
Looks User friendly, has Web interface (multi platform). No installs required.
Support: incl free community and paid for. See here and here.
Looks like what ever you can do on a Debian system, you can do on a ProxMox system.
See this link. Also includes ESXi comparisons.
Proxmox VE is free to use and open source.
Easy backups and restores.
Video tutorials here and here.
Archipel Also looks good.
Free and Open Source, licensed under AGPL (which more specifically targets distributed applications).
Team of 6 voluntary developers. Lots of info here.
Supports all libvirt-supported virtualisation engines like KVM, Xen, VMware
The install on first appearance, looks more work than ProxMox.
Documentation, IRC channel (members are very helpful), etc.
KVM is supported by major industry players such as…
- Novell amongst others.
Looks like migration of guests from most platforms to KVM is covered.
VMware to Proxmox, XenServer to Proxmox.
Can be shutdown by an APC Smart-UPS
using the APCUPSD daemon This will shutdown immediately.
Or better, by using PCNS for Linux.
Using PCNS we can specify when to shutdown and all sorts of other things.
Installation Stage Archipel
Links found useful for the Debian setup
Setting up Debian
Download Debian Wheezy from here
Give it a hostname. For example “vmhost” without the quotes.
When prompted, select the SSH Server option.
Update your package index and install the necessary packages.
As root, run:
apt-get update apt-get install qemu-kvm libvirt-bin virtinst virt-top
virtinst is for virt-install tools etc.
qemu-kvm is the new name for the kvm package in squeeze
libvirt-bin is what will control kvm and start guests on boot etc.
virt-top is a ‘top’-like utility for virtualisation stats
Add user to groups
Add the currently logged in user that will be using the associated programmes.
usermod -a -G libvirt myusername usermod -a -G kvm myusername
Then check that the user was added to the groups.
or view all users in all groups
cat /etc/group | less
Your /etc/network/interfaces needs to have a similar section:
As root, run the following…
# The primary network interface
iface eth0 inet static
Now restart your interface:
Check that the changes have taken affect:
ip addr show
Setup Bridged networking
You also need to set up a network bridge on our server.
Rather than use NAT based connectivity, we need bridge networking.
install the package bridge-utils.
apt-get install bridge-utils
I’ve yet to set the bridge up.
Will add this once done
Setting up Archipel
Links I found helpful:
apt-get install ejabberd
According to this, which is linked if you follow the install guide through,
we will need to update the path to the tls certificate.
Not sure where that is, but will have to find out.
the sample file contains the ejabberd configuration needed for Archipel.
It is not ready for production, so will need some modification. Yet to find out what.
Change all occurrences of FQDN to vmhost.mydomain.local and follow the other directions.
Once the ejabberd.cfg file is modified as suggested, download pscp.exe from here.
Put both the pscp.exe file and the ejabberd.cfg in the same folder (just to save typing paths and adding environment variables).
The help page is here if you get stuck.
Run a cmd prompt from the directory you have the 2 previous mentioned files within.
pscp ejabberd.cfg email@example.com:ejabberd.cfg
Enter your password when prompted.
The file will be securely copied via SSH to your ~ dir.
You can’t copy directly to the /etc/ejabberd/ directory as you would need to be root of the destination machine.
Now go to the Debian box. cd into ~.
and move the config file to where it belongs.
Enter your password when prompted.
mv ejabberd.cfg /etc/ejabberd/ejabberd.cfg
Then check that the move was successful.
Start the jabber server if it’s not already.
Wait a few seconds and run:
And you should get a result of running, with the version details.
You need to register a XMPP admin account (if you want archipel to work out of the box, just name it admin):
ejabberdctl register admin vmhost.mydomain.local MyCrazyPassWordHere
You should get something like:
User firstname.lastname@example.org successfully registered.
Although I didn’t the last time because I wasn’t running as root.
Continue with the Archipel installation
The client is easy, just fetch and un-compress and your ready to go.
The agent, you will need to install qemu-utils if it’s not already.
It was for me.
As root, run:
apt-get install python-setuptools python-imaging python-numpy python-libvirt
python-libvert is Python bindings for the libvirt library which was already installed.
I also installed subversion:
apt-get install subversion
Now… as root, I chose to install the published packages on Pypi.
Post installation formalities
Finalise the installation:
Follow the additional output instructions on the screen.
Now as root:
Create the pubsub nodes
archipel-tagnode --email@example.com --password=MyCrazyPassWordHere --create archipel-rolesnode --firstname.lastname@example.org --password=MyCrazyPassWordHere --create archipel-adminaccounts --email@example.com --password=MyCrazyPassWordHere --create archipel-vmparkingnode --firstname.lastname@example.org --password=MyCrazyPassWordHere --create
The last two commands were, introduced after beta 4, so they didn’t exist on the binary I installed.
You can now start the archipel agent.
The logs are printed to /var/log/archipel/archipel.log
To be completely sure Archipel is up and your hypervisor is connected you can run:
If you choose to just dump the archipel client somewhere and browse to the index.html,
you will have to use Safari as the browser.
Alternatively, you can use Chrome,
but you need to pass the argument… –disable-web-security
Or the better way is to just uncompress the archive into a HTTP server directory,
and access it with your browser.
I’ve been told nginx works well with serving Archipel.
At this stage I just set the client up on IIS locally.
In saying that, I’m getting the index.html,
but I’m getting 404’s for Info.plist and main.j
I need to look into this.
Once you have the page in your browser, enter the following details into the dialog.
Jabber ID: email@example.com
BOSH service: http://vmhost.mydomain.local:5280/http-bind
If you can’t access vmhost, try navigating to http://vmhost.mydomain.local:5280/http-bind in your browser.
You should get something like the following:
If you don’t,
try pinging vmhost.mydomain.local.
If the IP works but the host.FQDN doesn’t, it’s a dns issue.
I checked the /etc/hosts file and it had the host name as expected.
127.0.1.1 vmhost.mydomain.local vmhost
For some reason, the Debian box’s hostname wasn’t getting registered on the DNS server.
The way around this is to add the following entry to the hosts file of the machine you have your client running from.