Bare-metal Hypervisor Setup Evaluation

The views expressed in this post are my own and don’t reflect the views of my employer.

Recently I had the opportunity for work, to carry out some research on what’s in the market in regards to bare-metal hypervisors.

The following is the result of an in depth research and deployment project of the following bare-metal hyper-visors.
This will enable us to trial the hypervisors out for performance, ease of setup, ease of administration, and ease of use.

I’ve also looked at hardware costs, but first it needs to be decided which hypervisor we are going to go with.
As this would be a team decision, I thought the best way to go about this was to record some of my existing experience with further research into some of the product leaders offerings.

I haven’t used KVM before.
I knew it existed, but when I was last in the market comparing hypervisors, KVM was an infant.
Now it appears to have grown up and is comparable with it’s commercial rivals.
This pretty much sums up the KVM vs VMware battle
This pretty much sums up the Xen vs KVM battle


I’ve used these extensively and am well aware of their pros and cons.
Supports iscsi.
I prefer not to have to pay for a product if there are FOS (Free & Open Source) offerings that get the job done just as well.
In looking at the likes of KVM and Xen, the cons of ESX/ESXi really stand out, not to mention the fact that KVM is completely free, more efficient and has a faster pace of growth.
With the free version, that’s ESXi, you get (as of version 5) 32GB vRAM, and that’s only because the community kicked up such a fuss about paying per CPU for a product that was originally free.
VMware keep changing the rules and pricing strategies when users go else where. I’d prefer not to pay at all.
I’m not going to spend time recording the pros and cons of VMware at this stage, as I think the other contenders have more to offer, and ask for less or nothing in return.
If we find that there are un-foreseen hurdles in the other products, we should look at ESXi as a backup.


vSphere client (only runs on windows).
vSphere CLI (read-only, unless you pay for license)
Have very limited access to the hypervisor


  • General
  • Potential migration of KVM to VMware.
    Although this link says  the above won’t work, but has some other suggestions.


See my blog posts.

Citrix XenServer

XenServer support for iscsi

Xen is a type 1 bare-metal hypervisor. This means it runs as close to the hardware as possible.
To take full advantage of it’s speed, you have to run paravirtualised (modified OS’s).
Since most of our work at this stage would be on Windows, there would be no benefit here for us.
Runs in a small custom Linux system.
Intel VT-x or AMD-V is required to run full hardware virtualisation (HVM) rather than paravirtualised.

Licensing for XenServer Express

Be aware, Citrix can change their licensing structure at any time.
Features and current licensing model
XenServer Licensing FAQ
XenCenter can only connect to a single instance of XenServer at any one time.
XenServer currently free
XenCenter free




ESX(i) to XenServer

Seemed to have struggles (windows guest).
Seemed to be a little more successful (windows guest).


Integrating XenServer and APC PowerChute. Also see this.
Using apcupsd as KVM can.

Installation Stage

The getting started page. You can find the quick installation guide here.

The full installation guide.
The Administrators guide.

Download and install XenServer on your host.
Download and install XenCenter on your management box.

You’ll need the following details:

  1. Hostname
  2. Host IP and mask
  3. Gateway
  4. DNS Server
  5. NTP Address

This was a very straight forward install.
I was expecting some trouble, but there wasn’t any.


KVM has support for iscsi.
Expected to run all production OS’s.
Why will KVM be the leader amongst hypervisors?

Interesting articles:

Is completely free.
Considerably more resource efficient than the alternatives
There are no resource constraints. We pay for nothing and get an enterprise level product with a huge community.

KVM on Debian


Web based KVM management offerings of which ProxMox VE seems to be the stand-out.
Many of these can also be used for Xen. Also see this.


ProxMox is a commercial company.
ProxMox VE Looks Good.
From what I’ve seen, looks easier to setup than Archipel.
Proxmox VE is licensed under GPLv2 (Open source).
My understanding of the GPLv2 license, is that the suplier of the GPL’d software can decide to charge a fee for download at any time.
As far as I’m aware, Proxmox are within their rights to do so at any time.
Correct me if I’m wrong?
The ISO installer is packaged with Debian, although you can install on top of Debian.
Looks User friendly, has Web interface (multi platform). No installs required.
Support: incl free community and paid for. See here and here.
The wiki
Looks like what ever you can do on a Debian system, you can do on a ProxMox system.
See this link. Also includes ESXi comparisons.
Proxmox VE is free to use and open source.
Easy backups and restores.
Video tutorials here and here.


Archipel Also looks good.
Free and Open Source, licensed under AGPL (which more specifically targets distributed applications).
Team of 6 voluntary developers. Lots of info here.
Supports all libvirt-supported virtualisation engines like KVM, Xen, VMware
The install on first appearance, looks more work than ProxMox.
Documentation, IRC channel (members are very helpful), etc.
The Archipel client is JavaScript, which is run locally.

Industry support

KVM is supported by major industry players such as…

  1. IBM
  2. Cisco
  3. Intel
  4. AMD
  5. Redhat
  6. Novell amongst others.


Looks like migration of guests from most platforms to KVM is covered.
VMware to Proxmox, XenServer to Proxmox.


Can be shutdown by an APC Smart-UPS
using the APCUPSD daemon This will shutdown immediately.
Or better, by using PCNS for Linux.
Using PCNS we can specify when to shutdown and all sorts of other things.

Installation Stage Archipel

Links found useful for the Debian setup

Setting up Debian

Download Debian Wheezy from here
Install it.
Give it a hostname. For example “vmhost” without the quotes.
When prompted, select the SSH Server option.
Update your package index and install the necessary packages.

As root, run:

apt-get update
apt-get install qemu-kvm libvirt-bin virtinst virt-top

virtinst is for virt-install tools etc.
qemu-kvm is the new name for the kvm package in squeeze
libvirt-bin is what will control kvm and start guests on boot etc.
virt-top is a ‘top’-like utility for virtualisation stats

Add user to groups

Add the currently logged in user that will be using the associated programmes.

usermod -a -G libvirt myusername
usermod -a -G kvm myusername

Then check that the user was added to the groups.

groups myusername


id myusername

or view all users in all groups

cat /etc/group | less
Setup networking

Your /etc/network/interfaces needs to have a similar section:
As root, run the following…

vi /etc/network/interfaces

# The primary network interface
allow-hotplug eth0
iface eth0 inet static

Now restart your interface:

ifdown eth0
ifup eth0

Check that the changes have taken affect:

ip addr show
Setup Bridged networking

You also need to set up a network bridge on our server.
Rather than use NAT based connectivity, we need bridge networking.

install the package bridge-utils.

apt-get install bridge-utils

I’ve yet to set the bridge up.
Will add this once done

Setting up Archipel

Links I found helpful:

FAQ and supported browsers &nbsp

Install ejabberd

apt-get install ejabberd

According to this, which is linked if you follow the install guide through,
we will need to update the path to the tls certificate.
Not sure where that is, but will have to find out.
the sample file contains the ejabberd configuration needed for Archipel.
It is not ready for production, so will need some modification. Yet to find out what.
Change all occurrences of FQDN to vmhost.mydomain.local and follow the other directions.

Once the ejabberd.cfg file is modified as suggested, download pscp.exe from here.
Put both the pscp.exe file and the ejabberd.cfg in the same folder (just to save typing paths and adding environment variables).
The help page is here if you get stuck.
Run a cmd prompt from the directory you have the 2 previous mentioned files within.
Then run:

pscp ejabberd.cfg myusername@

Enter your password when prompted.
The file will be securely copied via SSH to your ~ dir.
You can’t copy directly to the /etc/ejabberd/ directory as you would need to be root of the destination machine.
Now go to the Debian box. cd into ~.
and move the config file to where it belongs.

su root

Enter your password when prompted.

mv ejabberd.cfg /etc/ejabberd/ejabberd.cfg

Then check that the move was successful.

Start the jabber server if it’s not already.
As root:

/etc/init.d/ejabberd start

Wait a few seconds and run:

/usr/sbin/ejabberdctl status

And you should get a result of running, with the version details.

You need to register a XMPP admin account (if you want archipel to work out of the box, just name it admin):

ejabberdctl register admin vmhost.mydomain.local MyCrazyPassWordHere

You should get something like:

User admin@vmhost.mydomain.local successfully registered.

Although I didn’t the last time because I wasn’t running as root.

Continue with the Archipel installation

The client is easy, just fetch and un-compress and your ready to go.

The agent, you will need to install qemu-utils if it’s not already.
It was for me.

As root, run:

apt-get install python-setuptools python-imaging python-numpy python-libvirt

python-libvert is Python bindings for the libvirt library which was already installed.

I also installed subversion:

apt-get install subversion

Now… as root, I chose to install the published packages on Pypi.
I ran:

easy_install archipel-agent
Post installation formalities

Finalise the installation:


Follow the additional output instructions on the screen.

Now as root:

Create the pubsub nodes
archipel-tagnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-rolesnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-adminaccounts --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-vmparkingnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create

The last two commands were, introduced after beta 4, so they didn’t exist on the binary I installed.

You can now start the archipel agent.

/etc/init.d/archipel start

The logs are printed to /var/log/archipel/archipel.log

To be completely sure Archipel is up and your hypervisor is connected you can run:

ejabberdctl connected_users

If you choose to just dump the archipel client somewhere and browse to the index.html,
you will have to use Safari as the browser.
Alternatively, you can use Chrome,
but you need to pass the argument… –disable-web-security
Or the better way is to just uncompress the archive into a HTTP server directory,
and access it with your browser.
I’ve been told nginx works well with serving Archipel.
At this stage I just set the client up on IIS locally.
In saying that, I’m getting the index.html,
but I’m getting 404’s for Info.plist and main.j
I need to look into this.

Using Archipel

Once you have the page in your browser, enter the following details into the dialog.

Jabber ID: admin@vmhost.mydomain.local
Password: MyCrazyPassWordHere
BOSH service: http://vmhost.mydomain.local:5280/http-bind

If you can’t access vmhost, try navigating to http://vmhost.mydomain.local:5280/http-bind in your browser.

You should get something like the following:

If you don’t,
try pinging vmhost.mydomain.local.
If the IP works but the host.FQDN doesn’t, it’s a dns issue.
I checked the /etc/hosts file and it had the host name as expected.   vmhost.mydomain.local   vmhost

For some reason, the Debian box’s hostname wasn’t getting registered on the DNS server.
The way around this is to add the following entry to the hosts file of the machine you have your client running from.    vmhost.mydomain.local


Tags: , , , , , , , , ,

15 Responses to “Bare-metal Hypervisor Setup Evaluation”

  1. Virtualising your environment « The Small Business Company Says:

    […] my blog post for additional technical […]

  2. StylusPilot Says:

    Thank you for this. you also should look at oVirt.


    • binarymist Says:

      Are you using it StylusPilot?
      Looks like minimal support for Debian?
      IRC channel has one user on it.
      In regards of being ready for the enterprise, oVirt as with Archipel also, looks like it has quite a way to go?

      • styluspilot Says:

        yep, using oVirt in a Lab currently, just in the process of getting archipel run up and installed now for a side by side comparison. Archipel also has ANSOS now which is the same as the oVirt nodes so should be interesting.

      • binarymist Says:

        ANSOS looks interesting

  3. Vijay Says:

    when i am installing the Archipel..on my baremetail debian squzee server its giving error

    _imaging.c:3152: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘functions’
    _imaging.c:3250: warning: return type defaults to ‘int’
    _imaging.c: In function ‘DL_EXPORT’:
    _imaging.c:3250: error: expected declaration specifiers before ‘init_imaging’
    _imaging.c:3281: error: expected ‘{‘ at end of input
    error: Setup script exited with error: command ‘gcc’ failed with exit status 1

  4. Vijay Says:

    i think above line is correct answer..?

  5. Virtualising your environment » The Small Business Company Says:

    […] my blog post for additional technical […]

  6. NAS solutions for the home and office | tech tinker Says:

    […] which virtualization to choose? I spent some time mulling over the options as laid out at Bare-metal Hypervisor Setup Evaluation « Binarymist. I went so far as to install the 64-bit version of VMWare ESXi before I realised that there was no […]

  7. taren marquis Says:

    I’ll right away clutch your rss as I can’t in finding your e-mail subscription hyperlink
    or e-newsletter service. Do you’ve any? Please let me recognize in order that I may subscribe. Thanks.

  8. Nitrohuffer Says:

    KVM is complete shit, it is great for stoned college Kids in dorm rooms with no money. There is no value in free.

  9. styluspilot Says:

    What makes you say that? I know plently of large scale environments running on KVM without issue

    Any issues you have dealt with or just your opinion without actually using it?

  10. bielizna damska sklep Says:

    Does your blog have a contact page? I’m having problems locating it but, I’d like to shoot you an e-mail.
    I’ve got some creative ideas for your blog you might be interested in hearing. Either way, great website and I look forward to seeing it improve over time.

Leave a Reply to Nitrohuffer Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: