Archive for the ‘GNU/Linux’ Category

Up and Running with Express on Node.js … and friends

July 27, 2013

This is a result of a lot of trial and error, reading, notes taken, advice from more knowledgeable people than myself over a period of a few months in my spare time. This is the basis of a web site I’m writing for a new business endeavour.

Web Frameworks evaluated

  1. ExpressJS Version 3.1 I talked to quite a few people on the #Node.js IRC channel and the preference in most cases was Express. I took notes around the web frameworks, but as there were not that many good contenders, and I hadn’t thought about pushing this to a blog post at the time, I’ve pretty much just got a decision here.
  2. Geddy Version 0.6

MV* Frameworks evaluated

  1. CompoundJS (old name = RailwayJS) Version 1.1.2-7
  2. Locomotive Version 0.3.6. built on Express

At this stage I worked out that I don’t really need a server side MV* framework, as Express.js routes are near enough to controllers. My mind may change on  this further down the track, if and when it does, I’ll re-evaluate.

Templating Engines evaluated

  1. jade Version 0.28.2, but reasonably mature and stable. 2.5 years old. A handful of active contributors headed by Chuk Holoway. Plenty of support on the net. NPM: 4696 downloads in the last day, 54 739 downloads in the last week, 233 570 downloads in the last month (as of 2013-04-01). Documentation: Excellent. The default view engine when running the express binary without specifying the desired view engine. Discussion on LinkedIn. Discussed in the Learning Node book. Easy to read and intuitive. Encourages you down the path of keeping your logic out of the view. The documentation is found here and you can test it out here.
  2. handlebars Version 1.0.10 A handful of active contributors. NPM: 191 downloads in the last day, 15 657 downloads in the last week, 72 174 downloads in the last month (as of 2013-04-01). Documentation: Excellent: nettuts. Also discussed in Nicholas C. Zakas’s book under Chapter 5 “Loose Coupling of UI Layers”.
  3. EJS Most of the work done by the Chuk Holoway (BDFL). NPM: 258 downloads in the last day, 13 875 downloads in the last week, 56 962 downloads in the last month (as of 2013-04-01). Documentation: possibly a little lacking, but the ASP.NET syntax makes it kind of intuitive for developers from the ASP.NET world. Discussion on LinkedIn. Discussed in the “Learning Node” book by Shelley Powers. Plenty of support on the net. deoxxa from #Node.js mentioned: “if you’re generating literally anything other than all-html-all-the-time, you’re going to have a tough time getting the job done with something like jade or handlebars (though EJS can be a good contender there). For this reason, I ended up writing node-ginger a while back. I wouldn’t suggest using it in production at this stage, but it’s a good example of how you don’t need all the abstractions that some of the other libraries provide to achieve the same effects.”
  4. mu (Mustache template engine for Node.js) NPM: 0 downloads in the last day, 46 downloads in the last week, 161 downloads in the last month (as of 2013-04-01).
  5. hogan-express NPM: 1 downloads in the last day, 183 downloads in the last week, 692 downloads in the last month (as of 2013-04-01). Documentation: lacking

Middleware AKA filters

Connect

Details here https://npmjs.org/package/connect express.js shows that connect().use([takes a path defaulting to ‘/’ here], andACallbackHere) http://expressjs.com/api.html#app.use the body of andACallbackHere will only get executed if the request had the sub directory that matches the first parameter of connect().use

Styling extensions etc evaluated

  1. less (CSS3 extension and (preprocessor) compilation to CSS3) Version 1.4.0 Beta. A couple of solid committers plus many others. runs on both server-side and client-side. NPM: 269 downloads in the last day, 16 688 downloads in the last week, 74 992 downloads in the last month (as of 2013-04-01). Documentation: Excellent. Wiki. Introduction.
  2. stylus (CSS3 extension and (preprocessor) compilation to CSS3) Worked on since 2010-12. Written by the Chuk Holoway (BDFL) that created Express, Connect, Jade and many more. NPM: 282 downloads in the last day, 16 284 downloads in the last week, 74 500 downloads in the last month (as of 2013-04-01).
  3. sass (CSS3 extension and (preprocessor) compilation to CSS3) Version 3.2.7. Worked on since 2006-06. Still active. One solid committer with lots of other help. NPM: 12 downloads in the last day, 417 downloads in the last week, 1754 downloads in the last month (as of 2013-04-01). Documentation: Looks pretty good. Community looks strong: #sass on irc.freenode.net. forum. less, stylus, sass comparison on nettuts.
  • rework (processor) Version 0.13.2. Worked on since 2012-08. Written by the Chuk Holoway (BDFL) that created Express, Connect, Jade and many more. NPM: 77 downloads in the last week, 383 downloads in the last month (as of 2013-04-01). As explained and recommended by mikeal from #Node.js its basically a library for building something like stylus and less, but you can turn on the features you need and add them easily.  No new syntax to learn. Just CSS syntax, enables removal of prefixes and provides variables. Basically I think the idea is that rework is going to use the likes of less, stylus, sass, etc as plugins. So by using rework you get what you need (extensibility) and nothing more.

Responsive Design (CSS grid system for Responsive Web Design (RWD))

There are a good number of offerings here to help guide the designer in creating styles that work with the medium they are displayed on (leveraging media queries).

Keeping your Node.js server running

Development

During development nodemon works a treat. Automatically restarts node when any source file is changed and notifies you of the event. I install it locally:

$ npm install nodemon

Start your node app wrapped in nodemon:

$ nodemon [your node app]

Production

There are a few modules here that will keep your node process running and restart it if it dies or gets into a faulted state. forever seems to be one of the best options. forever usage. deoxxa’s jesus seems to be a reasonable option also, ningu from #Node.js is using it as forever was broken for a bit due to problems with lazy.

Reverse Proxy

I’ve been looking at reverse proxies to forward requests to different process’s on the same machine based on different domain names and cname prefixes. At this stage the picks have been node-http-proxy and NGinx. node-http-proxy looks perfect for what I’m trying to do. It’s always worth chatting to the hoards of developers on #Node.js for personal experience. If using Express, you’ll need to enable the ‘trust proxy’ setting.

Adding less-middleware

I decided to add less after I had created my project and structure with the express executable.
To do this, I needed to do the following:
Update my package.json in the projects root directory by adding the following line to the dependencies object.
“less-middleware”: “*”

Usually you’d specify the version, so that when you update in the future, npm will see that you want to stay on a particular version, this way npm won’t update a particular version and potentially break your app. By using the “*” npm will download the latest package. So now I just copy the version of the less-middleware and replace the “*”.

Run npm install from within your project root directory:

my-command-prompt npm install
npm WARN package.json my-apps-name@0.0.1 No README.md file found!
npm http GET https://registry.npmjs.org/less-middleware
npm http 200 https://registry.npmjs.org/less-middleware
npm http GET https://registry.npmjs.org/less-middleware/-/less-middleware-0.1.11.tgz
npm http 200 https://registry.npmjs.org/less-middleware/-/less-middleware-0.1.11.tgz
npm http GET https://registry.npmjs.org/less
npm http GET https://registry.npmjs.org/mkdirp
npm http 200 https://registry.npmjs.org/mkdirp
npm http 200 https://registry.npmjs.org/less
npm http GET https://registry.npmjs.org/less/-/less-1.3.3.tgz
npm http 200 https://registry.npmjs.org/less/-/less-1.3.3.tgz
npm http GET https://registry.npmjs.org/ycssmin
npm http 200 https://registry.npmjs.org/ycssmin
npm http GET https://registry.npmjs.org/ycssmin/-/ycssmin-1.0.1.tgz
npm http 200 https://registry.npmjs.org/ycssmin/-/ycssmin-1.0.1.tgz
less-middleware@0.1.11 node_modules/less-middleware
├── mkdirp@0.3.5
└── less@1.3.3 (ycssmin@1.0.1)

So you can see that less-middleware pulls in less as well.
Now you need to require your new middleware and tell express to use it.
Add the following to your app.js in your root directory.

var lessMiddleware = require('less-middleware');

and within your function that you pass to app.configure, add the following.

app.use(lessMiddleware({
   src : __dirname + "/public",
   // If you want a different location for your destination style sheets, uncomment the next two lines.
   // dest: __dirname + "/public/css",
   // prefix: "/css",
   // if you're using a different src/dest directory, you MUST include the prefix, which matches the dest public directory
   // force true recompiles on every request... not the best for production, but fine in debug while working through changes. Uncomment to activate.
   // force: true
   compress : true,
   // I'm also using the debug option...
   debug: true
}));

Now you can just rename your css files to .less and less will compile to css for you.
Generally you’ll want to exclude the compiled styles (.css) from your source control.

The middleware is made to watch for any requests for a .css file and check if there is a corresponding .less file. If there is a less file it checks to see if it has been modified. To prevent re-parsing when not needed, the .less file is only reprocessed when changes have been made or there isn’t a matching .css file.
less-middleware documentation

Bootstrap

Twitters Bootstap is also really helpful for getting up and running and comes with allot of helpful components and ideas to get you kick started.
Getting started.
Docs
.

Bootstrap-for-jade

As I decided to use the Node Jade templating engine, Bootstrap-for-Jade also came in useful for getting started with ideas and helping me work out how things could fit together. In saying that, I came across some problems.

ReferenceError: home.jade:23

body is not defined
    at eval (eval at <anonymous> (MySite/node_modules/jade/lib/jade.js:171:8), <anonymous>:238:64)
    at MySite/node_modules/jade/lib/jade.js:172:35
    at Object.exports.render (MySite/node_modules/jade/lib/jade.js:206:14)
    at View.exports.renderFile [as engine] (MySite/node_modules/jade/lib/jade.js:233:13)
    at View.render (MySite/node_modules/express/lib/view.js:75:8)
    at Function.app.render (MySite/node_modules/express/lib/application.js:506:10)
    at ServerResponse.res.render (MySite/node_modules/express/lib/response.js:756:7)
    at exports.home (MySite/routes/index.js:19:7)
    at callbacks (MySite/node_modules/express/lib/router/index.js:161:37)
    at param (MySite/node_modules/express/lib/router/index.js:135:11)
GET /home 500 22ms

I found a fix and submitted a pull request. Details here.

I may make a follow up post to this titled something like “Going Steady with Express on Node.js … and friends'”

Advertisements

Running Wireshark as non-root user

April 13, 2013

As part of my journey with Node.js I decided I wanted to see exactly what was happening on the wire. I decided to use Burp Suite as the Http proxy interceptor and Wireshark as the network sniffer (not an interceptor). Wireshark can’t alter the traffic, it can’t decrypt SSL traffic unless the encryption key can be provided and Wireshark is compiled against GnuTLS.

This post is targeted at getting Wireshark running on Linux. If you’re a windows user, you can check out the Windows notes here.

When you first install Wireshark and try to start capturing packets, you will probably notice the error “You didn’t specify an interface on which to capture packets.”

When you try to specify an interface from which to capture, you will probably notice the error “There are no interfaces on which a capture can be done.”

You can try running Wireshark as root: gksudo wireshark

Wireshark as root

This will work, but of course it’s not a good idea to run a comprehensive tool like Wireshark (over 1’500’000 lines of code) as root.

So what’s actually happening here?

We have dumpcap and we have wireshark. dumpcap is the executable responsible for the low level data capture of your network interface. wireshark uses dumpcap. Dumpcap needs to run as root, wireshark does not need to run as root because it has Privilege Separation.

If you look at the above suggested “better way” here, this will make a “little” more sense. In order for it to make quite a lot more sense, I’ll share what I’ve just learnt.

Wireshark has implemented Privilege Separation which means that the Wireshark GUI (or the tshark CLI) can run as a normal user while the dumpcap capture utility runs as root. Why can’t this just work out of the box? Well there is a discussion here on that. It doesn’t appear to be resolved yet. Personally I don’t think that anybody wanting to use wireshark should have to learn all these intricacies to “just use it”. As the speed of development gets faster, we just don’t have time to learn everything. Although on the other hand, a little understanding of what’s actually happening under the covers can help in more ways than one. Anyway, enough ranting.

How do we get this to all “just work”

from your console:

sudo dpkg-reconfigure wireshark-common

You’ll be prompted:

Configuring wireshark-common

Respond yes.

The wireshark group will be added

If the Linux Filesystem Capabilities are not present at the time of installing wireshark-common (Debian GNU/kFreeBSD, Debian GNU/Hurd), the installer will fall back to set the set-user-id bit to allow non-root users to capture packets. Custom built kernels may lack Linux Capabilities.

The help text also warns about a security risk which isn’t an issue because setuid isn’t used. Rather what actually happens is the following:

addgroup --quiet --system wireshark
chown root:wireshark /usr/bin/dumpcap
setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap

You will then have to manually add your user to the wireshark group.

sudo adduser kim wireshark # replacing kim with your user

or

usermod -a -G wireshark kim # replacing kim with your user

log out then back in again.

I wanted to make sure that what I thought was happening was actually happening. You’ll notice that if you run the following before and after the reconfigure:

ls -liah /usr/bin/dumpcap | less

You’ll see:

-rwxr-xr-x root root /usr/bin/dumpcap initially
-rwxr-xr-x root wireshark /usr/bin/dumpcap after

And a before and after of my users and groups I ran:

cat /etc/passwd | cut -d: -f1
cat /etc/group | cut -d: -f1

Alternatively to using the following as shown above, which gives us a nice abstraction (if that’s what you like):

sudo dpkg-reconfigure wireshark-common

We could just run the following:

addgroup wireshark
sudo chgrp wireshark /usr/bin/dumpcap
sudo chmod 750 /usr/bin/dumpcap
sudo setcap cap_net_raw,cap_net_admin+eip /usr/bin/dumpcap

The following will confirm the capabilities you just set.

getcap /usr/bin/dumpcap

What’s with the setcap?

For full details, run:

man setcap
man capabilities

setcap sets the capabilities of each specified filename to the capabilities specified (thank you man ;-))

For sniffing we need two of the capabilities listed in the capabilities man page.

  1. CAP_NET_ADMIN Perform various network-related operations (e.g., setting privileged socket options, enabling multicasting, interface configuration, modifying routing tables). This allows dumpcap to set interfaces to promiscuous mode.
  2. CAP_NET_RAW Use RAW and PACKET sockets. Gives dumpcap raw access to an interface.

For further details check out Jeremy Stretch’s explanation on Linux Filesystem Capabilities and using setcap. There’s also some more info covering the “eip” in point 2 here and the following section.

man capabilities | grep -A24 "File Capabilities"

Lets run Wireshark as our usual low privilege user

Now that you’ve done the above steps including the log off/on, you should be able to run wireshark as your usual user and configure your listening interfaces and start capturing packets.

Also before we forget… Ensure Wireshark works only from root and from a user in the “wireshark” group. You can add a temp user (command shown above).

Log in as them and try running wireshark. You should have the same issues as you had initially. Remove the tempuser:

userdel -r tempuser

Setup of Chromium, Burp Suite, Node.js to view HTTP on the wire

March 30, 2013

As part of my Node.js development I really wanted to see what was going over the wire from chromium-browser to my Node.js web apps.

I have node.js installed globaly, express installed locally, a very simple express server listening on port 3000

var express = require('express');
var app = express();

app.get('/', function (request, response) {
   response.send('Welcome to Express!');
});

app.listen(3000);

Burp Suite setup in my main menu. Added the command via System menu -> Preferences -> Main Menu

Burp Suite Command

The Command string looks like the following.

java -jar -Xmx1024m /WhereTheBurpSuiteLives/burpsuite_free_v1.5.jar

Setting up Burp Suite configuration details are found here. I’ve used Burp Suite before several times. Most notably to create my PowerOffUPSGuests library which I discuss here. In that usage I reverse engineered how the VMware vSphere client shuts down it’s guests and replicated the traffic in my library code. For a simple setup, it’s very easy to use. You can spend hours exploring Burps options and all the devious things you can use it for, but to get started it’s simple. Set it up to listen on localhost and port 3001 for this example.

Burp Suite Proxy Listeners

Run the web app

to start our express app from the directory where our above server is located, from a console, run:

node index.js

Where index.js is the name of the file that contains our JavaScript.

To test that our express server is active. We can browse to http://localhost:3000/ or we can curl it:

curl -i  http://localhost:3000/

Should give us something in return like:


HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 19
Date: Sun, 24 Mar 2013 07:53:38 GMT
Connection: keep-alive

Welcome to Express!

Now for the Proxy interception (Burp Suite)

Now that we’ve got end to end comms, lets test the interceptor.

Run burpsuite with the command I showed you above.

Fire the Http request at your web app via the proxy:

curl -i --proxy http://localhost:3001 http://localhost:3000/

Now you should see burps interceptor catch the request. On the Intercept tab, press the Forward button and curl should show a similar response to above.

Burp Suite Proxy Intercept

If you look at the History tab, you can select the message curl sent and also see the same Response that curl received.

Burp Suite Proxy History

Now you can also set Burp to intercept the server responses too. In fact Burp is extremely configurable. You can also pass the messages to different components of Burp to process how ever you see fit. As you can see in the above image looking at all the tabs that represent burp tools. These can be very useful for penetration testing your app as you develop it.

I wanted to be able to use chromium normally and also be able to open another window for browsing my express apps and viewing the HTTP via Burp Suite. This is actually quite simple. Again with your app running locally on port 3000 and burp listening on port 3001, run:

chromium-browser --temp-profile --proxy-server=localhost:3001

For more chromium options:

chromium-browser -help

Now you can just browse to your web app and have burp intercept your requests.

chromium proxied via burp

You may also want to ignore requests to your search provider, because as your typing in URL’s chromium will send searches when you pause. Under Proxy->Options tab you can do something like this:

Ignore Client Requests

Erasing data from your drives

March 17, 2013

Disclaimer

I take no responsibility for any damage caused by following any of the directions in this post. These tools and methods are destructive and likely to destroy your data or worse.

Deleting files from your drives does not remove them, it simply dereferences the memory. The data still exists. For further details, there is a good read here. This also covers some recovery tools.

Zero filling your disk/s

This is the process of setting all the bits on a drive to 0. Some say this is not the most secure way and that someone who knows what they’re doing can still in many cases recover the original data and that at least multiple passes of this technique are required. Others however disagree with this and say that a single pass is enough.
Thanks Miles for pointing this out and providing another view point.

dd

A cloning tool. AKA “data destroyer”.
To zero-fill: direct the output of the character file /dev/zero to the device you want zero-filled.

How?
Boot your machine from a live Linux disk that includes the dd programme. Most Linux distros will have dd included. I’ve done this using Knoppix as it loads reasonably fast.
From the shell terminal as root:

dd if=/dev/zero of=/dev/[device you want to wipe] bs=1M

/dev/zero, /dev/random and /dev/urandom are character special files. /dev/random and /dev/urandom are interfaces to the Linux kernel’s random number generator.

To find the device you want to wipe, run

fdisk

You’ll get something along these lines:

/dev/hda = primary master IDE
/dev/hdb = primary slave IDE
/dev/hdc = secondary master IDE
/dev/hdd = secondary slave IDE
/dev/sda = first SCSI hard drive
/dev/sdb = second SCSI hard drive

So for example if you want to zero your primary master:

sudo dd if=/dev/zero of=/dev/hda bs=1M

UBCD

AKA Ultimate Boot CD.
Once you’ve downloaded UBCD and have it written to your boot media and have your machine booted into it.
Press F2 to enter the Hard Drive tool section.
Press right arrow key to enter the diagnostic tools.
Select the most recent version of the diagnostic tool under the name of the manufacturer of your drive.

Applying patterns to the bits

A more effective approach to zero-filling, is to use bit flipping patterns in your wiping approach and perform multiple passes.

dd if=/dev/random of=/dev/[device you want to wipe] bs=1M

should be a little more effective.

Better still, run the following 3 – 7 times, as discussed here


dd if=/dev/random of=/dev/[device you want to wipe] && dd if=/dev/zero of=/dev/[device you want to wipe]

Wipe

I haven’t used this, but it looks good.

dban

Recommended by Stanford University’s Disk and Data Sanitisation Policy and Guidelines.
Stanford also lists a collection of other useful disk sanitisation tools.
Download the iso from https://sourceforge.net/projects/dban/
Burn the image to a CD / DVD or USB drive (using something like ISO to USB.
Set your BIOS to boot from which ever device has the ISO image.

Once dban loads you’ll be given options to proceed.
dban start options

I hit Enter to start in interactive mode.

In the next screen, you’ll be able to see that dban is using urandom as it’s Entropy. This must be /dev/urandom which will be used to set your bits on/off randomly rather than just zeroing or oneing (probably not a word ;-)).
This is considered a far better technique to make it forensically close to impossible to reconstruct the original contents of the disk.

NukeOptions

In this screen you can also select other options.
Method: allows you to use a selection of different techniques.
The current default is DoD Short.
Both DoD 5220.22-M Short and DoD 5220.22-M Standard are used by the American Department of Defense
DoD 5220.22-M Short performs 3 passes
DoD 5220.22-M Standard performs 7 passes

See here for the standards for data erasure

Once dban has performed the sanitisation, you’ll see a screen similar to the following with the details

FinishDetails

As always, feel free to offer corrections and comments on things I may have missed out that you think worth mentioning.

How to Increase Software Developer Productivity

March 2, 2013

Is your organisation:

  • Wanting to get more out of your Software Developers?
  • Wanting to increase RoI?
  • Spending too much money fixing bugs?
  • Development team not releasing business value fast enough?
  • Maybe your a software developer and you want to lift your game to the next level?

If any of these points are of concern to you… read on.

There are many things we can do to lift a software developers productivity and thus the total output of The Development Team. I’m going to address some quick and cheap wins, followed by items that may take a little longer to implement, but non the less, will in many cases provide even greater results.

What ever it takes to remove friction and empower your software developers to work with the least amount of interruptions, do it.
Allow them to create a space that they love working in. I know when I work from home my days are far more productive than when working for a company that insists on cramming as many workers around you into a small space as possible. Chitter chatter from behind, both sides and in front of you will not help one get their mind into a state of deep thought easily.

I have included thoughts from Nicholas C. Zakas post to re-iterate the common fallacies uttered by non-engineers.

  • I don’t understand why this is such a big deal. Isn’t it just a few lines of code? (Technically, everything is a few lines of code. That doesn’t make it easy or simple.)
  • {insert name here} says it can be done in a couple of days. (That’s because {insert name here} already has perfect knowledge of the solution. I don’t, I need to learn it first.)
  • What can we do to make this go faster? Do you need more engineers? (Throwing more engineers at a problem frequently makes it worse. The only way to get something built faster is to build a smaller thing.)

Screen real estate

When writing code, a software developers work requires a lot of time spent deep in thought. Holding multiple layers of complexity within immediately accessible memory.
One of the big wins I’ve found that helps with continuity, is maximising your screen real estate.
I’ve now moved up to 3 x 27″ 2560×1440 IPS flat panels. These are absolutely gorgeous to look at/work with.
Software development generally requires a large number of applications to be running at any one time.
For example in any average session for me, I generally have somewhere around 30 windows open.
The more screen real estate a developer has, the less he/she has to fossick around for what he/she needs and switch between them.
Also, the less brain cycles he/she has to spend locating that next running application, means the more cycles you have in order to do real work.
So, the less gap there is switching between say one code editor and another, the easier it is for a developer to keep the big picture in memory.
We’re looking at:

  1. physical screen size
  2. total pixel count

The greater real estate available (physical screen size and pixel count) the more information you can have instant access to, which means:

  • less waiting
  • less memory loss
  • less time spent rebuilding structures in your head
  • greater continuity

Which then gives your organisation and developers:

  • greater productivity
  • greater RoI

These screens are cheaper than many realise. I set these up 4 months ago. They continue to drop in price.

  1. FSM-270YG 27″ PC Monitor LED S-IPS WIDE 2560×1440 16:9 WQHD DVI-D $470.98 NZD
  2. [QH270-IPSMS] Achieva ShiMian HDMI DVI D-Sub 27″ LG LED 2560×1440 $565.05 NZD
  3. [QH270-IPSMS] Achieva ShiMian HDMI DVI D-Sub 27″ LG LED 2560×1440 $565.05 NZD

It’s just simply not worth not to upgrading to these types of panels.

korean monitors

In this setup, I’m running Linux Mint Maya. Besides the IPS panels, I’m using the following hardware.

  • Video card: 1 x Gigabyte GV-N650OC-2GI GTX 650 PCIE
  • PSU: 1200w Corsair AX1200 (Corsair AX means no more PSU troubles (7 yr warranty))
  • CPU: Intel Core i7 3820 3.60GHz (2011)
  • Mobo: Asus P9X79
  • HDD: 1TB Western Digital WD10EZEX Caviar Blue
  • RAM: Corsair 16GB (2x8GB) Vengeance Performance Memory Module DDR3 1600MHz

One of the ShiMian panels is using the VGA port on the video card as the FSM-270YG only supports DVI.
The other ShiMian and the FSM-270YG are hooked up to the 2 DVI-D (dual link) ports on the video card. The two panels feeding on the dual link are obviously a lot clearer than the panel feeding on the VGA. Also I can reduce the size of the text considerably giving me greater clarity while reading, while enabling me to fit a lot more information on the screens.

With this development box, I’m never left waiting for the machine to catchup with my thought process.
So don’t skimp on hardware. It just doesn’t make sense any way you look at it.

Machine Speed

The same goes for your machine speed. If you have to wait for your machine to do what you’ve commanded it to do and at the same time try and keep a complex application structure in your head, the likelihood of loosing part of that picture increases. Plus your brain has to work harder to hold the image in memory while your trying to maintain continuity of thought. Again using precious cycles for something that shouldn’t be required rather than on the essential work. When a developer looses part of this picture, they have to rebuild it again when the machine finishes executing the last command given. This is re-work that should not be necessary.

An interesting observation from Joel Spolsky:

“The longer it takes to task switch, the bigger the penalty you pay for multitasking.
OK, back to the more interesting topic of managing humans, not CPUs. The trick here is that when you manage programmers, specifically, task switches take a really, really, really long time. That’s because programming is the kind of task where you have to keep a lot of things in your head at once. The more things you remember at once, the more productive you are at programming. A programmer coding at full throttle is keeping zillions of things in their head at once: everything from names of variables, data structures, important APIs, the names of utility functions that they wrote and call a lot, even the name of the subdirectory where they store their source code. If you send that programmer to Crete for a three week vacation, they will forget it all. The human brain seems to move it out of short-term RAM and swaps it out onto a backup tape where it takes forever to retrieve.”

Many of my posts so far have been focused on productivity enhancements. Essentially increasing RoI. This list will continue to grow.

Coding Standards and Guidelines

Agreeing on a set of Coding Standards and Guidelines and policing them (generally by way of code reviews and check-in commit scripts) means software developers get to spend less time thinking about things that they don’t need to and get to throw more time at the real problems.

For example:

Better Tooling

Improving tool sets has huge gains in productivity. In most cases many of the best tools are free. Moving from the likes of non distributed source control systems to best of bread distributed.

There are many more that should be considered.

Wiki

Implementing an excellent Wiki that is easy to use. I’ve put a few wiki’s in place now and have used even more. My current pick of the bunch would have to be Atlassians Confluence. I’ve installed this on a local server and also migrated the instance to their cloud. There are varying plans and all very reasonably priced with excellent support. If the wiki you’re planning on using is not as intuitive as it could be, developers just wont use it. So don’t settle for anything less.

Improving Processes

Code Reviews

Also a very important step in all successful development teams and often a discipline that must be satisfied as part of Scrums Definition of Done (DoD). What this gives us is high quality designs and code, conforming to the coding standards. This reduces defects, duplicate code (DRY) and enforces easily readable code as the reviewer has to understand it. Saves a lot of money in re-work.

Cost of Change

Scott Amblers Cost of change curve

Definition of Done (DoD)

Get The Team together and decide on what it means to have each Product Backlog Item that’s pulled into the Sprint Done.
Here’s an example of a DoD that one of my previous Development Teams compiled:

Definition of Done

What does Done actually mean?

Come Sprint Review on the last day of the Sprint, everyone knows what it means to be done. There is no “well I thought it was Done because I’ve written the code for it, but it’s not tested yet”.

Continuous Integration (CI)

There are many tools and ways to implement CI. What does CI give you? Visibility of code quality, adherence to standards, reports on cyclomatic complexity, predictability and quite a number of other positive side effects. You’ll know as soon as the code fails to build and/or your fast running tests (unit tests) fail. This means The Development Team don’t keep writing code on top of faulty code, thus reducing technical debt by not having to undo changes on changes later down the track.
I’ve used a number of these tools and have carried out extensive research and evaluation spikes on a number of the most popular offerings. In order of preference, the following are my candidates.

  1. Jenkins (free and open source, with a great community)
  2. TeamCity
  3. Atlassian Bamboo

Release Plans

Make sure you have these. This will reduce confusion and provide a clear definition of the steps involved to get your software out the door. This will reduce the likelihood of screwing up a release and re-work being required. You’ll definitely need one of these for the next item.

Here’s an example of a release notes guideline I wrote for one of the previous companies I worked for.

release notes

Continuous Deployment

If using Scrum, The Scrum Team will be forecasting a potentially releasable Increment (the sum of all the Product Backlog items completed during a Sprint and all previous Sprints).
You may decide to actually release this. When you do, you can look at the possibility of automating this deployment. Thus reducing the workload of the release manager or who ever usually deploys (often The Development Team in a Scrum environment). This has the added benefit of consistency, predictability, reliability and of course happy customers. I’ve also been through this process of research and evaluation on the tools available and the techniques to implement.

Here’s a good podcast that got me started. I’ve got a collection of other resources if you need them and can offer you my experience in this process. Just leave a comment.

Implement Scrum (and not the Flaccid flavour)

I hope this goes without saying?
Implementing Scrum to provide ultimate visibility

Get maximum quality out of the least money spent

How to get the most out of your limited QA budget

Driving your designs with tests, thus creating maintainable code, thus reducing technical debt.

Hold Retrospectives

Scrum is big on continual inspection and adaption, self-organisation and fostering innovation. The military have another term for inspection and adaption. It’s called the OODA Loop.
The Retrospective is just one of the Scrum Events that enable The Scrum Team to continually inspect the way they are doing things and improve the way they develop and deliver business value.

Invest a little into your servant leaders

Empowering the servant leaders.

Context Switching

Don’t do it. This is a real killer.
This is hard. What you need to do is be aware of how much productivity is killed with each switch. Then do everything in your power to make sure your Development Team is sheltered from as much as possible. There are many ways to do this. For starters, you’re going to need as much visibility as possible into how much this is currently happening. track add-hock requests and any other types of interruptions that steel the developers concentration. In the last Scrum Team that I was Scrum Master of, The Development Team decided to include another metric to the burn down chart that was on the middle of the wall, clearly visible to all. Every time one of the developers was interrupted during a Sprint, they would record this time, the reason and who interrupted them, on the burn down chart. The Scrum Team would then address this during the Retrospective and empirically address why this happened and work out how to stop it happening every Sprint. Jeff Atwood has an informative post on why and how context-switching/multitasking kills productivity. Be sure to check it out.

As always, if anything I’ve mentioned isn’t completely clear, or you have any questions, please leave a comment 🙂

Establishing your SSH Server’s Key Fingerprint

February 16, 2013

When you connect to a remote host via SSH that you haven’t established a trust relationship with before,
you’re going to be told that the authenticity of the host your attempting to connect to can’t be established.

me@mybox ~ $ ssh me@10.1.1.40
The authenticity of host '10.1.1.40 (10.1.1.40)' can't be established.
RSA key fingerprint is 23:d9:43:34:9c:b3:23:da:94:cb:39:f8:6a:95:c6:bc.
Are you sure you want to continue connecting (yes/no)? y
Please type 'yes' or 'no':

Do you type yes to continue without actually knowing that it is the host you think it is? Well, if you do, you should be more careful. The fingerprint that’s being put in front of you could be a Man In The Middle (MITM). You can query the target (from “it’s” shell of course) for the fingerprint of it’s key easily. On Debian you’ll find the keys in /etc/ssh/

On

ls /etc/ssh/

you should get a listing that reveals the private and public keys. Run the following command on the appropriate key to reveal it’s fingerprint. For example if SSH is using rsa:

ssh-keygen -lf ssh_host_rsa_key.pub

For example if SSH is using dsa:

ssh-keygen -lf ssh_host_dsa_key.pub

If you try the command on either the private or publick key you’ll be given the public key’s fingerprint, which is exactly what you need for verifying the authenticity from the client side.

Sometimes you may need to force the output of the fingerprint_hash algorithm as ssh-keygen may be displaying it in a different form than it’s shown when you try to SSH for the first time. The default when using ssh-keygen to show the key fingerprint is sha256, but in order to compare apples with apples you may need to specify md5 if that’s what’s being shown when you attempt to login. You would do that like the following:

ssh-keygen -lE md5 -f ssh_host_dsa_key.pub

Details on the man page for the options.

Do not connect remotely and then run the above command, as the machine you’re connected to is still untrusted. The command could be dishing you up any string replacement if it’s an attackers machine. You need to run the command on the physical box or get someone you trust (your network admin) to do this and hand you the fingerprint.

Now when you try to establish your SSH connection for the first time, you can check that the remote host is actually the host you think it is by comparing the output of one of the previous commands with what SSH on your client is telling you the remote hosts fingerprint is. If it’s different it’s time to start tracking down the origin of the host masquerading as the address your trying to hook up with.

Now, when you get the following message when attempting to SSH to your server, due to something or somebody changing the hosts key fingerprint:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
23:d9:43:34:9c:b3:23:da:94:cb:39:f8:6a:95:c6:bc.
Please contact your system administrator.
Add correct host key in /home/me/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/me/.ssh/known_hosts:6
  remove with: ssh-keygen -f "/home/me/.ssh/known_hosts" -R 10.1.1.40
RSA host key for 10.1.1.40 has changed and you have requested strict checking.
Host key verification failed.

The same applies. Check that the fingerprint is indeed the intended target hosts key fingerprint. If it is, run the specified command.

Painless git diff

February 2, 2013

I’ve been using Node.js quite a bit lately and decided it was time to start using Git for my projects.

I’m used to using Mercurial (Hg) for DVCS, but have only used it on Windows and a little on Linux via command line.

I was looking for a similar experience that Windows gave me for Hg (file explorer integration with tortoisehg), but for Linux. I had created a repository using tortoisehg. When I attempted to add files to the repository using tortoisehg or straight from the command line, I was getting a few errors. tortoisehg, nautilus integration is broken on my distro at the time of writing this too. So this encouraged me to invest a little more time in Git. I had done a bit of reading and listened to a few good podcasts on Git, so I felt it was a good time.
think like a git is also good for a read.

As I was creating repositories, dealing with remote repositories, cloning, setting up all the config files, adding, committing, pulling, pushing, viewing status and diffing. What I quickly came to realise, was that the Git commands were very extensive, made more sense to me than Hg, and there is a lot of good documentation around. In saying that, it’s been a while since I used hg from the command line and most of my work has been through the GUI tools.

One area I was struggling with was the diffing of files and directories on the command line. There are a couple of good ways to make this experience a lot more pleasurable.
I like using meld on Linux for my file and directory comparisons, so already had that installed.

git diff

Create a bash file in the /bin directory.
I called it git-meld, and it looks like the following:

#!/bin/bash
meld $2 $5

Turn the executable bit on, so it can be executed.

chmod git-meld +x

Now modify your ~/.gitconfig file

git config --global diff.external git-meld

To make sure your’ve added git-meld as the script that’ll run meld with the correct parameters:

cat .gitconfig

and you should see at least the following:

[diff]
external = git-meld

Now that should be all you need to get git to pop meld on diff.

git diff [options] <commit> <commit> [<path_to_file_to_compare>]

If you have a stack of files (rather than just one, as shown in my above example) that were changed between these commits, diff will pop each file open in meld. One at a time until you’ve finished with each one

meld

git difftool

git also comes with difftool. I found this really nice to use. There is no setting up for it. All you do is replace the diff command with difftool. Optionally you can specify the GUI diff tool you want to use, simply by appending -t [your_GUI_diff_tool] like this if you like using meld.

git difftool -t meld <commit> <commit> [<path_to_file_to_compare>]

If you do this without specifying the file you want to compare, you are prompted if you want to view each file, rather than how diff works by just opening every one.

Launchmeld

If you choose to leave the -t option out, difftool will give you the option of all the possible tools able to perform the diff (some of which may need installation).

multiple diff tools

So using difftool is a better diff IMHO. This is how git difftool behaves whether or not you set up ~/.gitconfig file with your prefered diff tool.

A Decent Console for Windows

January 19, 2013

On *nix we’re kind of spoilt when it comes to the CLI experience.
The console I use most in a GUI environment is the great terminator.

terminator

No, not that one.
This one

terminator

Multi tab, split screen, transparency, the works.
Then we’ve also got tmux (and a comparison between terminator and tmux).
Taking things further, we’ve got awesome

Well I’ve been looking for something similar for Windows for a while.
I’ve tried terminator on Cygwin, but it’s just not the same, plus it only supports the single shell.

Meet Console2

Console2 PS

With PowerShell as the currently active tab.
It’s a stand alone executable and crucially it’s free.
Console2 is just that, a console or terminal that seems to be able to host any shell that’s thrown at it.
As you can see with the image above, I’ve setup Console2 to host the following shells:

  1. Windows Command shell
  2. The Visual Studio Command Prompt (which is just the Windows Command shell (with some paths and variables added?))
  3. PowerShell
  4. The node.js Read-Eval-Print-Loop (REPL)
  5. VMwares vSphere PowerCLI
  6. And of course the bash shell we all know and love.
    Cygwin required

Although project activity looks minimal to non existent currently.

How I setup Console2

Once running, right click the console -> Edit -> Settings…

  • Setup the hot keys under the Hotkeys node to behave like the terminal I use on Linux (currently terminator).
    • Select the specific command, put your cursor in the Hotkey text box, press your preferred key combination, press the Assign key.
    • For opening new tabs I use Ctrl+Shift+T
    • Change the Copy selection node to what it should be: Ctrl+C
    • Change the Past to what it should be: Ctrl+V
  • Under the Console node, enter the directory to have each shell start in
  • Under the Appearance/More… node, I deselect the Show menu, Show toolbar and Show status bar
    • I make sure the Window transparency is set to None, as it just distracts me being able to see stuff behind the surface I’m concentrating on.
      It looks cool to turn it on, but I personally find it harder to read the text when you’ve got to lots of text overlapping
    • Under the Behavior node, I turn on Copy on select, as this is on in Linux by default
  • Now under the Tabs node is where we set up all of our shells.
    • Click the Add button
    • Change the name you want the shell tab to appear as in the Main tab under the Title text box
    • Now for the Icons I just got images I wanted for them and opened them in GIMP and changed the size to 32×32 pixels and saved as .ico files to the same directory that the Console.exe runs from
    • I Then select them here
    • Under the Shell section I just copy the short cuts from the likes of the start menu and past them in there
    • You can then override the default startup dir by specifying your path in the Startup dir text box
    • You can also specify if you want the shell to run as a specific user. Administrator for example.
      When you run this shell, you’ll be prompted for the users credentials if it’s not you.

As I was working through the Console2 set up, I ran into another offering…

Meet ConEmu

The actively maintained ConEmu lives here.
I had a quick play with this and a flick through the documentation.
The simple tasks of setting up different shells as pre-sets seemed to evade me.
There seems to be a lot more configuration options too.
As I’d just set up the Console2 and it seemed to be doing everything I needed for now, I decided to call it quits with ConEmu.
I think it’s worth checking out though if you need more power than Console2.
Scott Hanselmans post on Conemu.

copying with scp

March 25, 2012

I was having some trouble today copying a file (1.5GB .iso) from a notebook to a file server.
The notebook I was using was running Linux Ubuntu.
The server FreeBSD.
I was trying to copy this file using SMB/CIFS via Nautilus.
I tried several times, it failed each time.
Then I thought, what are you doing… drop to the command line.

scp to the rescue

The command I used:

From the directory on my local machine I was copying the file from

scp -P <MyPortNumberHere> MyFile.iso <MyUserName>@<MyServer>:/Path/To/Where/I/Want/MyFile/ToGo/MyFile.iso

This also took about half  the time to copy that SMB took, and SMB didn’t even complete. Not to mention the transfer is secure (SSH)

Some additional resources

http://www.linuxtutorialblog.com/post/ssh-and-scp-howto-tips-tricks

http://amath.colorado.edu/computing/software/man/scp.html

Also don’t forget to check the man page out 😉

man scp

Bare-metal Hypervisor Setup Evaluation

January 23, 2012

The views expressed in this post are my own and don’t reflect the views of my employer.

Recently I had the opportunity for work, to carry out some research on what’s in the market in regards to bare-metal hypervisors.

The following is the result of an in depth research and deployment project of the following bare-metal hyper-visors.
This will enable us to trial the hypervisors out for performance, ease of setup, ease of administration, and ease of use.

I’ve also looked at hardware costs, but first it needs to be decided which hypervisor we are going to go with.
As this would be a team decision, I thought the best way to go about this was to record some of my existing experience with further research into some of the product leaders offerings.

I haven’t used KVM before.
I knew it existed, but when I was last in the market comparing hypervisors, KVM was an infant.
Now it appears to have grown up and is comparable with it’s commercial rivals.
This pretty much sums up the KVM vs VMware battle
This pretty much sums up the Xen vs KVM battle


ESX(i)

I’ve used these extensively and am well aware of their pros and cons.
Supports iscsi.
I prefer not to have to pay for a product if there are FOS (Free & Open Source) offerings that get the job done just as well.
In looking at the likes of KVM and Xen, the cons of ESX/ESXi really stand out, not to mention the fact that KVM is completely free, more efficient and has a faster pace of growth.
With the free version, that’s ESXi, you get (as of version 5) 32GB vRAM, and that’s only because the community kicked up such a fuss about paying per CPU for a product that was originally free.
VMware keep changing the rules and pricing strategies when users go else where. I’d prefer not to pay at all.
I’m not going to spend time recording the pros and cons of VMware at this stage, as I think the other contenders have more to offer, and ask for less or nothing in return.
If we find that there are un-foreseen hurdles in the other products, we should look at ESXi as a backup.

Management

vSphere client (only runs on windows).
vSphere CLI (read-only, unless you pay for license)
Have very limited access to the hypervisor

Migration

  • General
  • Potential migration of KVM to VMware.
    Although this link says  the above won’t work, but has some other suggestions.

UPS

See my blog posts.


Citrix XenServer

XenServer support for iscsi

Xen is a type 1 bare-metal hypervisor. This means it runs as close to the hardware as possible.
To take full advantage of it’s speed, you have to run paravirtualised (modified OS’s).
Since most of our work at this stage would be on Windows, there would be no benefit here for us.
Runs in a small custom Linux system.
Intel VT-x or AMD-V is required to run full hardware virtualisation (HVM) rather than paravirtualised.

Licensing for XenServer Express

Be aware, Citrix can change their licensing structure at any time.
Features and current licensing model
XenServer Licensing FAQ
XenCenter can only connect to a single instance of XenServer at any one time.
XenServer currently free
XenCenter free
http://www.citrix.com/English/NE/news/news.asp?newsID=1687130

FAQ

Management

Migration

ESX(i) to XenServer

Seemed to have struggles (windows guest).
Seemed to be a little more successful (windows guest).

UPS

Integrating XenServer and APC PowerChute. Also see this.
Using apcupsd as KVM can.

Installation Stage

The getting started page. You can find the quick installation guide here.

The full installation guide.
The Administrators guide.

Download and install XenServer on your host.
Download and install XenCenter on your management box.

You’ll need the following details:

  1. Hostname
  2. Host IP and mask
  3. Gateway
  4. DNS Server
  5. NTP Address

This was a very straight forward install.
I was expecting some trouble, but there wasn’t any.


KVM

KVM has support for iscsi.
Expected to run all production OS’s.
Why will KVM be the leader amongst hypervisors?

Interesting articles:

Is completely free.
Considerably more resource efficient than the alternatives
There are no resource constraints. We pay for nothing and get an enterprise level product with a huge community.

KVM on Debian

Management

Web based KVM management offerings of which ProxMox VE seems to be the stand-out.
Many of these can also be used for Xen. Also see this.

ProxMoxVE

ProxMox is a commercial company.
ProxMox VE Looks Good.
From what I’ve seen, looks easier to setup than Archipel.
Proxmox VE is licensed under GPLv2 (Open source).
My understanding of the GPLv2 license, is that the suplier of the GPL’d software can decide to charge a fee for download at any time.
As far as I’m aware, Proxmox are within their rights to do so at any time.
Correct me if I’m wrong?
The ISO installer is packaged with Debian, although you can install on top of Debian.
Looks User friendly, has Web interface (multi platform). No installs required.
Support: incl free community and paid for. See here and here.
The wiki
Looks like what ever you can do on a Debian system, you can do on a ProxMox system.
See this link. Also includes ESXi comparisons.
Proxmox VE is free to use and open source.
Easy backups and restores.
Video tutorials here and here.

Archipel

Archipel Also looks good.
Free and Open Source, licensed under AGPL (which more specifically targets distributed applications).
Team of 6 voluntary developers. Lots of info here.
Supports all libvirt-supported virtualisation engines like KVM, Xen, VMware
The install on first appearance, looks more work than ProxMox.
Documentation, IRC channel (members are very helpful), etc.
The Archipel client is JavaScript, which is run locally.

Industry support

KVM is supported by major industry players such as…

  1. IBM
  2. Cisco
  3. Intel
  4. AMD
  5. Redhat
  6. Novell amongst others.

Migration

Looks like migration of guests from most platforms to KVM is covered.
VMware to Proxmox, XenServer to Proxmox.

UPS

Can be shutdown by an APC Smart-UPS
using the APCUPSD daemon This will shutdown immediately.
Or better, by using PCNS for Linux.
Using PCNS we can specify when to shutdown and all sorts of other things.

Installation Stage Archipel

Links found useful for the Debian setup

http://www.debian-tutorials.com/virtualization/kvm-virtualization-on-debian-squeeze-server

http://wiki.debian.org/KVM

http://wiki.libvirt.org/page/Networking#Bridged_networking_.28aka_.22shared_physical_device.22.29

http://wiki.kartbuilding.net/index.php/KVM_Setup_on_Debian_Squeeze

Setting up Debian

Download Debian Wheezy from here
Install it.
Give it a hostname. For example “vmhost” without the quotes.
When prompted, select the SSH Server option.
Update your package index and install the necessary packages.

As root, run:

apt-get update
apt-get install qemu-kvm libvirt-bin virtinst virt-top

virtinst is for virt-install tools etc.
qemu-kvm is the new name for the kvm package in squeeze
libvirt-bin is what will control kvm and start guests on boot etc.
virt-top is a ‘top’-like utility for virtualisation stats

Add user to groups

Add the currently logged in user that will be using the associated programmes.

usermod -a -G libvirt myusername
usermod -a -G kvm myusername

Then check that the user was added to the groups.

groups myusername

or

id myusername

or view all users in all groups

cat /etc/group | less
Setup networking

Your /etc/network/interfaces needs to have a similar section:
As root, run the following…

vi /etc/network/interfaces

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
   address 192.168.1.20
   netmask 255.255.255.0
   gateway 192.168.1.254
   broadcast 192.168.1.255

Now restart your interface:

ifdown eth0
ifup eth0

Check that the changes have taken affect:

ip addr show
Setup Bridged networking

You also need to set up a network bridge on our server.
Rather than use NAT based connectivity, we need bridge networking.

install the package bridge-utils.

apt-get install bridge-utils

I’ve yet to set the bridge up.
Will add this once done

Setting up Archipel

Links I found helpful:

FAQ and supported browsers
https://github.com/primalmotion/Archipel/wiki/General%3A-FAQ&nbsp;&nbsp

Install ejabberd

apt-get install ejabberd

According to this, which is linked if you follow the install guide through,
we will need to update the path to the tls certificate.
Not sure where that is, but will have to find out.
the sample file contains the ejabberd configuration needed for Archipel.
It is not ready for production, so will need some modification. Yet to find out what.
Change all occurrences of FQDN to vmhost.mydomain.local and follow the other directions.

Once the ejabberd.cfg file is modified as suggested, download pscp.exe from here.
Put both the pscp.exe file and the ejabberd.cfg in the same folder (just to save typing paths and adding environment variables).
The help page is here if you get stuck.
Run a cmd prompt from the directory you have the 2 previous mentioned files within.
Then run:

pscp ejabberd.cfg myusername@192.168.1.20:ejabberd.cfg

Enter your password when prompted.
The file will be securely copied via SSH to your ~ dir.
You can’t copy directly to the /etc/ejabberd/ directory as you would need to be root of the destination machine.
Now go to the Debian box. cd into ~.
and move the config file to where it belongs.

su root

Enter your password when prompted.

mv ejabberd.cfg /etc/ejabberd/ejabberd.cfg

Then check that the move was successful.

Start the jabber server if it’s not already.
As root:

/etc/init.d/ejabberd start

Wait a few seconds and run:

/usr/sbin/ejabberdctl status

And you should get a result of running, with the version details.

You need to register a XMPP admin account (if you want archipel to work out of the box, just name it admin):

ejabberdctl register admin vmhost.mydomain.local MyCrazyPassWordHere

You should get something like:

User admin@vmhost.mydomain.local successfully registered.

Although I didn’t the last time because I wasn’t running as root.

Continue with the Archipel installation

The client is easy, just fetch and un-compress and your ready to go.

The agent, you will need to install qemu-utils if it’s not already.
It was for me.

As root, run:

apt-get install python-setuptools python-imaging python-numpy python-libvirt

python-libvert is Python bindings for the libvirt library which was already installed.

I also installed subversion:

apt-get install subversion

Now… as root, I chose to install the published packages on Pypi.
I ran:

easy_install archipel-agent
Post installation formalities

Finalise the installation:

archipel-initinstall

Follow the additional output instructions on the screen.

Now as root:

Create the pubsub nodes
archipel-tagnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-rolesnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-adminaccounts --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create
archipel-vmparkingnode --jid=admin@vmhost.mydomain.local --password=MyCrazyPassWordHere --create

The last two commands were, introduced after beta 4, so they didn’t exist on the binary I installed.

You can now start the archipel agent.

/etc/init.d/archipel start

The logs are printed to /var/log/archipel/archipel.log

To be completely sure Archipel is up and your hypervisor is connected you can run:

ejabberdctl connected_users

If you choose to just dump the archipel client somewhere and browse to the index.html,
you will have to use Safari as the browser.
Alternatively, you can use Chrome,
but you need to pass the argument… –disable-web-security
Or the better way is to just uncompress the archive into a HTTP server directory,
and access it with your browser.
I’ve been told nginx works well with serving Archipel.
At this stage I just set the client up on IIS locally.
In saying that, I’m getting the index.html,
but I’m getting 404’s for Info.plist and main.j
I need to look into this.

Using Archipel

https://github.com/primalmotion/Archipel/wiki/User-manual

Once you have the page in your browser, enter the following details into the dialog.

Jabber ID: admin@vmhost.mydomain.local
Password: MyCrazyPassWordHere
BOSH service: http://vmhost.mydomain.local:5280/http-bind

If you can’t access vmhost, try navigating to http://vmhost.mydomain.local:5280/http-bind in your browser.

You should get something like the following:

If you don’t,
try pinging vmhost.mydomain.local.
If the IP works but the host.FQDN doesn’t, it’s a dns issue.
I checked the /etc/hosts file and it had the host name as expected.

127.0.1.1   vmhost.mydomain.local   vmhost

For some reason, the Debian box’s hostname wasn’t getting registered on the DNS server.
The way around this is to add the following entry to the hosts file of the machine you have your client running from.

192.168.1.20    vmhost.mydomain.local