Posts Tagged ‘Linux’

copying with scp

March 25, 2012

I was having some trouble today copying a file (1.5GB .iso) from a notebook to a file server.
The notebook I was using was running Linux Ubuntu.
The server FreeBSD.
I was trying to copy this file using SMB/CIFS via Nautilus.
I tried several times, it failed each time.
Then I thought, what are you doing… drop to the command line.

scp to the rescue

The command I used:

From the directory on my local machine I was copying the file from

scp -P <MyPortNumberHere> MyFile.iso <MyUserName>@<MyServer>:/Path/To/Where/I/Want/MyFile/ToGo/MyFile.iso

This also took about half  the time to copy that SMB took, and SMB didn’t even complete. Not to mention the transfer is secure (SSH)

Some additional resources

Also don’t forget to check the man page out 😉

man scp

OpenSSH from Linux to Windows 7 via tunneled RDP

December 27, 2011

I recently acquired a new second hand Asus laptop from my work,
that will be performing a handful of responsibilities on one of my networks.

This is the process I took to set up OpenSSH on Cygwin running on the Windows 7 box.

I won’t be going over the steps to tunnel RDP as I’ve already done this in another post

Make sure your LAN Manager Authentication Level is set as high as practical.
Keeping in mind, that some networked printers using SMB may struggle with these permissions set to high.

  1. Windows Firewall -> Allowed Programs -> checked Remote Desktop.
  2. System Properties -> Remote tab -> turn radio button on to at least “Allow connections from computers running any version of Remote Desktop”
    If you like, this can be turned off once SSH is set-up, or you can just turn the firewall rule off that lets RDP in.

CopSSH which I used on my last set of Linux to Windows RDP via SSH set-ups is no longer free.
So I’m not paying for something I can get for free, but with a little extra work involved.

So I looked at some other Windows SSH offerings

  1. freeSSHd which looked like a simple set-up, but it didn’t appear to be currently maintained.
  2. OpenSSH the current latest version of 5.9 released September 6, 2011
    A while back OpenSSH wasn’t being maintained. Looks like that’s changed.

OpenSSH is part of Cygwin, so you need to create a
c:\cygwin directory and download setup.exe into it.

    1. Right click on c:\cygwin\setup.exe and select “Run as Administrator”.
      Click Next.
    2. If Install from Internet is not checked, check it. Then click Next.
    3. Accept the default “Root Directory” of C:\cygwin. Accept the default for “Install For” as All Users.
    4. Accept the default “Local Package Directory” of C:\cygwin.
    5. Accept the default “Select Your Internet Connection” of “Direct Connection”. Click Next.
    6. Select the closest mirror to you. Click Next.
    7. You can expand the list by clicking the View button, or just expand the Net node.
    8. Find openssh and click the Skip text, so that the Bin check box for the item is on.
    9. Find tcp_wrappers and click the Skip text, so that the Bin check box for the item is on.

If you selected tcp_wrappers and get the “ssh-exchange-identification: Connection closed by remote host” error,
you’ll need to edit /etc/hosts.allow and add the following two lines before the PARANOID line.

ALL: : allow
 ALL: [::1]/128: allow

These lines were already in the /etc/hosts.allow

(optional) find the package “diffutils”, click on the word “skip” so that an x appears in Column B,
find the package “zlib”, click on the word “skip” (it should be already selected) so that an x appears in Column B.

Click Next to start the install.
Click Next again to… Resolving Dependencies, keep default “Select required packages…” checked.
At the end of the install, I got the “Program compatibility Assistant” stating… This program might not have installed correctly.
I clicked This program installed correctly.

Add an environment variable to your Systems Path variable.
Edit the Path and append ;c:\cygwin\bin

Right click the new Cygwin Terminal shortcut and Run as administrator.
Make sure the following files have the correct permissions.

/etc/passwd -rw-r–r–
/etc/group -rw-r–r–
/var drwxr-xr-x

Create a sshd.log file in /var/log/

touch /var/log/sshd.log
chmod 664 /var/log/sshd.log

Run ssh-host-config

  1. Cygwin will then ask Should privilege separation be used? Answer Yes
  2. Cygwin will then ask Should this script create a local user ‘sshd’ on this machine? Answer Yes
  3. Cygwin will then ask Do you want to install sshd as service? Answer Yes
  4. Cygwin will then ask for the value of CYGWIN for the daemon: []? Answer ntsec tty
  5. Cygwin will then ask Do you want to use a different name? Answer no
  6. Cygwin will then ask Please enter a password for new user cyg_server? Enter a password twice and remember it.

replicate your Windows user credentials with cygwin

mkpasswd -cl &gt; /etc/passwd
mkgroup --local &gt; /etc/group

I think (although I haven’t tried it yet) when you change your user password, which you should do regularly,
you should be able to run the above 2 commands again to update your password.
As I haven’t done this yet, I would take a backup of these files before I ran the commands.

to start the service, type the following:

net start sshd

Test SSH

ssh localhost

When you make changes to the /etc/sshd_config,
because it’s owned by cyg_server, you’ll need to make any changes as the owner.
I added the following line to the end of the file:

Ciphers blowfish-cbc,aes128-cbc,3des-cbc

As it sounds like Blowfish runs faster than the default AES-128

There are also a collection of changes to be made to the /etc/sshd_config

for example:

  • Change the LoginGraceTime to as small as possible number.
  • PermitRootLogin no
  • Set PasswordAuthentication to no once you get key pair auth set-up.
  • PermitEmptyPasswords no
  • You can also setup AllowUsers and DenyUsers.

The options available are here in the man page (link updated 2013-10-06).
This is also helpful, I used this for my CopSSH setup.

Open firewalls TCP port 22 and close the RDP port once SSH is working.

As my blog post says:
ssh-copy-id MyUserName@MyWindows7Box

I already had a key pair with pass phrase, so I used that.
Now we should be able to ssh without being prompted for a password, but instead using key pair auth.

Centerim, Irssi, Alpine on Screen

November 27, 2011

I’ve recently acquired access to my own shell from

This allows me to carry out development, testing, and any on-line activity anonymously.
All via SSH.

One of the tasks I needed to do,
was to set up my date/time to my local time zone.
Rather than set the system wide time,
because there are many users on this machine,
I needed to set the time zone on a per user basis.

The behaviour of your interactive shell is defined by your ~/.bashrc and ~/.bash_profile files.
Edit one of these files and append or alter the TZ as follows:

 vim /home/myuser/.bashrc

where myuser is just that, my user name.

Append the following:

export TZ="/usr/share/zoneinfo/yourcountry"

Where yourcountry is one of the country files in /usr/share/zoneinfo/


Screen is a Linux shell session manager.
It’s great, because you can leave multiple sessions running and switch between them,
all in a single console.
Then you can just detatch from screen, leaving your programmes running on it.
Terminate your SSH session, and re-connect from another machine,
re-attach to screen, and carry on working where you left off,
with your programmes all still running.

This is a quick run down on what it is and how to use it.

Create a new screen session:


List screens:

screen -ls


Ctrl-a, d

To re-attach to a screen:

screen -r


screen -raAd

Reattach (-r), do some sizing stuff (a,A), and detach (d) before reattaching if necessary.
If your screen session is attached elsewhere, using -raAd will detach that session, and reattach it here.

Cycle through each screen:

Ctrl-a n
Ctrl-a p

You can kill a screen by typing exit.

Terminate a screen:

screen -X -S ID kill

Where ID is the id of the screen you want to terminate.

Useful links
Full list of commands and their usage


CenterIM is a Linux command line instant messenger client.
Getting started
with CenterIM

Setting up GTalk in CenterIM:
Assuming you have centerim installed.
cd into your .centerim directory and edit the config file.

vim config

Add the following to the file:

jab_osinfo 1
jab_prio 4
jab_ssl 1

Enter the command mode by pressing the Esc key.


This will write and quit.
run centerim:


or better, run it in screen…

screen centerim

Press F4 for the general menu.
Select Accounts..

Under the Jab protocol, you will now see the connection details reflected.


Irssi is a Linux command line IRC client.
When I use Irssi,
these are the links I use most commonly.
IRC command reference
and full help for commands
For the beginner
The Full manual
Splitting Windows

I’ll probably end up adding more to this.


Alpine is a Linux command line mail client.
is an accurate guide on how to setup your GMail accounts using IMAP in alpine.
I used this for my first account setup.

When you need to setup multiple accounts,
you have to do a little bit more configuration.
I followed this.

Then create a Role.

I run all my external shell apps on screen.
So I run the following command…

screen alpine

You should be presented with the Main Menu.

Press S (Setup), L (collectionLists)

Press A (Add Cltn)
Add a Nickname that makes sense to you to reference your account by,
and the Server, as you did in the initial account setup,
save as you did in the initial setup.
Your Setup Collection List should look similar to the following.

From the Main Menu, press S (Setup), C (Config).
Scroll down until you find “Enable Incoming Folders Collection” and turn the radio button on.

Press E (Exit), and Y (Yes) to the Commit changes prompt.
You should be back on the Main Menu now.
Now you need to add a role for each account you’ve just setup.
Press S (Setup), R (Rules).

Then choose R (Roles).
Press A (Add).
Setup each role like the following.

Press E (Exit Setup), and Y to the save prompt.

Again in the S (Setup), C (Config).
Some of the settings that need to be turned on are:

  • alternate-compose-menu is optional
  • confirm-role-even-for-default

I set the following fields, so they show up in new messages you are composing.

Create a new message

There are a few ways you can compose a new email message.
This depends on where you start the process from.
If you’re in one of your mail folders,
you can press C (Compose).
You’ll be asked which role you would like to use to compose the message.
These are the role’s you set up before,
each one applies to one of your email accounts.
Once you choose one,
you’ll see a template with the fields you set up before.
Fill out the fields.
When your done composing your message,
press Ctrl-X to send.

Move a message from folder to another folder

  1. Select the message you want to move.
  2. Press the S (Save) key.
  3. If you have multiple email accounts, press Ctrl+N (Next Collection) or Ctrl+P (Prev Collection) to cycle through your accounts.
  4. Press Ctrl+T (To Folders).
    You will be presented with the collection of your email folders for your account.
  5. Select Which folder you want to put your message into.
  6. Press enter, unless you have to move the message down another level.
  7. If this is the case, press ‘/’ (the slash key).
  8. Then either the Tab key twice, or Ctrl+X (List matches).
    This will show you the next layer of folders to choose from.
    Either select the folder you want to move your message to and press Enter,
    or to go to another level, repeat steps 5 to 8.
  9. Once you’ve located the target folder (and selected it) to save (move) your message to,
    you’ll be provided with the path that you are about to save to.
  10. Press Enter. The message [Saving DONE] will be displayed.
    You message is now moved.
    When you return to the source folder,
    you will be asked if you want the message that is there deleted,
    so that you have moved, not copied the message.
    You have the option to either copy or move.

Multi selecting (Selecting multiple emails)

  1. Select the email and press the ‘;’ (semicolon) key.
  2. You will be prompted chose a selection criteria.
    I selected C (just select current message).
    When you do this, zoom will come into effect.
    So you will only see the currently selected messages.
  3. To un-zoom, so you can see all messages from the folder you were in, just press Z
    You will now see an ‘X’ next to the messages you have multi selected.
  4. Press the Z key again to zoom to the selected messages.
  5. Press A (Apply), then select the command you want to apply and that’s it.
  1. Select the link.
  2. Press Enter.
  3. Right click the link and select “Open link”.

Enable Spell Check in Alpine

First check that it’s not enabled

When composing a message, press  Ctrl+T
If you don’t get spell check, you’ll need to do the following.

Make sure you have aspell installed

On a debian based system, you can run

dpkg-query -l '*aspell*'

This will show you the aspell components installed

Or more precisely, just search for aspell

dpkg -l aspell

Once you find it, you can run

dpkg-query -W -f='${Status} ${Version}\n' aspell

This will tell you whether or not it’s installed.
If it’s not, you’ll need to install it:

sudo apt-get install aspell

From the Main menu in Alpine, S (Setup), C (Config).
Look for “spell”.
You can press ‘W’ to search and type in “spell” without the quotes.
Press Enter.
The first option you will find should be “Spell Check Before Sending”.
You can turn this on if you like.
Press ‘W’ again, accept the default, press Enter.
You should now see the option “Speller”.
Press Enter, and type in

aspell -c

Press Enter to accept.
Press ‘E’ to exit config.
Press ‘Y’ to the Commit changes prompt.

If you run the following at the command prompt


You should get a little information about what the -c switch does.

Shell script to provide monthly backup of your blog

July 28, 2011

This is just a quick bash script I put together last night that performs a wget on your blog.
It should be self documenting, so should be pretty easy to follow.
You’ll just have to change three variables at the top of the file to suit your environment.

#! /bin/bash

# Update fetch_domain to the domain you wish to fetch the web content from.
readonly fetch_domain=''
# Update fetch_domain_files to the sub domain where your files are located.
readonly fetch_domain_files=''
# Update wgetmonthly_dir to the local directory location you wish to save web content to.
readonly wgetmonthly_dir='/media/EXTERNAL/Documents/PenDrive/BinaryMist-Blog/WGetMonthly'
readonly help_message="Enter year and month that you want to wget from $fetch_domain in the following format: yyyy-mm

wget will fetch to the directory: ${wgetmonthly_dir}/yyyy-mm"

function print_blankline_then_message {
   echo "$1"

function usage {
   if [ "$1" != "" ]; then
      print_blankline_then_message "$1"
   print_blankline_then_message "$help_message"
   exit 1 #exit shell script

function check_if_help_required {
   if [ "$1" -eq 0 ]; then
      usage "No arguments detected."
   if [ "$2" == "help" ]; then

function validate_year_month {
   if [ "${#year_month}" -ne "$year_month_expected_length" ]; then
      usage "The length of the yyyy-mm should be $year_month_expected_length"

   readonly seperator='-'
   readonly seperator_index=4

   if test ${year_month:seperator_index:1} != $seperator; then
      usage "The seperating character '$seperator' should be the fifth character in the argument."

   readonly year_index=0
   readonly month_index=5


   # check that the year and month are positive numbers
   if [ $year -eq $year -a $year -gt -1 > /dev/null ]; then
      echo > /dev/null
   usage "The year \"${year}\" is not a positive number."
   if [ $month -eq $month -a $month -gt -1 > /dev/null ]; then
      echo > /dev/null
      usage "The month \"${month}\" is not a positive number."

function prepare_download_dir {
   cd "$wgetmonthly_dir"

   if [ -d "$year_month" ]; then
      print_blankline_then_message "Directory \"$wgetmonthly_dir/${year_month}\" already exists."
   if [ -d ${year_month}-old ]; then
   print_blankline_then_message "Directory \"$wgetmonthly_dir/${year_month}-old\" already exists. Will now delete."
   rm -rf ${year_month}-old
   print_blankline_then_message "Moving directory \"$wgetmonthly_dir/${year_month}\" to \"$wgetmonthly_dir/${year_month}-old\""
   mv $year_month ${year_month}-old
   print_blankline_then_message "Creating directory: \"$wgetmonthly_dir/$year_month\""
   mkdir "$year_month"

check_if_help_required $# $1
readonly year_month=$1
readonly year_month_expected_length=7
print_blankline_then_message "Moving into directory: $wgetmonthly_dir/$year_month"
cd "$year_month"
print_blankline_then_message "Starting wget at `date`"
print_blankline_then_message "Retreiving web content from http://$fetch_domain/$year/$month"
wget --random-wait --limit-rate=20k -kmnp -N -E -H -D $fetch_domain,$fetch_domain_files $fetch_domain/$year/$month/

Preparing APC Smart-UPS 1500 clients

July 26, 2011

Part two of a three part series

on Setting up a UPS solution, to enable clean shutdown of vital network components.

This post is about setting up the software that will be responsible for cleanly shutting down servers and workstations.

We have to decide which machine/s is/are going to be used to launch our script (which in turn is run by what APC call a command file).

Currently I’ve got an old laptop I pulled out of the rubbish about 5 years ago, with Windows XP running on it.
It’s got just enough battery capacity to stay alive for long enough to receive the event from the NMC (Network Management Card) and run my .dll that issues the shutdown sequence.
A couple of EeePC 901’s have also recently been made redundant, and I may use one of those with Windows 7 installed at some stage.
Currently all of my workstations and servers that don’t have batteries, I.E. notebooks are VM’s running on ESXi.
Oh… or servers that have their entire file system loaded into volatile memory, so if they are powered off, I.E. cold shutdown, there is no possible corruption of the file system.
What you can also do is host the PCNS (PowerChute Network Shutdown) on a VM, because once the shutdown of ESXi has been initiated, there is no stopping the sequence, and the VM’s will all be cleanly shutdown.
Or better still, use more than one machine to host PCNS, as they will operate on a first in first served basis.
As you’ll see here the NMC’s firmware and PCNS are quite extensible.
The above document is recommended reading if your planning on setting up an APC UPS and want to automate clean shutdowns.
Without reading, the comms can get a little confusing.

Setting up PCNS

Install the PowerChute Network Shutdown service

You can get a copy of v2.2.3 here
I later found out that there were later versions:
v2.2.4 linked to from here, which has additional documentation.
v3.0.0 linked to from here, which has additional documentation.
Both of which were linked to from here, which has additional manuals etc.
You can find the installation guide here.
The PCNS service needs to be run as a local Administrator as the default Local System account doesn’t have sufficient rights.
In saying all that, William Tournas from APC recommended I use PCNS 2.2.1 for Windows XP.
Additional 2.2.1 resources are found here.

If using vMA with PCNS 3.0, you go through a Web UI configuration wizard once installed.
If using PCNS with Windows, the configuration is part of the install.

Either way, the steps will look similar to the following:

netstat -a

Should show that PCNS is listening on TCP and UDP ports 3052

If it’s not, you’ll need to open those ports on your firewall.

If you’re looking at using a Linux based VM to host PCNS,
VMware provide vMA (vSphere Management Assistant) a CentOS VM image.
You can get the binary here.
You’ll also need PCNS.
Take your pick of the following binaries:
Along with the documentation:
You’ll have to have the same ports open, as PCNS will be listening on them.
A listing of iptables for the filter (default unless otherwise specified) table should look like the following:

For an easier to read output, try the following:

sudo iptables  -L -v -n --line-numbers | column -t

Once again, if these ports are not open, you’ll have to find which script is being used to set up the rules.
I’m not sure about CentOS, but in a Debian based system, you would normally put the firewall init script in /etc/init.d/
This script would call a script that sets up the rules and one that tears them down.
I’m going to be making a post about how I set up my firewall (iptables) rules for the netfilter module on our notebooks at some stage soon.
If I haven’t already done this and you need more help, just sing out.

I found the following links quite helpful with the setup:


ESXi.pdf linked to from here linked to from here.

This has a list of the ports that are supposed to be open on pcns 2.2.3 with ESX 3.5
I think this also applies to PCNS 3.0 and ESXi 4.1 which I tried out.

Also be aware that there’s a known issue with special characters in the credentials for PCNS 3.0

I read somewhere that the PCNS needs to have the same credentials as the NMC, so just be aware of this.

Could be useful for trouble shooting vMA (vSphere Management Assistant)
I made a couple of posts there.

PowerChute Network Shutdown v3.0 – Release Notes
goes through a whole lot of issues and work-arounds with PowerChute.
For example, discuss’s the correct way to run the command file, PowerOff.bat in our case.

The APC PCNS receives an event from the AP9606 (that’s the NMC (Network Management Card)) fitted to the UPS.
The script is launched by APC PCNS from a Windows or Linux box.
I read that PCNS will always shutdown the windows machine it’s running on.
This is not true.

My attempt at using a PowerShell script utilizing mostly VMware’s cmdlets to shutdown ESXi

PowerChute has an option to ‘run this command’ but it’s limited to 8.3 paths and won’t accept command line parameters.
A separate batch file is needed (I called it poweroff.bat)
that runs the shutdown script with the parameters – but that could shut down other ESXi boxes as well if required.

I was keen to use PowerShell to perform the shutdowns, as I’d read it was quite capable and also VMware supplied a large set of management cmdlets.

Install PowerCLI from here.
read the installation guide.
As an admin, run the following:

set-executionpolicy remotesigned

Details of executionpolicy here.

If running PowerOffUPSGuests.ps1 from command shell rather than from a batch file.
You need to add the PowerCLI snapin.

PS C:\scripts&gt; Add-PSSnapin VMware.VimAutomation.Core
PS C:\scripts&gt; . .\PowerOffUPSGuests.ps1 MyESXiHostName AdminUserName

This will establish the SSL connection to MyESXiHostName

Following are the PowerShell scripts I used.

First we had to create our password file to use to log in to vSphere.
See this post for how this was done.

PowerOffUPSGuests.bat (the command file)

echo off
REM VMware would have used the Export-Console cmdlet to export the name of the PowerShell snap-in PowerCLI uses.
REM to the PowerShell console file (.psc1)

REM Invoke the command with the call operator (The ampersand).
PowerShell.exe -PSConsoleFile "C:\Program Files\VMware\Infrastructure\vSphere PowerCLI\vim.psc1" "& "C:\Scripts\PowerOffUPSGuests.ps1" MyESXiServer.MyDomain MyUser

PowerOffUPSGuests.ps1 (the script that was going to do the work)

param ( [parameter(Mandatory=$true)][string] $vSphereServername,
   [parameter(Mandatory=$true)][string] $user

$HostCredential = C:\Scripts\Get-myCredential.ps1 $user C:\Scripts\mp.txt

Set-StrictMode -Version 2.0
Write-Host "Establishing connection to $vSphereServername" -ForegroundColor Yellow
Connect-VIServer -Server $vSphereServername -Protocol https -Credential $HostCredential

function Stop-VMOnVMHost {
   Write-Host "Shutting down guests." -ForegroundColor Yellow

   $vM = Get-VM | Where-Object {$_.PowerState -eq "PoweredOn" -and $_.Guest.State -eq "Running"}
   Write-Host "Shutting down the following guests: $vM " -ForegroundColor Yellow
   $vM | Shutdown-VMGuest -Confirm:$False
   $seconds = 300
   Write-Host "Waiting $seconds Seconds. "
   Start-Sleep -Seconds $seconds

   $vM = Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"}
   Write-Host "Stopping the following guests: $vM " -ForegroundColor Yellow
   $vM | Stop-VM -RunAsync -Confirm:$False
   $seconds = 60
   Write-Host "Waiting %seconds Seconds. "
   Start-Sleep -Seconds $seconds

function Stop-VMHost {
   Write-Host "Setting state of $vSphereServername to maintenance mode. " -ForegroundColor Yellow
   Get-VMHost | ForEach-Object {
      $hostName = $_.Name
      Write-Host "Putting $hostName into maintenance mode. "
      Set-VMHost -vmhost $_ -state maintenance
      Write-Host "Stopping $hostName. "
      Stop-VMHost -vmhost $_ -RunAsync

Write-Host "Shutdown Complete" -ForegroundColor Yellow

Tried my script and got the following:

Shutdown-VMGuest     Operation “Shutdown VM guest.” failed for VM “MyGuestNameHere” for the following reason: fault.Restriction.summary

I had a hunch that it was due to the read only restriction I had heard about.
So tried command straight from PowerShell console…
same result.
More details here.
PowerCLI references to shutting down ESX

So as it turned out, VMware has removed write access from PowerCLI to ESXi, in 4.0 onwards I think.

Back to Scripting SOAP

As I was kind of out of luck with using PowerCLI cmdlets,
I decided to write my own library,
that I would execute using PowerShell.

First command needs to shutdown my fileserver.
hello, authenticate, shutdown

Used Burp suite to diagnose the http frames being sent received from/to vSphere client/ESXi.
I haven’t used this tool before, but it gave very good visibility of the messages being sent/received.
The vSphere client has a config file here:
C:\Program Files\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient.exe.config
that you can change the ports that the vSphere client sends/receives on,
but I found it easier to just set the IP address / Name field in the GUI
to point to This is where the Burp proxy listens on by default.

In Burp, you will also need to add another proxy listener to the proxy->options tab.
Set Local Listener to 8080,
Uncheck listen on loopback interface only,
Check support invisible proxying for non-proxy-aware clients.
The in-app help has good documentation on this.
Set redirect to host to the ESXi host name.
Set redirect to port to the ESXi’s default SSL port of 443
Select the generate CA-signed per-host certificates radio button.

I also made sure the new proxy rule was the only one running.
When Burp captures each frame, you can forward each one onto any one of the other tools in the suite.
This is a really nice tool.

My PowerOffUPSGuests.ps1 was about to significantly change too.
Running my PowerOffUPSGuests.ps1 script using PowerShell

PS C:\Scripts\UPS&gt; . ".\PowerOffUPSGuests.ps1"

We no longer need to pass any arguments to PowerOffUPSGuests.ps1

I was going to be using .net 4 libraries in my PowerOffUPSGuests.dll,
so needed to Let PowerShell know about the .net 4 CLR.
By default PS 2.0 is only aware of the .net 2.0 framework.

Some insight on this:

So needed to create a couple of config files for
with config appended
with the following contents:

<?xml version="1.0"?>
  <startup useLegacyV2RuntimeActivationPolicy="true">
     <supportedRuntime version="v4.0.30319"/>
     <supportedRuntime version="v2.0.50727"/>

This works for PowerShell, but not for PowerGUI (obviously) which I was using for debugging.

So If you still need PowerGUI you’ll have to add the registry hacks explained in the links above.
Remember to remove them once finished as they take affect system wide.

I also had some trouble with later versions of C# than 2.0 when compiling on the fly in PowerShell.
Although I was specifying the language.

Add-Type -Path $typePath -CompilerParameters $compilerParameters -Language csharpversion3

Found a workaround this bug here.

# add the block of code we call into
$code = [io.file]::ReadAllText((Join-Path -Path $scriptPath -ChildPath $powerOffUPSGuestsFile))
Add-Type $code -CompilerParameters $compilerParameters -Language CSharpVersion3

We’ll go over the library code in the third part of this series.

As it stands now, the C:\Scripts\UPS\PowerOff.bat looks like this

echo PowerOffUPSGuests.ps1 started at the following time: %time% &gt;&gt; C:\Scripts\UPS\Log.txt
"C:\WINDOWS\system32\WindowsPowerShell\v1.0\PowerShell.exe" C:\Scripts\UPS\PowerOffUPSGuests.ps1
echo PowerOffUPSGuests.ps1 finished at the following time: %time% &gt;&gt; C:\Scripts\UPS\Log.txt

The PowerOffUPSGuests.ps1 looks like this

Set-StrictMode -Version 2.0

# add the assembly that does the work.
Add-Type -Path C:\Scripts\UPS\PowerOffUPSGuests.dll

# instantiate a PowerOffUPSGuests instance
$powerOffUPSGuestsInstance = New-Object -TypeName BinaryMist.Networking.Infrastructure.PowerOffUPSGuests

Write-Host $powerOffUPSGuestsInstance.InitShutdownOfServers() -ForegroundColor Green

The essential files

Testing that everything works

I was unsure whether we were going to be able to get ESXi to cleanly shutdown it’s guest VM’s.
As I’d had some trouble with this previously.

I was thinking about writing a WCF service and client to shutdown windows guests for now.
The service being on the machine that needed to be cleanly shutdown.
Could use something like the following command line in the service.

shutdown.exe -m //MachineNeedingShutdown -t 10 -c "Shutting down due to UPS running on battery." -s -f

Wrapped in something like this…

Process shutdownMyBox = new Process();
shutdownMyBox.StartInfo.FileName = shutdown.exe;
shutdownMyBox.StartInfo.Arguments = "-m //MachineNeedingShutdown -t 10 -c \"Shutting down due to UPS running on battery.\" -s -f";

I was sure there was a better way though.

The sequence of events I was thinking of was something like the following:

First we try to shutdown every VM guest, set vMGuestTimmer
If all VM guests shutdown
——try put host into maintenance mode, set timer.
——when in maintenance mode
———shutdown host
——if enter maintenance mode not successful within time set
———shutdown host
On vMGuestTimmer
——shutdown host

There was a better/easier way though

In the PCNS Web UI -> PowerChute->MachineName->Configure Events
You can set PowerOff.bat to run after 30 seconds, or for testing,
set it to something really small, so it runs the command file sooner.
Set the time that’s required for the command file to complete to 5 minutes.
Although I don’t think it matters that much, as long as there’s enough time to start the execution of the PowerShell script.
Once the script is running, we don’t care how long PCNS thinks it should wait, as it’s non blocking.

To test that pcns will run your batch file:
Just put some temporary script, something like the following in your

time/T &gt;&gt; C:\Scripts\UPS\MyTest.txt

These are some links from APC to help get your PCNS command file running:

What is needed for ESXi to shut down all machines cleanly?

Graceful shutdown work around for ESXi guests.

Also it’s important to make sure the root user of ESXi has the Administrator Role.

What is needed for Windows VM’s to shutdown cleanly?

First ascertain whether or not your VM is/isn’t being shutdown cleanly.
eventvwr is your friend.

The scripts that may play a part in the shutting down of the Windows VM’s.
If you have a look at the VMware Tools Properties->Scripts tab
You can see for the shutdown script, that it actually does nothing.
If you find that your Windows box is not shutting down cleanly…
Add a custom script to the “Shut Down Guest Operating System” Script Event
I just created a shutdown.bat with the following in it.

C:\Windows\System32\shutdown.exe -s -t 1

This cleared up any errors I was getting in my Windows7 logs.

What is needed for Linux VM’s to shutdown cleanly?

If you’re looking at Debian based systems…
View the relevant log that contains shutdown info.

sudo vi /var/log/messages


sudo vi /var/log/syslog

From command mode (that’s [Esc]) to show line numbers,


:set number


:set nu

To find the matches for “shutdown” (without quotes) ignoring case

sudo grep -i -n "shutdown" /var/log/messages

Or easier still…
Once the file’s open in vi,
From command mode


[n]            will repeat the search forward
[N]            will repeat the search in opposite direction

My Debian wheezy server wasn’t getting shutdown cleanly.
So tried to install vmware tools, but found the easier way was to use open-vm-tools
Added contrib to my /etc/apt/sources.list
Installed open-vm-tools open-vm-source
Had some trouble with the NZ repo for those packages, they were corrupt.
So renamed /etc/apt/apt.conf so apt-get wasn’t using my cached packages from apt-cacher.

sudo apt-get clean
sudo apt-get update
sudo apt-get install open-vm-tools open-vm-source

The scripts that may play a part in the shutting down of the Linux VM’s.
Read this link.
There are also vmware-tools scripts
Mine didn’t appear to do much, but my server was being shutdown cleanly now.

Shout out if anythings unclear.

In part three I’ll be going over the library I’ve written that actually does the work 😉

rsync over SSH from Linux workstation to FreeNAS

March 6, 2011

I’ve been intending for quite some time to setup an automated or at least a thoughtless
one click backup procedure from my family members PC’s to a file server.
Now if you put files directories in the place where we are going to rsync to, and run the command we’re going to setup, those new files directories will be deleted.
So in this case, we have a master / slave model.
You can also set it up so that no files directories are automatically deleted. That’s not what I’m doing here though.

Links I found helpful

rsync man page
SSH man page
Ken Fallon’s “A private data cloud” podcast

I wanted to setup the script to mirror the local disk or several directories on it to the file server.
So the local disk would be the master.
I often use the file server as an intermediate step to pass files around my network.
So I just need to be aware not to put files in the directories that are going to get written to on the file server, but use alternative ones instead.
Otherwise they will be overwritten when rsync runs.


Provide a regular (on the hour) or one click sync of files (once the fileserver is on a decent UPS) from:

  1. My external drive to the file server.
  2. My wife’s thumb drive to the file server.
  1. /media/EXTERNAL/Applications to MyFileServer/MyShare/ExternalBackup/Applications
    /media/EXTERNAL/Documents to MyFileServer/MyShare/ExternalBackup/Documents
    /media/EXTERNAL/media/Books to MyFileServer/media/Books
    /media/EXTERNAL/media/EducationalMedia to MyFileServer/media/EducationalMedia
    /media/EXTERNAL/media/Images to MyFileServer/media/Images
  2. /media/disk to MyFileServer/WifesShare/disk

——via SSH

Until the file server is being powered by a UPS I can set up shutdown scripts for,
so when we’re not about, it will still shutdown gracefully on power outage,
we’ll be running the rsync scripts manually.
As I don’t want an hourly script syncing data to the file server when the power gets cut.
Why? because RAID arrays often get destroyed by being written to when they loose power.
Currently if we loose power the file server is on a small UPS and we can halt any sync scripts interacting with the file server before she goes down.
We can manually shutdown the file server gracefully.

You need to take good precaution with rsync as you can erase data easily.
I like to use the –dry-run or -n until I’m happy that the command I’ve got is going to actually do what I think it is.
You can use -v the verbose option with levels of verbosity up to -vvv for debugging rsync. Generally -vv is heaps.
Archive mode -a is actually -rlptgoD. Check the man page for details.
–delete delete extraneous files from dest dirs that are not on the source.
–force will delete directories from dest even if not empty
It’s a good idea to setup some test directories for source and dest.
You can also (if you want to be extra careful) mount your dest and source or just your dest directory read only.
Put a copy of some files and directories in each, and make some changes to source and/or dest.
Then once you run the command, you can check that the sync has done what you expected.

My initial test command after I created the rsyncTestSource and rsyncTestDest dirs:

rsync -vva --dry-run --delete --force /media/EXTERNAL/rsyncTestSource/ /media/EXTERNAL/rsyncTestDest/

Perform checks.

Then remove the –dry-run.

Perform checks again.

Now to file server:

You’ll have to, if you havn’t already, setup SSH on your file server.
You can follow the steps on my post here for that if you like…

The initial command I used:

rsync -vva --dry-run --delete --force -e 'ssh -p 2222' /media/EXTERNAL/rsyncTestSource/ myUser@myFileServer:/mnt/FileServer/myUserDir/rsyncTestDest/

You can specify the -e option followed by the remote shell.
rsync must be installed on both source and dest machines.
By default FreeNAS already has rsync, as does a standard debian install.

Then remove the –dry-run

Perform checks again.

Now for the first real backup, add the dry run to start with:

rsync -vva --delete --force -e 'ssh -p 2222' /media/EXTERNAL/Applications/ myUser@myFileServer:/mnt/FileServer/myUserDir/External-Backup/Applications/

Then remove the –dry-run.

Perform checks again.

I added a collection of these commands to a file (rsync_EXTERNAL_to_fileserver) to run for each directory and saved to my ~ directory.

Turn the executable bit on.
Make sure owner and group is correct.

chmod 750 rsync_EXTERNAL_to_fileserver
chown MyUserName:MyGroupName rsync_EXTERNAL_to_fileserver

Add a command drawer to the task bar.
Add a Custom Application Launcher to the drawer that points to the rsync_EXTERNAL_to_fileserver file.
You can even add an image that makes sense to the drawer.
Mine looks like this, with 1 command launcher.







Ok, it’s 2 clicks for me, but you don’t have to use a drawer 🙂

There are also other ways to do this.
Like this video.

Distributed Version Control the solution?

October 3, 2010

Due to the fact that I am starting to need a Version Control System at home for my own work and the company I currently work for during the day could potentially benefit from a real Version Control System.

I’ve set out to do an R&D spike on what is available and would best suite the above mentioned needs.
I’ve looked at a large range of products available.

At this stage, due to my research and in talking to some highly regarded technical friends and other people about their experiences with different systems, I’ve narrowed them down to the following.

Subversion, Git and Mercurial (or hg)
Subversion is server based.
Git and hg are distributed (Distributed Version Control System (DVCS)).

The two types of VCS and some of their attributes.

Centralised (or traditional)

  • Is better than no version control.
  • Serves as a single backup.
  • Server maintenance can be time consuming and costly.
  • You should be able to be confident that the server has your latest changeset.


  • Maintenance needs are significantly reduced, due to a number of reasons. One of which is… No central server is required.
  • Each peer’s working copy of the codebase is a complete clone.
  • There is no need to be connected to a central network. Which means users can work productively, even when network connectivity is unavailable.
  • Uses a peer-to-peer approach rather than a client-server approach that the likes of Subversion use.
  • Removes the need to rely on a single machine as a single point of failure.
    Although it is often a good idea to have a server that is always online and ready to accept changesets.
    As you don’t always know whether another peer has accepted all your changes or is online.
  • Most operations are much faster than the centralised model, as no network is involved.
  • Each copy of the repository effectively acts as a remote backup. Which has multiple benefits.
  • There is no canonical code base, only working copies.
  • Operations such as commits, viewing history and rolling back are fast, because there is no need to communicate with a central server.
  • A web of trust is used to merge code from disparate repositories.
  • Branching and Merging made easier.
  • No forced structure: a central server can be implemented or peers can control the codebase.
  • Although I don’t see huge benefits for a central server in my target scenario.
  • Buddy builds. A team member can pass a change set to another member to try before committing to a central location.
    This would stop broken CI builds.
  • There is a huge amount of flexibility with your layout.
  • With a well planned layout a Distributed Version Control System can do anything a Centralised system can do, with the additional benefit of easy merges.

In weighing up the pros and cons of distributed versus the centralised model.

I think for my target requirements,
a distributed system has more to offer in the way of time savings and hardware savings.
This page has a good explanation of the differences between Centralised and Distributed.
Here is a detailed list of comparisons of some of the more common systems.

Mercurial is ticking quite a few boxes for me.
Mercurial has a VisualStudio plug-in.
There is a GUI available for windows platforms and others that integrates Mercurial directly into your explorer.
It’s free, open, and being actively maintained.
Projects using Mercurial.

Mercurial is written in Python, which is another plus for me.
Binaries are freely available for Windows, GNU/Linux, Mac OS X, OpenSolaris.
The source is also available, so you can build it for most platforms.

Plenty of documentation here, plus the book.

Installation and Configuration. Covering Windows, Debian and more.
TortoiseHg has binaries for windows and debian, but only for Squeeze onwards by the look of it.
If your running Lenny, you can just use hg. apt-get install mercurial.
When I downloaded and installed the 64 bit version of TortoiseHg (v1.1.3 hg v1.6.3), it came with 4 comprehensive documents.

  1. Mercurial: The Definitive Guide 2010-02-21 as pdf
  2. TortoiseHg v1.1.3 Documentation in both pdf and chm
  3. Mercurial Command Reference

Very nice!
Turn off the indexing service on the working copies and repositories, and exclude them from virus scans
Can also get TortoiseHg here (For Debian, TortoiseHq isn’t available for Lenny).
Click the Tutorial link for the Quick start guide to TortoiseHg.

Once installed, start working through the following links.

Comments or thoughts?

Installation of SSH on 64bit Windows 7 to tunnel RDP

August 26, 2010

This post covers two scenarios.

Scenario one

With this setup I have a Windows 7 VM (the server) on the same network segment as the client PC which will be taking over any work I would normally do on my Windows XP box.
My existing XP box is used for any development that is easier to do on a Windows machine than a *nix machine.
Mostly .Net development.

Scenario two

Includes tunneling to a NATed Windows 7 machine on a different network

Access to my existing Windows XP box:
Is by way of RDP session tunneled through SSH.
SSH link being established from one of my Debian eeepc’s (The computer I use most of the time) to the existing Windows XP machine.

Used OpenSSH for the existing Windows XP machine. which is no longer supported.
Couldn’t get key pair authentication working though when I set it up.

I thought I’d give OpenSSH a try on the Windows 7 machine and see how far we could get.
Once followed all directions in the ssh readme.txt and comparing with the setup on my existing Windows XP box.
The OpenSSH Server service wouldn’t start.
Followed directions here.
Tried everything I could think of and still couldn’t get the service to start.

So going on some others advise, decided to give copSSH a try, as it is an implementation of OpenSSH, but currently being maintained.
Thanks to Tevfik Karagülle.
This worked out well and was a very easy setup.
The version of CopSSH used for this was 4.1.0 from here.

Initial sites used for copSSH install

Installation of copSSH

When you add a user to the CopSSH Control Panel, make sure you run the CopSSH Control Panel as an administrator (probably best to runas administrator for any actions),
else the user appears to be added, but when you try to SSH to the server, you get something along the lines of…
Unable to authenticate
Failed password for invalid user

Setup for the tunnel

Create a file in your ~ dir. TunnelToWin7Box for example, and put the following command in it.

ssh -v -f -L 3391:localhost:3389 -N MyUserName@MyWindows7Box

Turn the executable bit on.
Make sure owner and group is correct.

chmod 750 TunnelToWin7Box
chown MyUserName:MyGroupName TunnelToWin7Box

Add a command drawer to the task bar.
Add a Custom Application Launcher to the drawer that points to the TunnelToWin7Box file.
You can even add an image that makes sense to the drawer.
Mine looks like this, with 3 command launchers…

The first port there can be any port not currently in use.
The second port is the port that RDP listens on in Windows.
You also need to add an inbound rule to open port 22 or a port of your choosing on the Windows Firewall.
Also close the Remote Desktop port TCP 3389 on the Windows box.
If the server you are trying to tunnel to is behind a NAT and not on your network, I.E. you are trying to tunnel to your work machine from home for example, There is a little more involved in setting up the firewall rule and a change to the sshd_config.
You’ll need to add an inbound rule. I called it SSH. In the Programs and Services tab… selected “All programs that meet the specified conditions”.
For the Service Settings, only one that would work was “Apply to services only”. I thought it would be best to select only the ssh service, but this wouldn’t allow SSH in.
General tab just had Enabled on. Computers tab was untouched. Users and Scope was untouched. Advanced tab only needed to select Private check box.
“Protocols and ports” tab Protocol type is TCP, Local port is port 22, Remote port is All Ports.
Edit the C:\Program Files (x86)\ICW\etc\sshd_config as an administrator.
Add the line… GatewayPorts yes
Or uncomment it and set to yes rather than no if it already exists.

Command I used for the NATed scenario

ssh -v -f -L 3392:localhost:3389 -N -p 2222

The port is the port that your network admin has setup for you to forward to the machine you want to tunnel to.

When I run the command to try establish the tunnel I was getting an error message.
I made a post here.
So I un-installed copSSH and re-installed a few times trying different things.
Before last un-install, I removed the users that copSSH adds, because it doesn’t remove them on un-install,
and deleted the OpenSSHServer service using the “sc delete OpenSSHServer” command in cmd.exe shell running as administrator.
Installed again using all defaults.
It appears as even though SSH gives the message that it won’t tunnel, if you then try and open the port forwarded RDP session, it works.
In saying that, sometimes it didn’t work.
This happens if you click the command launcher more than once and you end up with more than one tunnel established.
In which case you just kill one of them and your away laughing.

Setup your Remote Desktop Session now

I’ve been using Gnome-RDP for my RDP sessions.
Set the session up to look like this.

Once done, click Connect, and you should have your RDP session from your Linux box to your Windows 7 box secured courtesy of SSH

Setup Key pair authentication

On Debian epc, or any other Debian machine for that matter

Copy the existing public key I used for SSHing to other servers to MyWindows7Box.
This is considerably more difficult if you want to scp the key to a NATed machine on another network.
Read the likes of this if your interested.
It’s the public key, so sniffing it is not such a big deal.

scp ~/.ssh/ MyUserName@MyWindows7Box:

Make sure you have the Colan at the end of the above command, else the file won’t be copied.
You may receive a prompt that the authenticity of the server you are trying to scp to can’t be established and you want to continue.
The server you are trying to connect to is added to the list of known hosts on the local machine.
Thats /home/MyUserName/.ssh/known_hosts
I didn’t get that with scp’ing to MyWindows7Box because my known_hosts already knew about MyWindows7Box from my previous OpenSSH install.

On MyWindows7Box

In the dir C:\Program Files (x86)\ICW\home\MyUserName\.ssh\
I copied the authorized_keys file to authorized_keys-OrigWithInstall (rename).
Wasn’t allowed to edit the authorized_keys file for some reason, so opened a Bash shell that comes with copSSH
and edited ~/.ssh/authorized_keys with nano. Deleting the public key.
When I tried to open this file in file explorer, it didn’t appear to have been edited.
This is because the file I thought I had edited (C:\Program Files (x86)\ICW\home\MyUserName\.ssh\authorized_keys)
was actually C:\Users\MyUserName\AppData\Local\VirtualStore\Program Files (x86)\ICW\home\MyUserName

From C:\Program Files (x86)\ICW\home\MyUserName\.ssh (or at least what I thought was there),
the public key needs to be put into the list of authorized clients that may connect to the ssh daemon.
Can do this using the Bash shell that comes with copSSH.

$ cat >> .ssh/authorized_keys

You can now delete the on the target machine.

Copied C:\Users\MyUserName\AppData\Local\VirtualStore\Program Files (x86)\ICW\home\MyUserName\authorized_keys
to C:\Program Files (x86)\ICW\home\MyUserName\.ssh\authorized_keys

With scenario two, there were a few differences.
I’m thinking some of which were probably due to a more recent version of CopSSH (4.1.0).
For starters there was no authorized_keys file anywhere, so I created one (in C:\Program Files (x86)\ICW\home\User\.ssh).
As stated above, it’s considerably more difficult to scp the from a remote pc to a NATed server.
Put in C:\Program Files (x86)\ICW\home\User\.ssh along with the authorized_keys I created, and from the bash shell
(accessible from the Copssh folder in the start menu) who’s root dir is C:\Program Files (x86)\ICW\
ran the cat command shown above.

This is probably a better way to copy the public key:

ssh-copy-id MyUserName@MyWindows7Box

Anapnea showed me this.

Could now connect via key pair auth

Made the usual changes to C:\Program Files (x86)\ICW\etc\sshd_config on MyWindows7Box

I.E. turn root access off, password auth off,
AllowUsers MyUserName
Although this is done by the CopSSH Control Panel in version 4.1.0
I think a service restart is required to reload changes.
When you make changes to the sshd_config, you’ll need to do them as an administrator (similar to how you would on a *nix system as root).
This site has example of setting up SSH to be even more secure by modifying the sshd_config.
It’s specific to copSSH.
There are many items on the net that show and describe the options when it comes to the sshd_config.
The available options are in the man page


grammer check for OpenOffice

March 27, 2010

Installing and setting up LanguageTool in OpenOffice on Debian Lenny.

First of all you need to have OpenOffice 3.1 installed.
If you want to install a later version of OpenOffice than the default (2.6.4) supplied with Debian Lenny,
you’ll have to fetch it from the Debian Lenny backports repo.

Edit your /etc/apt/sources.list and add the repo:
deb lenny-backports main contrib non-free

Install the debian backports keyring:

#apt-get install debian-backports-keyring
#apt-get update
#apt-get -t lenny-backports install

Make sure is installed.
Make sure sun-java5-jre is installed and selected in OpenOffice under Tools->Options->>Java.
Java versions other than Sun, may not work.

Install LanguageTool:
Download the LanguageTool extension from
In OpenOffice Tools menu Extension Manager, Browse to the language tool file you downloaded and select it.

That should be it. Can’t remember if I had to restart OpenOffice.