Archive for May, 2015

Evaluation of Host Intrusion Detection Systems (HIDS)

May 30, 2015

Followed up with a test deployment and drive.

The best time to install a HIDS is on a fresh install before you open the host up to the internet or even your LAN if it’s corporate. Of course if you don’t have that luxury, there are a bunch of tools that can help you determine if you’re already owned. Be sure to run one or more over your target system before your HIDS bench-marks it.

The reason I chose Stealth and OSSEC to take further into an evaluation was because they rose to the top of a preliminary evaluation I performed during a recent Debian Web server hardening process where I looked at several other HIDS as well.

OSSEC

ossec-hids

Source

ossec-hids on github

Who’s Behind Ossec

  • Many developers, contributors, managers, reviewers, translators (infact the OSSEC team looks almost as large as the Stealth user base)

Documentation

Lots of documentation. Not always the easiest to navigate. Lots of buzz on the inter-webs.
Several books.

Community / Communication

IRC channel #ossec at irc.freenode.org Although it’s not very active.

Components

  • Manager (sometimes called server): does most of the work monitoring the Agents. It stores the file integrity checking databases, the logs, events and system auditing entries, rules, decoders, major configuration options.
  • Agents: small collections of programs installed on the machines we are interested in monitoring. Agents collect information and forward it to the manager for analysis and correlation.

There are quite a few other ancillary components also.

Architecture

You can also go the agent-less route which may allow the Manager to perform file integrity checks using agent-less scripts. As with Stealth, you’ve still got the issue of needing to be root in order to read some of the files.

Agents can be installed on VMware ESX but from what I’ve read it’s quite a bit of work.

Features (What does it do)

  • File integrity checking
  • Rootkit detection
  • Real-time log file monitoring and analysis (you may already have something else doing this)
  • Intrusion Prevention System (IPS) features as well: blocking attacks in real-time
  • Alerts can go to a databases MySQL or PostgreSQL or other types of outputs
  • There is a PHP web UI that runs on Apache if you would rather look at pretty outputs vs log files.

What I like

To me, the ability to scan in real-time off-sets the fact that the agents need binaries installed. This hinders the attacker from covering their tracks.

Can be configured to scan systems in real–time based on inotify events.

Backed by a large company Trend Micro.

Options. Install options for starters. You’ve got the options of:

  • Agent-less installation as described above
  • Local installation: Used to secure and protect a single host
  • Agent installation: Used to secure and protect hosts while reporting back to a
    central OSSEC server
  • Server installation: Used to aggregate information

Can install a web UI on the manager, so you need Apache, PHP, MySQL.

What I don’t like

  • Unlike Stealth, The fact that something has to be installed on the agents
  • The packages are not in the standard repositories. The downloads, PGP keys and directions are here.
  • I think Ossec may be doing to much and if you don’t like the way it does one thing, you may be stuck with it. Personally I really like the idea of a tool doing one thing, doing it well and providing plenty of configuration options to change the way it does it’s one thing. This provides huge flexibility and minimises your dependency on a suite of tools and/or libraries
  • Information overload. There seems to be a lot to get your head around to get it set-up. There are a lot of install options documented (books, interwebs, official docs). It takes a bit to workout exactly the best procedure for your environment. Following are some of the resources I used:
    1. Some official documentation
    2. Perving in the repository
    3. User take one install
    4. User take two install
    5. User take three install

Stealth

Stealth file integrity checker

Source

sourceforge

Who’s Behind Stealth

Author: Frank B. Brokken. An admirable job for one person. Frank is not a fly-by-nighter though. Stealth was first presented to Congress in 2003. It’s still actively maintained and used by a few. It’s one of GNU/Linux’s dirty little secrets I think. It’s a great idea, makes a tricky job simple and does it in an elegant way.

Documentation

  • 3.00.00 (2014-08-29)
  • 2.11.03 (2013-06-18)
    • Once you install Stealth, all the documentation can be found by sudo updatedb && locate stealth. I most commonly used: HTML docs (/usr/share/doc/stealth-doc/manual/html/) and (/usr/share/doc/stealth-doc/manual/pdf/stealth.pdf) for easy searching across the HTML docs
    • man page (/usr/share/doc/stealth/stealthman.html)
    • Examples: (/usr/share/doc/stealth/examples/)
  • More covered in my demo set-up below

Binaries

  • 3.00.00 (2014-08-29) is in the repository for Debain Jessie
  • 2.11.03-2 (2013-06-18) This is as recent as you can go for Linux Mint Qiana (17) and Rebecca (17.1) within the repositories, unless you want to go out of band. There are quite a few dependencies ldd will show you: 
    libbobcat.so.3 => /usr/lib/libbobcat.so.3 (0xb765d000)
libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0xb7641000)
libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xb754e000)
libm.so.6 => /lib/i386-linux-gnu/i686/cmov/libm.so.6 (0xb7508000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xb74eb000)
libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xb7340000)
libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xb71ee000)
libcrypto.so.1.0.0 => /usr/lib/i386-linux-gnu/i686/cmov/libcrypto.so.1.0.0 (0xb7022000)
libreadline.so.6 => /lib/i386-linux-gnu/libreadline.so.6 (0xb6fdc000)
libmilter.so.1.0.1 => /usr/lib/i386-linux-gnu/libmilter.so.1.0.1 (0xb6fcb000)
/lib/ld-linux.so.2 (0xb7748000)
libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xb6fa5000)
libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0xb6fa0000)
libtinfo.so.5 => /lib/i386-linux-gnu/libtinfo.so.5 (0xb6f7c000)
libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xb6f78000)
libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xb6f72000)
  • 2.10.01 (2012-10-04) is in the repository for Debian Wheezy

Community / Communication

No community really. I see it as one of the dirty little secretes that I’m surprised many diligent sysadmins haven’t jumped on. The author is happy to answer emails. The author doesn’t market.

Components

Controller

The computer initiating the scan.

Needs two kinds of outgoing services:

  1. ssh to reach the clients
  2. mail transport agent (MTA)(sendmail, postfix)

Considerations for the controller:

  1. No public access.
  2. All inbound services should be denied.
  3. Access only via its console.
  4. Physically secure location (one would think goes without saying, but you may be surprised).
  5. Sensitive information of the clients are stored on the controller.
  6. Password-less access to the clients for anyone who gains controller root access, unless ssh-cron is used, which appears to have been around since 2014-05.

Client

The computer/s being scanned. I don’t see any reason why a Stealth solution couldn’t be set-up to look after many clients.

Architecture

The controller stores one to many policy files. Each of which is specific to a single client and contains use directives and commands. It’s recommended policy to take copies of the client programmes such as the hashing programme sha1sum, find and others that are used extensively during the integrity scans and copy them to the controller to take bench-mark hashes. Subsequent runs will do the same to compare with the initial hashes stored.

Features (What does it do)

File integrity tests leaving virtually no sediments on the tested client.

Stealth subscribes to the “dark cockpit” approach. I.E. no mail is sent when no changes were detected. If you have a MTA, Stealth can be configured to send emails on changes it finds.

What I like

  • It’s simplicity. There is one package to install on the controller. Nothing to install on the client machines. Client just needs to have the controllers SSH public key. You will need a Mail Transfer Agent on your controller if you don’t already have one. My test machine (Linux Mint) didn’t have one.
  • Rather than just modifying the likes of sha1sum on the clients that Stealth uses to perform it’s integrity checks, Stealth would somehow have to be fooled into thinking that the changed hash of the sha1sum it’s just copied to the controller is the same as the previously recorded hash that it did the same with. If the previously recorded hash is removed or does not match the current hash, then Stealth will fire an alert off.
  • It’s in the Debian repositories. Which is a little surprising considering I couldn’t find any test suite results.
  • The whole idea behind it. Systems being monitored give little appearance that they’re being monitored, other than I think the presence of a single SSH login when Stealth first starts in the auth.log. This could actually be months ago, as the connection remains active for the life of Stealth. The login could be from a user doing anything on the client. It’s very discrete.
  • unpredictability of Stealth runs is offered through Stealth’s --random-interval and --repeat options. E.g., --repeat 60 --random-interval 30 results in new Stealth-runs on average every 75 seconds.
  • Subscribes to the Unix philosophy: “do one thing and do it well”
  • Stealth’s author is very approachable and open. After talking with Frank and suggesting some ideas to promote Stealth and it’s community, Frank started a discussion list.

What I don’t like

  • Lack of visible code reviews and testing.Yes it’s in Debian, but so was OpenSSL and Bash
  • One man band. Support provided via one person alone via email. Comparing with the likes of Ossec which has …
  • Lack of use cases. I don’t see anyone using/abusing it. Although Frank did send me some contacts of other people that are using it, so again, a very helpful author. Can’t find much on the interwebs. The documentation has clear signs that it’s been written and is targeting people already familiar with the tool. This is understandable as the author has been working on this project for nine years and could possibly be disconnected with what’s involved for someone completely new to the project to dive in and start using it. In saying that, that’s what I did and so far it worked out well.
  • This tells me that either very few are using it or it has no bugs and the install and configuration is stupidly straight forward or both.
  • Small user base. This is how many people are happy to reveal they are using Stealth.
  • Reading through the userguide, the following put me off a little: “preferably the public ssh-identity key of the controller should be placed in the client’s root .ssh/authorized_keys file, granting the controller root access to the client. Root access is normally needed to gain access to all directories and files of the client’s file system.” I never allow SSH root access to servers. So I’m not about to start. What’s worse, is that Stealth SSHing from server to client with key-pair can only do so automatically if the pass-phrase is blank. If it’s not blank then someone has to drive Stealth and the whole idea of Stealth (as far as I can tell) is to be an automatic file integrity checker.
    There are however some work-arounds to this. ssh-cron can run scheduled Stealth jobs, but needs to aquire the pass-phrase from the user once. It does this via ssh-askpass which is a X11 based pass-phrase input dialog. If your running ssh-cron and Stealth from a server (which would be a large number of potential users I would think) you won’t have X11. So if that’s the case ssh-cron is out of the question. At least that’s how I understand it from the man page and emails from Frank. Frank mentioned in an email: “Stealth itself could perfectly well be started `by hand’ setting up the connection using, e.g., an existing ssh private-key, which could thereafter completely be removed from the system running Stealth. The running Stealth process would then continue to use the established connection for as long as it’s running. This may be a very long time if the –daemon option is used.” I’ve used Monit to do any checks on the client that need root access. This way Stealth can run as a low privileged user.
  • In order to use an SSH key-pair with passphrase and have your controller resume scans on reboot, you’re going to need ssh-cron. Some distros like Mint and Ubuntu only have very old versions of libbobcat (3.19.01 (December 2013)) in their repositories. You could re-build if you fancy the dependency issues it may bring. Failing that, use Debian which is way more up to date, or just fire stealth off each time you reboot your controller machine manually and run it as a daemon with arguments such as --keep-alive (or --daemon if running stealth >= 4.00.00), --repeat. This will cause Stealth to keep running (sleeping) most of the time, then doing it’s checks at the intervals you define.

Outcomes

In making all of my considerations, I changed my mind quite a few times on which offerings were most suited to which environments. I think this is actually a good thing, as I think it means my evaluations were based on the real merits of each offering rather than any biases.

If you already have real-time logging to an off-site syslog server set-up and alerting. OSSEC would provide redundant features.

If you don’t already have real-time logging to an off-site syslog server, then OSSEC can help here.

The simplicity of Stealth, flatter learning curve and it’s over-all philosophy is what won me over.

Stealth Up and Running

I tested this out on a Mint installation.

Installed stealth and stealth-doc (2.11.03-2) via synaptic package manager. Then just did a locate for stealth to find the docs and other example files. The following are the files I used for documentation, how I used them and the tab order that made sense to me:

  1. The main documentation index: file:///usr/share/doc/stealth-doc/manual/html/stealth.html
  2. Chapter one introduction: file:///usr/share/doc/stealth-doc/manual/html/stealth01.html
  3. Chapter three to help build up a policy file: file:///usr/share/doc/stealth-doc/manual/html/stealth03.html
  4. Chapter five for running Stealth and building up policy file: file:///usr/share/doc/stealth-doc/manual/html/stealth05.html
  5. Chapter six for running Stealth: file:///usr/share/doc/stealth-doc/manual/html/stealth06.html
  6. Chapter seven for arguements to pass to Stealth: file:///usr/share/doc/stealth-doc/manual/html/stealth07.html
  7. Chapter eight for error messages: file:///usr/share/doc/stealth-doc/manual/html/stealth08.html
  8. The Man page: file:///usr/share/doc/stealth/stealthman.html
  9. Policy file examples: file:///usr/share/doc/stealth/examples/
  10. Useful scripts to use with Stealth: file:///usr/share/doc/stealth/scripts/usr/bin/
  11. All of the documentation in simple text format (good for searching across chapters for strings): file:///usr/share/doc/stealth-doc/manual/text/stealth.txt

Files I would need to copy and modify were:

  • /usr/share/doc/stealth/scripts/usr/bin/stealthcleanup.gz
  • /usr/share/doc/stealth/scripts/usr/bin/stealthcron.gz
  • /usr/share/doc/stealth/scripts/usr/bin/stealthmail.gz

Files I used for reference to build up a policy file:

  • /usr/share/doc/stealth/examples/demo.pol.gz
  • /usr/share/doc/stealth/examples/localhost.pol.gz
  • /usr/share/doc/stealth/examples/simple.pol.gz

As mentioned above, providing you have a working MTA, then Stealth will just do it’s thing when you run it. The next step is to schedule it’s runs. This can be also (as mentioned above) with a pseudo random interval.

Feel free to leave a comment if you need help setting Stealth up, as it did take a bit of fiddling, but does what it says it does on the box very well.

Tags:, , , , , , , , , , , , ,
Posted in GNU/Linux, Networking, Security | Leave a Comment »