Category: OSSEC Update

A friend of mine recently asked me what is the maximum number of logs per second that ossec could handle, but I didn’t have an answer for him. I heard of a few reports of ossec handling more than 508 logs per second in a setup with more than 400 agents. I also installed it on an ISP that on average receives between 200/250 logs per second. However, there is no definitive answer to this question because it depends a lot on the hardware being used and the format of the logs… Anyway, I decided to run some quick tests to see how good (or bad) it would perform here.

My performance test was very simple: send as many logs as possible to analysisd (main ossec process) and check the /var/ossec/stats/totals directory to see how many logs it was able to process per second.

Test setup:
I created 5 ossec configurations for logcolletor and initiated one separated daemon for each config. Each process was monitoring one log file. The operating system was OpenBSD 3.9 on an old PIII 700 with 512M of RAM.
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log1.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log2.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log3.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log4.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log5.conf

# tail /var/ossec/logs/ossec.log
2007/01/20 15:14:49 ossec-logcollector(1950): Analyzing file: ‘/data/test-logs/log1′.
2007/01/20 15:14:50 ossec-logcollector(1950): Analyzing file: ‘/data/test-logs/log2′.
..
2007/01/20 15:14:56 ossec-logcollector(1950): Analyzing file: ‘/data/test-logs/log5′.
2007/01/20 15:14:56 ossec-logcollector: Started (pid: 15448).

To be fair, I chose 5 different log formats and wrote a simple script to keep filling the logs as faster as possible.

# cd /data/test-logs
# while [ 1 ]; do ./fill-logs.sh; done
# cat fill-logs.sh
cat PIX-sample >> log1 &
cat accesslog-sample >> log2 &
cat authlog-sample >> log3 &
cat messages-sample >> log4
cat squid-sample >> log5

Test results:
I let this setup running for a while (6 to 7 hours) and also wrote a small script to monitor the process/memory utilization. On average, the CPU utilization was around 10% with peaks of 18%. Memory usage was constant of about 2100K.

The average number of events that I got during this 6 hour test was around 1,238,989 or 344 logs per second.

What does it proves? Nothing. Just that OSSEC v1.0 on an old PIII with 512 of RAM can support more than 340 logs per second without using more than 10% of CPU..
Do you run ossec on a large environment? How many logs per second is it monitoring on your environment? Would be nice to have a few examples for future comparison.

OSSEC version 1.0 is now publicly available. This version comes with numerous new features, including support for:Registry monitoring on WindowsDynamic/nat’ed IP addresses in the server/agent communicationASL (Apple system log)Lotus dominoSymantec AVWindows RAR

A full list with all the functionatilies and bug fixes are available at the Changelog.

Make sure to check, as well, our new web interface (in beta) for monitoring your logs and ossec alerts.

Download the new version (and the web interface) here.

As a late christmas gift to all curious OSSEC users out there, here is some information about the number of downloads in 2006 (note that I only included major released versions). I am very pleased to see that we went from 596 downloads on version 0.5 to almost 5,000 on 0.9-3 (great work to all contributors and to the dev team)…

It is also interesting to note that less than 10% of our users check their downloads using our pgp key and only 15% check the md5/sha1 sum of the packages. Actually, the average of downloads of the pgp keys is only 6% and only 13% for the checksum. I would expect these numbers to be much higher..

How I got these values? Just a simple combination of cat/grep/cut/sort/uniq made the deal:

cat $LOGS | cut -d ” -f 2 | cut -d ” ” -f 2 | grep -E “.tar.gz$|.exe$” | grep -v checksum | grep -o -E “([^/]+)$” | cut -d ? -f 1 | sort | uniq -c

Basically, I cated my log file, extracted the arguments from it (cut and grep), sorted (sort) and generated the total count using uniq… There is probably other ways to do that, but this is what I came up with. If you want a simple way to see the total number for all the files in your logs, the following should work:

cat access_log | cut -d ” -f 2 | cut -d ” ” -f 2 | cut -d ? -f 1 | sort | uniq -c

*This will only work on Apache access logs.

As requested, I am attaching a few screenshots of our web ui. I will also post a link to a demo site later…

Below is an explanation of each picture (click on them to expand).

1. Main page. Show the agent status, latest modified files and latest events.
2. Search results (looking for multiple authentication failures).
3. Search options.
4. Stats page.

The second beta version of oswui (ossec web ui) is available. It has numerous bug fixes (for all issues reported so far) and new features, including support for php5 and major design improvements made by Rafael Capovilla.Download it from here.

Installation instructions below:1.0 – Prerequisites

• -Apache with PHP (4.x or 5.x) installed.
• -OSSEC (version >0.9-3) already installed.

1.1 – Download the package (link)

1.2 – Uncompress/untar it and move the files to somewhere acessible by your web server

# tar -zxvf ossec-wui-0.1-BETA2.tar.gz
# mv ossec-wui* /var/www/htdocs/ossec-wui

1.3 – Run the setup script

# cd /var/www/htdocs/ossec-wui
# ./setup.sh

1.4 – Add your web server user to the ossec group

# vi /etc/group
..
From:
ossec:x:1002:
To (if your web server user is www):
ossec:x:1002:www

1.5 – Fix the permissions for the tmp directory and restart Apache (for the new permissions to work)

# chmod 770 tmp/
# chgrp www tmp/
# apachectl restart

As the ossec-hids project, it is released under the GPL and donations are welcome.

We have a new snapshot available for testing. It includes the following new features (in addition of multiple bug fixes):Rules for Symantec AV.More information to the active response scripts and changed them to log to the logs directorySupport for glob formated files in the logcollector listRules for Windows Routing and Remote Access logs

In addition of these features, it also has the ones reported at http://www.ossec.net/dcid/?p=19.

Download it from http://www.ossec.net/files/snapshots/ and always make sure to get the latest file available.

**Testing the snapshots is one of the best ways to help with ossec development!.

It has been a while since the last time I released a stable snapshot of ossec. However, after lots of new features and code changes, it is the time to start doing that again. Hopefully we will add all the features we need for version 1.0 before christmas and release it sometime in January.

What features this version have:Support for dynamic IP addresses.Events are now stored in its full format (with date, hostname).Support for Lotus Domino and Windows logs in the NTsylog format.

If you want to test the dynamic IP authentication, you need to give the network + netmask where the agent is (when adding it using manage_agents), instead of the unique IP.

For example, to add an agent that is in a 192.168.2.0/24 network, do the following (taken from manage_agents):

Please provide the following:
* A name for the new agent: test
* The IP Address of the new agent: 192.168.2.0/24

Please remember that all snapshots are in beta state and you should only use them for testing purposes. Download it from here (make sure to get the latest version available).

I just released the first beta version of oswui (ossec web ui). The code is very simple and does not require a database or anything special running in the server. Hopefully after this release some web programmers will join us to help improve it (I am not a gui person).Download it from here.

Installation instructions below:1.0 – Prerequisites

• -Apache with PHP (4.x or 5.x) installed.
• -OSSEC (version 0.9-3) already installed.

1.1 – Download the package (link)

1.2 – Uncompress/untar it and move the files to somewhere acessible by your web server

# tar -zxvf ossec-wui-0.1-BETA.tar.gz
# mv ossec-wui* /var/www/htdocs/ossec-wui

1.3 – Run the setup script

# cd /var/www/htdocs/ossec-wui
# ./setup.sh

1.4 – Add your web server user to the ossec group

# vi /etc/group
..
From:
ossec:x:1002:
To (if your web server user is www):
ossec:x:1002:www

1.5 – Fix the permissions for the tmp directory and restart Apache (for the new permissions to work)

# chmod 770 tmp/
# chgrp www tmp/
# apachectl restart

As the ossec-hids project, it is released under the GPL and donations are welcome.

I am in the process to document all ossec rules at the project’s wiki. We currently have 402 rules and I just finished beta-documenting all apache and arpwatch ones. My main goal is to provide for each rule a simple explanation, false positives causes and some sample logs…  Since this is a manual task, I am inviting all ossec users to help me out with that. Just go to the rules index page and choose one that is not documented yet. For a simple example, go to the rule 30108.