Month: April 2007

bookmark_borderCEE – Logging standard

Posted on by Daniel Cid

If you are not at the log analysis mailing list, you are missing a good discussion regarding the efforts to create a new logging standard, CEE (Common Event Expression). MITRE is in charge of the process, but it is probably sponsored by Log logic (1), since they were the first ones to report about it.

Before I go any further, I would like to say that I am very interested in this initiative and that I already contacted MITRE to be a part of the CEE working group. Unfortunately, I am not very optimistic that it is going to be widely adopted (hope I am wrong).

First of all, it will require significant changes to all major applications and if the protocols are not very well designed, no one is going to use it.

Secondly, the protocol must be simple enough to be fast and non-blocking (like syslog), but still reliable, with support for encryption, etc.

Thirdly, I am always worried by protocols designed by security people. Most of them have no software engineering experience and if CEE looks anything like IDMEF or SDEE it will go no where.

Anyway, besides my lack of optmism, I will still contribute to it and if it get past the design phase, I will volunteer to write free libraries (LGPL or BSD licensed) to support it.

If you want more information, check out the following blog entries (by Anton Chuvakin and Raffy’s:

Finally, Common Event Expression (CEE) is Out!!!
CEE brochure
Standard Logging Format – Common Event Expression (CEE)

[1] Edit to add (Apr 28 2007): Looks like I spoke too soon (actually without any base) that Log Logic is sponsoring CEE. Thanks Raffy for pointing it out in the comments.

bookmark_borderOSSEC at the SANS log management summit

Posted on by Daniel Cid

David Bianco recently spoke about OSSEC at the SANS Log Management Summit 2007, with the presentation: “How to Save $45k (and Look Great Doing it)”. Quoting his blog entry reviewing the summit:

On Tuesday morning, I gave my own presentation, “How to Save $45k (and Look Great Doing it).” This is the story of how we bought a commercial SEM product, only to find that it didn’t really do what we wanted, and replaced it with the free OSSEC. Bad on us for not having our ducks in a row at first, I know. To be totally honest, it wasn’t so easy to get up in front of 100 people and say, “You know, we made this really expensive mistake”, but sometimes you have to sacrifice for the greater good. 😉

He also mentions Mike Poor’s presentation, “Network Early Warning Systems: Mining Better Quality Data from Your Logging Systems”, where Mike speaks about OSSEC in many of his slides. Mike’s presentation is available here.

If you went to the summit, please share your experiences with us!

bookmark_borderFree Lunch :: OSSEC Review

Posted on by Daniel Cid

Andrew Storms from ncircle posted an interesting review of ossec in his blog (also at the ncircle main blog):

OSSEC is an open source host based intrusion detection system. The website states, “It performs log analysis, integrity checking, Windows registry monitoring, rootkit detection, time-based alerting and active response.” That is a mouthful.

Check it out: here.

bookmark_borderWiki editing blocked (vandalism)

Posted on by Daniel Cid

I decided to block any form of editing to the ossec wiki for reasons of vandalism. If you look at the wiki recent changes page you will see the changes that were made. Most of them were very strange to me, like removing every + (plus) from the pages or removing all the content (without adding any spam link or anything). Anyone seeing similar patterns?

After some log analysis I found that all the changes were made by the same IP address (200.238.102.170) across the last three days…

Sample log:


200.238.102.162 – – [11/Apr/2007:18:00:35 -0300] “GET /wiki/index.php?title=Log_Samples_Lotus_Domino&action=edit HTTP/1.1″ 200 6702 “http://www.ossec.net/wiki/index.php/Log_Samples_Lotus_Domino” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”

200.238.102.170 – – [11/Apr/2007:17:08:36 -0300] “GET /wiki/index.php?title=Log_Samples_Solaris&action=edit HTTP/1.1″ 200 6667 “http://www.ossec.net/wiki/index.php/Log_Samples_Solaris” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”

How do you guys handle vandalism/spam on your wikis? Any suggestion? Until I can find a solution to this, send me an e-mail if you need to edit the wiki.

bookmark_borderOSSEC performance (v2)

Posted on by Daniel Cid

During the release of ossec v1.0, I posted some performance numbers regarding that version. Even though I know most performance tests do not prove anything per se, I was able to see how many events per second an old PIII box with 512M of ram could handle.

Recently, I finished doing multiple performance improvements to ossec and I decided to re-run this test using the latest available CVS version.

I took the same steps as before, basically sending as many logs as possible to analysisd (main ossec process) and checking /var/ossec/stats/totals to see how many logs it was able to process per second.

Test setup:

I created 5 ossec configurations for logcolletor and initiated one separated daemon for each config. Each process was monitoring one log file. The operating system was OpenBSD 3.9 on an old PIII 700 with 512M of RAM.
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log1.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log2.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log3.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log4.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log5.conf

# tail /var/ossec/logs/ossec.log
2007/01/20 15:14:49 ossec-logcollector(1956): Analyzing file: ‘/data/test-logs/log1′.
2007/01/20 15:14:50 ossec-logcollector(20567): Analyzing file: ‘/data/test-logs/log2′.
..
2007/01/20 15:14:56 ossec-logcollector(32102): Analyzing file: ‘/data/test-logs/log5′.
2007/01/20 15:14:56 ossec-logcollector: Started (pid: 15448).

# cat /var/ossec/etc/log1.conf

<ossec_config>
<localfile>
<log_format>syslog</log_format>
<ocation>/data/test-logs/log1</location>
</localfile>
</ossec_config>

To be fair, I chose 5 different log formats and wrote a simple script to keep filling the logs as faster as possible.

# cd /data/test-logs
# while [ 1 ]; do ./fill-logs.sh; done
# cat fill-logs.sh
cat PIX-sample >> log1 &
cat accesslog-sample >> log2 &
cat authlog-sample >> log3 &
cat messages-sample >> log4
cat squid-sample >> log5

Test results:

To my surprise, my changes made a great improvement to the overall ossec performance. Previously, ossec was able to handle in average 1,238,000 events per hour (or 340 per second).

On the latest CVS version, this number jumped to 2,100,000events per hour (or 600 per second). This is a big jump! Note that I am using real data and that this sample is generating an average of 1,000,000 alerts per hour (rate or 1 alert per 2 events). On a normal environment this value would be much lower.

Summarized stats output from the UI:


Hour  	         Alerts  	Alerts %  	Total  	Total %
Hour 14  	1,046,427  	5.0%   	2,140,978  	5.2%
Hour 15 	1,034,108 	4.9% 	2,142,076 	5.2%
Hour 16 	1,041,571 	5.0% 	2,096,306 	5.1%
Hour 17 	1,016,219 	4.8% 	2,048,642 	5.0%

Performance values per version:

*on a PIII, 512RAM, OpenBSD 3.9

Version 1.0           1,238,000 per hour     340 per second
Version 1.2           2,100,000 per hour     600 per second

If you have some spares systems to execute a similar test, let us know how it goes and we will publish the results.

bookmark_borderHow to compile ossec on Windows?

Posted on by Daniel Cid

It is not the first time I was asked that, so I decided to write it in here in case anyone else is interested. First of all, ossec is compiled using MinGW, so we have only used it with gcc. We do not support any other compiler, but feel free to use them if you want.

With that being said, you now need the windows specific files. Go to your ossec source code (on Unix) and execute the “gen_win.sh” script. It will create all the necessary files for the Windows installation inside src/win-pkg.

$ pwd
/tmp/ossec-hids-1.1/src/win32/
$ ./gen_win.sh
..
$ cd ../win-pkg
$ zip ossec-win.zip win-pkg

After that, copy the “win-pkg” directory to your Windows system and execute the “make.bat” script on it.

C:>cd win-pkg
C:win-pkg>make
..

If your MinGW installation is not a C:MinGW, you will need to set the right paths on the make script. When you are done with your changes, you can generate the install.exe by compiling ossec-installer.nsi with NSIS.

bookmark_borderContributing to the UI development

Posted on by Daniel Cid

I have received a large feedback from the community regarding the web interface with lots of offers to help. If you are interested in being a part of the UI development team, check out the following link with some ideas to implement: OSSEC Wui todo list.

If you have any questions, just send us an e-mail or comment in here. If you decide to work on any of the suggestions in there, feel free to modify the list and set the item as “under work”.