A friend of mine recently asked me what is the maximum number of logs per second that ossec could handle, but I didn’t have an answer for him. I heard of a few reports of ossec handling more than 508 logs per second in a setup with more than 400 agents. I also installed it on an ISP that on average receives between 200/250 logs per second. However, there is no definitive answer to this question because it depends a lot on the hardware being used and the format of the logs… Anyway, I decided to run some quick tests to see how good (or bad) it would perform here.
My performance test was very simple: send as many logs as possible to analysisd (main ossec process) and check the /var/ossec/stats/totals directory to see how many logs it was able to process per second.
Test setup:
I created 5 ossec configurations for logcolletor and initiated one separated daemon for each config. Each process was monitoring one log file. The operating system was OpenBSD 3.9 on an old PIII 700 with 512M of RAM.
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log1.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log2.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log3.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log4.conf
# /var/ossec/bin/ossec-logcollector -c /var/ossec/etc/log5.conf
# tail /var/ossec/logs/ossec.log
2007/01/20 15:14:49 ossec-logcollector(1950): Analyzing file: ‘/data/test-logs/log1′.
2007/01/20 15:14:50 ossec-logcollector(1950): Analyzing file: ‘/data/test-logs/log2′.
..
2007/01/20 15:14:56 ossec-logcollector(1950): Analyzing file: ‘/data/test-logs/log5′.
2007/01/20 15:14:56 ossec-logcollector: Started (pid: 15448).
To be fair, I chose 5 different log formats and wrote a simple script to keep filling the logs as faster as possible.
# cd /data/test-logs
# while [ 1 ]; do ./fill-logs.sh; done
# cat fill-logs.sh
cat PIX-sample >> log1 &
cat accesslog-sample >> log2 &
cat authlog-sample >> log3 &
cat messages-sample >> log4
cat squid-sample >> log5
Test results:
I let this setup running for a while (6 to 7 hours) and also wrote a small script to monitor the process/memory utilization. On average, the CPU utilization was around 10% with peaks of 18%. Memory usage was constant of about 2100K.
The average number of events that I got during this 6 hour test was around 1,238,989 or 344 logs per second.
What does it proves? Nothing. Just that OSSEC v1.0 on an old PIII with 512 of RAM can support more than 340 logs per second without using more than 10% of CPU..
Do you run ossec on a large environment? How many logs per second is it monitoring on your environment? Would be nice to have a few examples for future comparison.