{"id":296,"date":"2007-04-10T03:51:10","date_gmt":"2007-04-10T03:51:10","guid":{"rendered":"https:\/\/defragged.org\/ossec\/?p=296"},"modified":"2020-07-03T03:52:00","modified_gmt":"2020-07-03T03:52:00","slug":"ossec-performance-v2","status":"publish","type":"post","link":"https:\/\/defragged.org\/ossec\/2007\/04\/ossec-performance-v2\/","title":{"rendered":"OSSEC performance (v2)"},"content":{"rendered":"\n

During the release of ossec v1.0<\/a>, I posted some performance numbers<\/a> regarding that version. Even though I know most performance tests do not prove anything per se, I was able to see how many events per second an old PIII<\/em> box with 512M of ram could handle.<\/p>\n\n\n\n

Recently, I finished doing multiple performance improvements to ossec<\/a> and I decided to re-run this test using the latest available CVS version.<\/p>\n\n\n\n

I took the same steps as before, basically sending as many logs as possible to analysisd<\/strong> (main ossec process) and checking \/var\/ossec\/stats\/totals<\/strong> to see how many logs it was able to process per second.<\/p>\n\n\n\n

Test setup:<\/h3>\n\n\n\n

I created 5 ossec configurations for logcolletor and initiated one separated daemon for each config. Each process was monitoring one log file. The operating system was OpenBSD 3.9 on an old PIII 700 with 512M of RAM.
# \/var\/ossec\/bin\/ossec-logcollector -c \/var\/ossec\/etc\/log1.conf
# \/var\/ossec\/bin\/ossec-logcollector -c \/var\/ossec\/etc\/log2.conf
# \/var\/ossec\/bin\/ossec-logcollector -c \/var\/ossec\/etc\/log3.conf
# \/var\/ossec\/bin\/ossec-logcollector -c \/var\/ossec\/etc\/log4.conf
# \/var\/ossec\/bin\/ossec-logcollector -c \/var\/ossec\/etc\/log5.conf<\/strong><\/p>\n\n\n\n

# tail \/var\/ossec\/logs\/ossec.log
<\/strong>2007\/01\/20 15:14:49 ossec-logcollector(1956): Analyzing file: \u2018\/data\/test-logs\/log1\u2032.
2007\/01\/20 15:14:50 ossec-logcollector(20567): Analyzing file: \u2018\/data\/test-logs\/log2\u2032.
..
2007\/01\/20 15:14:56 ossec-logcollector(32102): Analyzing file: \u2018\/data\/test-logs\/log5\u2032.
2007\/01\/20 15:14:56 ossec-logcollector: Started (pid: 15448).<\/em><\/p>\n\n\n\n

# cat \/var\/ossec\/etc\/log1.conf<\/strong>

<ossec_config>
<localfile>
<log_format>syslog<\/log_format>
<ocation>\/data\/test-logs\/log1<\/location>
<\/localfile>
<\/ossec_config><\/em><\/p>\n\n\n\n

To be fair, I chose 5 different log formats and wrote a simple script to keep filling the logs as faster as possible.<\/p>\n\n\n\n

# cd \/data\/test-logs
# while [ 1 ]; do .\/fill-logs.sh; done
# cat fill-logs.sh
<\/strong>cat PIX-sample >> log1 &
cat accesslog-sample >> log2 &
cat authlog-sample >> log3 &
cat messages-sample >> log4
cat squid-sample >> log5<\/em><\/p>\n\n\n\n

Test results:<\/h3>\n\n\n\n

To my surprise, my changes made a great improvement to the overall ossec performance. Previously, ossec was able to handle in average 1,238,000<\/strong> events per hour (or 340 per second<\/strong>).<\/p>\n\n\n\n

On the latest CVS version, this number jumped to 2,100,000<\/strong>events per hour (or 600 per second<\/strong>). This is a big jump<\/em>! Note that I am using real data<\/strong> and that this sample is generating an average of 1,000,000<\/strong> alerts per hour (rate or 1 alert per 2 events<\/em>). On a normal<\/em> environment this value would be much lower.<\/p>\n\n\n\n

Summarized stats output from the UI:<\/h3>\n\n\n\n
\nHour  \t         Alerts  \tAlerts %  \tTotal  \tTotal %\nHour 14  \t1,046,427  \t5.0%   \t2,140,978  \t5.2%\nHour 15 \t1,034,108 \t4.9% \t2,142,076 \t5.2%\nHour 16 \t1,041,571 \t5.0% \t2,096,306 \t5.1%\nHour 17 \t1,016,219 \t4.8% \t2,048,642 \t5.0%\n<\/em><\/pre>\n\n\n\n
Performance values per version:<\/h3>\n\n\n\n
*on a PIII, 512RAM, OpenBSD 3.9<\/em><\/p>\n\n\n\n
Version 1.0           1,238,000 per hour     340 per second\nVersion 1.2           2,100,000 per hour     600 per second<\/pre>\n\n\n\n
If you have some spares systems to execute a similar test, let us know how it goes and we will publish the results.<\/p>\n","protected":false},"excerpt":{"rendered":"
During the release of ossec v1.0, I posted some performance numbers regarding that version. Even though I know most performance tests do not prove anything per se, I was able to see how many events per second an old PIII box with 512M of ram could handle. Recently, I finished doing multiple performance improvements to ossec and I decided to re-run this […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[],"_links":{"self":[{"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/posts\/296"}],"collection":[{"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/comments?post=296"}],"version-history":[{"count":1,"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/posts\/296\/revisions"}],"predecessor-version":[{"id":297,"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/posts\/296\/revisions\/297"}],"wp:attachment":[{"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/media?parent=296"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/categories?post=296"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/defragged.org\/ossec\/wp-json\/wp\/v2\/tags?post=296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}