bookmark_borderRealtime file integrity monitoring

OSSEC supports realtime (continuous) file integrity monitoring on Linux systems (since v2.2) and on the latest snapshot we added support for Windows too.

The configuration is very simple. In the <directories> option where you specify what files or directories to monitor, you just need to add the realtime=”yes” attribute. For example:

<syscheck>
<directories realtime=”yes” check_all=”yes”>/etc,/usr/bin,/usr/sbin</directories>
<directories check_all=”yes”>/bin,/sbin</directories>
</syscheck>

In this case, the directories /etc/, /usr/bin and /usr/sbin will be monitored in real time. The same applies to Windows too. A few notes:

  1. The real time monitoring will not start right away. First OSSEC needs to scan the file system and adds each sub-directory to the realtime queue. It can take up to 30 minutes for that (wait for the log “ossec-syscheckd: INFO: Starting real time file monitoring” ).
  2. It only works with directories, not individual files. So you can monitor the /etc or C:program files directory, but not an individual file like /etc/file.txt.

If you are interested in this feature, please try it out on both Linux and Windows.

bookmark_borderWordPress to Syslog

WPsyslog2 is a global log plugin for WordPress. It keeps track of all system events and log them to syslog. It tracks events such as new posts, new profiles, new users, failed logins, logins, logouts, etc.

It also tracks the latest vulnerabilities and alerts if any of them are triggered, becoming very useful when integrated with a log analysis tool, like OSSEC HIDS. OSSEC requires the latest snapshot to properly analyze these logs: http://ossec.net/files/snapshots/ossec-hids-090826.tar.gz

More information, download, etc: http://www.ossec.net/wpsyslog2

bookmark_borderCompiling the Windows Agent from a Linux system

It has always been a pain to generate snapshots for Windows because it required me to open up my Windows VM (slow), push the code there, compile, etc. Well, until this week when I started to play with MinGW cross-compilation feature to completely build an Windows agent from Linux.

How it works? First, you need to install MinGW and makensis (to build the installer). On Ubuntu, it is just:

apt-get install mingw32 mingw32-binutils mingw32-runtime
apt-get install nsis

After that, download the latest snapshot: http://ossec.net/files/snapshots/ and generate the Windows package directory (where XX is the latest date):

wget http://www.ossec.net/files/snapshots/ossec-hids-xx.tar.gz
tar -zxvf ossec-hids-xx.tar.gz
cd ossec-hids-xx/src/win32
./gen-win.sh

Now, you will have the win-pkg directory under src. Just go there and run make.sh. Your Windows agent package should be created in a few minutes:

cd ../win-pkg
sh ./make.sh

You will see the following in the screen:

Making windows agent
rootcheck/win-common.c: In function ‘__os_winreg_querykey’:
rootcheck/win-common.c:279: warning: pointer targets in passing argument 7 of ‘RegEnumValueA’ differ in signedness
win-registry.c: In function ‘os_winreg_querykey’:

Output: “ossec-win32-agent.exe”
Install: 7 pages (448 bytes), 3 sections (3144 bytes), 379 instructions (10612 bytes), 247 strings (42580 bytes), 1 language table (346 bytes).
Uninstall: 5 pages (320 bytes),
1 section (1048 bytes), 301 instructions (8428 bytes), 166 strings (2646 bytes), 1 language table (290 bytes).
Datablock optimizer saved 8371 bytes (~0.7%).

Which means that your agent executable ossec-win32-agent.exe has been created properly.

bookmark_borderCentralized agent configuration

If you wanted to be able to configure your agents remotely, you will be happy to know that we are supporting it now. Right now, we allow centralized configuration for file integrity checking (syscheckd), rootkit detection (rootcheck) and log analysis.

This is how it works.

Create the file /var/ossec/etc/shared/agent.conf

Inside the file you can configure the agent just as you would normally at ossec.conf

<agent_config>
<localfile>
<location>/var/log/my.log</location>
<logtype>syslog</logtype>
</localfile>
</agent_config>

But you have a few more options. You can restrict the config by agent name or by operating system:

<agent_config name=”agent1|agent2″>
<localfile>
<location>/var/log/my.log</location>
<logtype>syslog</logtype>
</localfile>
</agent_config>

<agent_config os=”Linux|OpenBSD”>
<localfile>
<location>/var/log/my.log2</location>
<logtype>syslog</logtype>
</localfile>
</agent_config>

<agent_config os=”Windows”>
<localfile>
<location>C:myappmy.log</location>
<logtype>syslog</logtype>
</localfile>
</agent_config>

And only the proper agent will read them, giving us great granularity to push the configuration to all your agents.

After you configured, the manager will push it to the agents. Note that it can take a while for it to complete (since the manager caches the shared files and only re-reads them every few hours). If you restart the manager the configuration will be pushed much quicker.

bookmark_borderOSSEC to Twitter

Want to see your OSSEC alerts on twitter? We just added support for that on the latest snapshot.

To make it work, first update to the latest snapshot and add the following to your ossec.conf:

<command>
<name>ossec-tweeter</name>
<expect></expect>
<executable>ossec-tweeter.sh</executable>
</command>

<active-response>
<command>ossec-tweeter</command>
<location>server</location>
<level>6</level>
</active-response>

Where the first entry “command” is defining the active response script and the second one “active-response” is configuring when to execute it. In my example above, I chose to execute it for every alert >= level 6. Note, that you can’t run it on the individual agents, just on the management server.

After that, you need to configure the twitter script with the appropriate credentials. So, open the file /var/ossec/active-response/bin/ossec-tweeter.sh and modify the first 2 entries:

TWITTERUSER=”youruser”
TWITTERPASS=’yourpass’

If you want to send direct messages to yourself (or any other account), also set the direct msg user variable:

DIRECTMSGUSER=””

After that, just restart OSSEC and wait for the updates. If you want some examples, check the ossec twitter page.

bookmark_borderIntegrity checking application profiles

We are trying to make syscheck (the integrity checking module on ossec) more useful than what it is now and we are looking for contributions to create application profiles. What we are looking exactly is a list of files/directories (or registry entries) per application to be added to ossec.

For example, files used by qmail:

/var/qmail/control/
/var/qmail/rc

by Named:

/var/named/chroot/etc
/var/named/data/

By Apache:

/var/www/htdocs/

A few more is on the wiki: http://www.ossec.net/wiki/index.php/Dev:Syscheck

We need that for Windows and Unix applications (including IIS, Anti virus, etc), open source and commercial… Any feedback is more than welcome.

*You can add on your own to the wiki or just reply in here..

bookmark_borderAgentless monitoring using public key authentication

As a follow up to the previous post (Agentless monitoring), I will now show how to use public key authentication instead of passwords to access the remote boxes.

1- Adding the remote host

When adding the remote host, you need to provide the password as “NOPASS”:

  # /var/ossec/agentless/register_host.sh add root@xx.net NOPASS
   *Host root@xx.netl added.

2-Adding the public key

The public key must be created by the user ossec inside the ossec home directory. To do that, follow these steps:

# sudo -u ossec ssh-keygen

It will create the public keys inside /var/ossec/.ssh . After that, just scp the public key to the remote box and your password less connection should work.

bookmark_borderAgentless monitoring (v1.7 preview)

Agentless monitoring is another big feature coming up with our next version. Basically, it allows you to run integrity checking (and in the future log monitoring) on systems without an agent installed (including routers, firewalls, switches and even Linux/BSD systems).

*To try it out, download the latest snapshot from http://www.ossec.net/files/snapshots/

1-Getting started with agentless

After you installed the latest snapshot (or v1.7), you need to enable the agentless monitoring:

  # /var/ossec/bin/ossec-control enable agentless

And provide the SSH authentication to the host you want to access. For Cisco devices (PIX, routers, etc), you need to provide an additional parameter for the enable password. In this example, I am adding a Linux box (xx.net) and a PIX firewall (pix.fw.local):

  # /var/ossec/agentless/register_host.sh add root@xx.net mypass1 
   *Host root@xx.netl added.
  # /var/ossec/agentless/register_host.sh add pix@pix.fw.local pixpass enablepass
   *Host pix@pix.fw.local added.

# /var/ossec/agentless/register_host.sh list
*Available hosts:
pix@pix.fw.local
root@xx.net

2-Configuring agentless

Once you have added all your systems, you need to configure OSSEC to monitor them. By default, we have 4 agentless types (but we plan to add more soon): ssh_integrity_check_bsdssh_integrity_check_linuxssh_generic_diff and ssh_pixconfig_diff. For the first two, you give a list of directories in the configuration and OSSEC will do the integrity checking of them on the remote box. On the ssh_generic_diff, you give a set of commands to run on the remote box and OSSEC will alert when the output of them changes. The ssh_pixconfig_diff will alert when a Cisco PIX/router configuration changes.

So, for my first system (root@xx.net), I will monitor the /bin, /etc and /sbin directories every 10 hours (if I was using the ssh_integrity_check_bsd, the argument would be the directories as well):

 <agentless> 
    <type>ssh_integrity_check_linux</type>
    <frequency>36000</frequency>
    <host>root@xx.net</host>
    <state>periodic</state>
    <arguments>/bin /etc/ /sbin</arguments>
</agentless>

For my PIX, the configuration looks like:

  <agentless>
    <type>ssh_pixconfig_diff</type>
    <frequency>36000</frequency>
    <host>pix@pix.fw.local</host>
    <state>periodic_diff</state>
  </agentless>

And just to exemplify the ssh_generic_diff I will also monitor ls -la /etc; cat /etc/passwd on the root@xx.net. Note that if you want to monitor any network firewall or switch, you can use the ssh_generic_diff and just specify the commands in the arguments option.

 <agentless>
    <type>ssh_generic_diff</type>
    <frequency>36000</frequency>
    <host>root@xx.net</host>
    <state>periodic_diff</state>
    <arguments>ls -la /etc; cat /etc/passwd</arguments>
  </agentless>

3-Running it

Once the configuration is completed, you can restart OSSEC. You should see something like “Started ossec-agentlessd” in the output. Before each agentless connection is started, OSSEC will do a configuration check to make sure everything is fine. Look at /var/ossec/logs/ossec.log for any error. If you see:

  2008/12/12 15:20:06 ossec-agentlessd: ERROR: Expect command not found (or bad arguments) for 'ssh_integrity_check_bsd'.
  2008/12/12 15:20:06 ossec-agentlessd: ERROR: Test failed for 'ssh_integrity_check_bsd' (127). Ignoring.

It means that you don’t have the expect library installed on the server (it is not necessary to install anything on the agentless systems to monitor). On Ubuntu you can do the following to install:

# apt-get install expect

After installing expect, you can restart OSSEC and you should see:

  2008/12/12 15:24:12 ossec-agentlessd: INFO: Test passed for 'ssh_integrity_check_bsd'.

When it connects to the remote system, you will also see:

2008/12/12 15:25:19 ossec-agentlessd: INFO: ssh_integrity_check_bsd: root@xx.net: Starting.
2008/12/12 15:25:46 ossec-agentlessd: INFO: ssh_integrity_check_bsd: root@xx.net: Finished.

4-Alerts

These are some of the alerts you will get:

For the ssh_generic_diff:

OSSEC HIDS Notification.
2008 Dec 12 01:58:30

Received From: (ssh_generic_diff) root@xx.net->agentless
Rule: 555 fired (level 7) -> “Integrity checksum for agentless device changed.”
Portion of the log(s):

ossec: agentless: Change detected:
35c35
< -rw-r–r– 1 root wheel 34 Dec 10 03:55 hosts.deny

> -rw-r–r– 1 root wheel 34 Dec 11 18:23 hosts.deny

–END OF NOTIFICATION

For the PIX:

OSSEC HIDS Notification.
2008 Dec 01 15:48:03

Received From: (ssh_pixconfig_diff) pix@pix.fw.local->agentless
Rule: 555 fired (level 7) -> “Integrity checksum for agentless device changed.”
Portion of the log(s):

ossec: agentless: Change detected:
48c48
< fixup protocol ftp 21

> no fixup protocol ftp 21
100c100
< ssh timeout 30

> ssh timeout 50
More changes..

–END OF NOTIFICATION

That’s it. If you run into any problems, let us know.

bookmark_borderOSSEC Reports (v1.7 preview)

One of the most asked features in OSSEC is some kind of built-in reporting mechanism. For v1.7, we plan to have that included and we need some comments and ideas on what kind of features and formats would be most useful.

To get started, we created a standalone reporting tool so we can get the ideas flying. If you want to try it out, download the latest snapshot and look at the ossec-reportd tool. Here are some of the reports I am doing:

*the html is messing up with the output of the reports, so they will look a bit better in the terminal.

1-Show all IP addresses/users that logged in during the day

# cat /var/ossec/logs/alerts/alerts.log | ./src/monitord/ossec-reportd -n “Logins summary” -f group authentication_success

Report ‘Logins summary’ completed.
————————————————
->Processed alerts: 145557
->Post-filtering alerts: 401
->First alert: ..
->Last alert: ..

Top entries for ‘Source ip’:
————————————————
127.0.0.1 |280 |
192.168.2.10 |88 |
192.168.2.15 |16 |
192.168.2.26 |6 |
192.168.2.17 |2 |

Top entries for ‘Username’:
————————————————
dcid |336 |
aabbcxx |46 |
root |9 |
__vmware_user__ |4 |
vpxuser |2 |
Administrator |1 |
lac |1 |

2-Show all IP addresses/users that logged in during the day and related srcips locations for each user

# cat /var/ossec/logs/alerts/alerts.log | ./src/monitord/ossec-reportd -n “Logins summary” -f group authentication_success -r user srcip -r user location

Top entries for ‘Source ip’:
————————————————
127.0.0.1 |280 |
192.168.2.10 |88 |
192.168.2.15 |16 |
192.168.2.26 |6 |
192.168.2.17 |2 |

Top entries for ‘Username’:
————————————————
dcid |336 |
aabbcxx |46 |
root |9 |
__vmware_user__ |4 |
vpxuser |2 |
Administrator |1 |
lac |1 |

Related entries for ‘Username’:
————————————————
dcid |336 |
location: ‘enigma->/var/log/authlog’
srcip: ’192.168.2.15′
srcip: ’192.168.2.10′
srcip: ’127.0.0.1′
srcip: ’192.168.2.17′
srcip: ’192.168.2.26′
aabbcxx |46 |
location: ‘enigma->/var/log/authlog’
srcip: ’192.168.2.10′
root |9 |
location: ‘enigma->/var/log/authlog’
srcip: ’127.0.0.1′
srcip: ’192.168.2.15′
srcip: ’192.168.2.26′
srcip: ‘(none)’
__vmware_user__ |4 |
location: ‘(lili3win) 192.168.2.0->WinEvtLog’
srcip: ‘(none)’
vpxuser |2 |
location: ‘(vmesx51) any->/var/log/messages’
location: ‘(vmesx51) any->/var/log/vmware/hostd.log’
srcip: ’127.0.0.1′
Administrator |1 |
location: ‘(win2003-tbv4) any->WinEvtLog’
srcip: ‘(none)’
lac |1 |
location: ‘(lili3win) 192.168.2.0->WinEvtLog’
srcip: ‘(none)’

3-Show all multiple authentication failures (brute force attacks)

# cat /var/ossec/logs/alerts/alerts.log | ./src/monitord/ossec-reportd -n “Failures summary” -f group authentication_failures

Top entries for ‘Source ip’:
————————————————
127.0.0.1 |5 |
218.56.61.114 |5 |
117.36.192.75 |2 |
219.90.103.44 |2 |
121.22.8.148 |1 |
122.141.177.51 |1 |
203.171.227.18 |1 |
211.156.250.179 |1 |
222.73.0.101 |1 |
85.24.137.233 |1 |

Top entries for ‘Username’:
————————————————
root |7 |
dcid |5 |

Top entries for ‘Rule’:
————————————————
5720 – Multiple SSHD authentication failures. |12 |
5712 – SSHD brute force trying to get access.. |8 |

4-Show a summary for the month (or day)

# zcat /var/ossec/logs/alerts/2008/Nov/*.gz | ./src/monitord/ossec-reportd -n “Month Summary”

Report ‘Month Summary’ completed.
————————————————
->Processed alerts: 274744
->Post-filtering alerts: 274744
->First alert: 2008 Nov 01 00:00:03
->Last alert: 2008 Nov 25 21:00:03

Top entries for ‘Level’:
————————————————
Severity 4 |236552 |
Severity 10 |33194 |
Severity 3 |2219 |
Severity 7 |1649 |
Severity 5 |999 |
Severity 8 |57 |
Severity 6 |42 |
Severity 2 |25 |
Severity 12 |5 |
Severity 9 |2 |

Top entries for ‘Group’:
————————————————
windows |270107 |
syslog |2694 |
ossec |1798 |
syscheck |1624 |
pam |1339 |
authentication_success |1321 |
sshd |953 |
errors |378 |
system_error |318 |
authentication_failed |161 |
invalid_login |120 |
vmware |117 |
recon |42 |
authentication_failures |32 |
win_authentication_failed |25 |
account_changed |24 |
stats |17 |
time_changed |17 |
service_availability |16 |
accesslog |10 |
web |10 |
su |9 |
access_control |8 |
access_denied |8 |
rootcheck |5 |
attacks |4 |
policy_changed |4 |
low_diskspace |3 |
sudo |3 |
logs_cleared |2 |
postgresql_log |1 |
system_shutdown |1 |

Top entries for ‘Location’:
————————————————
(lili3win) 192.168.2.0->WinEvtLog |269806 |
(esqueleto2) 192.168.2.99->/var/log/auth.log |1338 |
(lili3win) 192.168.2.0->syscheck |1301 |
enigma->/var/log/authlog |960 |
enigma->/var/log/messages |321 |
(lili3win) 192.168.2.0->syscheck-registry |281 |
(win2003-tbv4) any->WinEvtLog |279 |
(vmesx51) any->/var/log/vmware/hostd.log |100 |
enigma->ossec-logcollector |80 |
(vmesx51) any->/var/log/messages |53 |
(win2003-tbv3) any->WinEvtLog |39 |
enigma->ossec-monitord |29 |
(win2003-tbv4) any->syscheck-registry |26 |
(esqueleto2) 192.168.2.99->/var/log/messages |24 |
(lili3win) 192.168.2.0->ossec |22 |
(esqueleto2) 192.168.2.99->ossec-logcollector |15 |
(vmesx51) any->ossec-logcollector |15 |
(esqueleto2) 192.168.2.99->/var/log/syslog |10 |
enigma->/var/www/logs/access_log |10 |
enigma->syscheck |7 |
(win2003-tbv4) any->syscheck |6 |
(vmesx51) any->/var/log/secure |4 |
(vmesx51) any->ossec |3 |
(win2003-tbv4) any->ossec |3 |
(lili3win) 192.168.2.0->rootcheck |2 |
(vmesx51) any->syscheck |2 |
(esqueleto2) 192.168.2.99->/var/log/postgres.. |1 |
(esqueleto2) 192.168.2.99->ossec |1 |
(esqueleto2) 192.168.2.99->rootcheck |1 |
(win2003-tbv3) any->ossec |1 |
(win2003-tbv4) any->rootcheck |1 |
enigma->/var/log/secure |1 |
enigma->dcid@127.0.0.1->syscheck |1 |
enigma->rootcheck |1 |

Top entries for ‘Rule’:
————————————————
18105 – Windows audit failure event. |236165 |
18153 – Multiple Windows audit failure events. |33140 |
550 – Integrity checksum changed. |1484 |
5501 – Login session opened. |666 |
5502 – Login session closed. |666 |
5715 – SSHD authentication success. |580 |
18108 – Failed attempt to perform a privileg.. |354 |
18103 – Windows error event. |318 |
1005 – Syslogd restarted. |313 |
5716 – SSHD authentication failed. |155 |
551 – Integrity checksum changed again (2nd .. |121 |
5710 – Attempt to login using a non-existent.. |119 |
591 – Log file rotated. |110 |
19104 – VMware ESX warning message. |47 |
5706 – SSH insecure connection attempt (scan). |42 |
503 – Ossec agent started. |29 |
19110 – VMWare ESX authentication success. |28 |
5704 – Timeout while logging in (sshd). |28 |
1002 – Unknown problem somewhere in the syst.. |25 |
1006 – Syslogd restarted. |25 |
18130 – Logon Failure – Unknown user or bad .. |25 |
504 – Ossec agent disconnected. |25 |
18111 – User account changed. |24 |
18151 – Multiple failed attempts to perform .. |19 |
552 – Integrity checksum changed again (3rd .. |19 |
11 – Excessive number of events (above norma.. |17 |
18107 – Windows Logon Success. |17 |
18140 – System time changed. |17 |
19112 – VMWare ESX user login. |17 |
5720 – Multiple SSHD authentication failures. |17 |
1004 – Syslogd exiting (logging stopped). |12 |
19120 – Virtual machine state changed to OFF. |12 |
5712 – SSHD brute force trying to get access.. |12 |
31101 – Web server 400 error code. |10 |
5303 – User successfully changed UID to root. |9 |
2503 – Connection blocked by Tcp Wrappers. |8 |
18147 – Application Installed. |6 |
18149 – Windows User Logoff. |6 |
5503 – User login failed. |6 |
18113 – Windows Audit Policy changed. |4 |
19103 – VMware ESX error message. |4 |
40112 – Multiple authentication failures fol.. |4 |
502 – Ossec server started. |4 |
510 – Host-based anomaly detection event (ro.. |4 |
1007 – File system full. |3 |
18152 – Multiple Windows Logon Failures. |3 |
19121 – Virtual machine being turned ON. |3 |
19122 – Virtual machine state changed to ON. |3 |
19150 – Multiple VMWare ESX warning messages. |3 |
18118 – Windows audit log was cleared. |2 |
18119 – First time this user logged in this .. |2 |
18126 – Remote access login success. |2 |
5402 – Successful sudo to ROOT executed |2 |
18109 – Session reconnected/disconnected to .. |1 |
18117 – Windows is shutting down. |1 |
18146 – Application Uninstalled. |1 |
501 – New ossec agent connected. |1 |
50521 – Database shutdown messge. |1 |
512 – Windows Audit event. |1 |
5403 – First time user executed sudo. |1 |
5504 – Attempt to login with an invalid user. |1 |

Comments and suggestions are welcome. We plan to make it a part of monitord to be able to do daily or hourly reports for certain options.