Richard Bejtlich posted an excellent entry in his blog (taosecurity) about the difference between alert centric tools and Network Security Monitoring (NSM). He says:
Network Security Monitoring (NSM) is different. Generating statistical, session, full content, and alert data gives analysts some place to go when they get an alert — or when they want to do manual analysis… With NSM, an alert is the beginning of the investigation, not the end..
He also points that with alert centric tools (including most SIMs), the investigation ends with the alert.
I agree with everything he says, but I would like to add an important point to his NSM approach: Log data. Even with full session/content/alert data, you can not see everything that is happening. One of the easiest ways to exemplify that is with encrypted protocols. How do you know what is happening in a specific sshd session? What about SSL traffic? In addition to that, what happens at the host level? Kernel or processes errors are not going to be seeing at all. I will give some examples bellow:
1- SSHD traffic
You see an SSH connection to your system, how do you know what happened? There is no difference at the network level between a failed login attempt (three password attempts) and a successful quick sshd session. However, if you look at your logs, you have:
Jan 10 11:23:02 enigma1 sshd[10732]: Failed password for invalid user tst from 192.168.2.56 port 10395 ssh2
You know that the authentication failed. However, failure events is not the only think to watch. For example, at OSSEC we use FTS (First time seen) to alerts on first time events. Whenever a user logs to a system that he/she has never logged in before, you get the following:
Received From: enigma2->/var/log/auth.log
Rule: 10100 fired (level 4) -> “First time user logged in.”
Portion of the log(s):
Jan 10 11:29:36 enigma2 sshd[30291]: Accepted publickey for vuser from 192.168.2.67 port 52636 ssh2
It can also alert on multiple failed login attempts that you would never know for sure from the network level:
Location: enigma->/var/log/authlog
Src IP: 85.25.147.156
SSHD brute force trying to get access to the system.
Jan 7 23:33:17 enigma sshd[392]: Failed password for invalid user ftp from 85.25.147.156 port 49786 ssh2
Jan 7 23:33:17 enigma sshd[392]: Invalid user ftp from 85.25.147.156
Jan 7 23:33:16 enigma sshd[23470]: Failed password for invalid user admin from 85.25.147.156 port 49717 ssh2
Jan 7 23:33:16 enigma sshd[23470]: Invalid user admin from 85.25.147.156
Jan 7 23:33:13 enigma sshd[29767]: Failed password for invalid user mysql from 85.25.147.156 port 49583 ssh2
And more importantly, how do you know what happens AFTER a valid user got access to the system? Did he tried to increase his privileges using su/sudo? Did he succeeded? I don’t think there is any way to do know besides monitoring your logs. Using FTS for su/sudo, we know whenever a user succeeded (for the first time) or failed using these commands.
2007 Jan 10 11:49:03
Rule: 5403 (level 4) -> ‘First time user executed sudo.’
Jan 10 11:47:26 localhost sudo: dcid-user : TTY=pts/2 ; PWD=/home/dcid ; USER=root ; COMMAND=/bin/dir
2- SSL traffic
Most NIDS (Network-based Intrusion detection systems) can not look at SSL encrypted traffic, but we use SSL everywhere now. How to monitor this traffic? Using log analysis you can see exactly what requests the client made and if they were successful or not. I always run Snort (with sguil) on my network and it misses some attacks that ossec using log analysis alerts me. The following one is a good example:
Received From: (wserver) 192.168.2.0->/var/log/apache-ssl/ssl.error_log
Rule: 30117 fired (level 10) -> “Invalid URI, file name too long.”
Portion of the log(s):
[Tue Jan 9 22:16:07 2007] [error] [client 142.162.1.79] request failed: URI too long
And I have numerous others cases where logging was the only way to know what happened (e.g sql injection or other web application attacks against SSL servers).
A few more examples are available in a paper that I wrote a while back ago: log analysis for intrusion detection.
To conclude, Richard also pointed in the past: “I guarantee I could determine if the system was compromised, and by how many parties, faster using NSM techniques than manual log analysis”. I completely agree with him. Manual log analysis is very hard to do and provide little value for the time spent… This is why I wrote OSSEC: to provide an automatized log analysis tool that is easy to use and extend, providing the alerts in a useful way that adds a lot to a security team. Together with NSM, log analysis can detect a compromised system much faster than without it…