When the LogWatch report from yesterday (for web.akmai.net) arrived in my Inbox, it had over 20,000 failed SSH login attempts. Today I decided it was finally time to do something about all those attacks.
After looking around a bit, I found several different solutions. Some solutions utilized firewall rules and others monitored your /var/log/secure (or /var/log/auth.log) log files for multiple failed login attempts and then added those IPs/Hosts to the /etc/hosts.deny file.
I decided to go with the latter method and quickly found a nice tutorial for setting up DenyHosts (be sure to download the latest version (2.6 as of this writing) instead of the older version 2.0). Rather than reinvent the wheel, here is what the DenyHosts website says about itself:
What is DenyHosts?
DenyHosts is a Python script that analyzes the sshd server log messages to determine what hosts are attempting to hack into your system. It also determines what user accounts are being targeted. It keeps track of the frequency of attempts from each host.
Additionally, upon discovering a repeated attack host, the /etc/hosts.deny file is updated to prevent future break-in attempts from that host.
An email report can be sent to a system admin.
Since I was setting up DenyHosts on a RedHat-based machine (CentOS) and not a Debian-based machine, I needed to change this line:
update-rc.d denyhosts defaults
to this:
chkconfig denyhosts --add
Other than that, the installation steps were just as the tutorial described. I decided to enable the ADMIN_EMAIL option so that I would receive an email every time something was added to hosts.deny, but within minutes of starting DenyHosts I had a dozen attacks with a dozen emails on my BlackBerry. I had to disable ADMIN_EMAIL to stop the spamming!
To make sure DenyHosts was working properly I tried logging in with the wrong password three times. When I tried to connect again, here is what I received:
ssh [email protected]
ssh_exchange_identification: Connection closed by remote host
DenyHosts also has the ability to report to a central server the hosts that are trying to break in and you can also download a list of hosts that have been reported by others. I choose to opt out of doing this for now. The DenyHosts statistics page is pretty cool. Notice how the majority of the hosts come from China? Hmm.
UPDATE:
I quickly discovered that DenyHosts was adding my IP address to the hosts.deny file. When I watched /var/log/secure I discovered the problem:
Jun 13 20:18:46 web sshd[5959]: reverse mapping checking getaddrinfo for 75-147-49-211-newengland.hfc.comcastbusiness.net failed - POSSIBLE BREAKIN ATTEMPT!
Jun 13 20:18:46 web sshd[5959]: Accepted publickey for fooUser from ::ffff:75.147.49.211 port 57926 ssh2
Jun 13 20:18:48 web sshd[5994]: Did not receive identification string from ::ffff:75.147.49.211
I'm not entirely sure how to fix this, but for now I added my IP address to /usr/share/denyhosts/data/allowed-hosts (I had to create this file) which prevents DenyHosts from blocking my IP no matter what (see this FAQ for more info). Also, I had to restart DenyHosts (/etc/init.d/denyhosts restart) before the change to allowed-hosts took effect.
Raam, you should check fail2ban.
I don’t know how it compares to denyhosts , but it covers any log file you want, and not only the one from ssh.
Thanks for the tip! I hadn’t heard of fail2ban until now. Looks like a great project.
Alright I just wrote a comment for how to fix the eval hack on WordPress…so I’m going to try this one as a hardening tip for my server. Maybe it can help with my other problem. Nice blog dude…btw…fail2ban is a great addition to your server…I wouldn’t install the one you mention above (DenyHosts) and fail2ban together…but the thing with fail2ban was that I found it gave some false positives and was difficult to set up (probably why I had the false positives) – in all fairness, I really didn’t mess around with it that much and had a lot going on at the time…either way, what you write about now seems easier.
Hey Chris,
This is definitely quite a big issue with WordPress websites and I’m not surprised by your frustration. I spent more time dealing with this issue than I’d like to admit. Besides changing passwords, the biggest and most important thing to remember is that the attacker most likely got in via a vulnerable plugin. If you’re reinstalling plugins, you may very well be reinstalling the vulnerable plugin. Try cleaning and reinstalling WordPress with no plugins and see if the problem returns. If so, narrow it down to the next possible thing (another PHP file somewhere on the server that isn’t part of WordPress might be letting them in). In the end, it very well may be a hole on the server itself and your web host will need to address that.
I’m glad you found fail2ban useful. I’ve been loving it every since I installed it. (I agree, DenyHosts and fail2ban together probably isn’t a great idea; I’ve switched entirely to fail2ban.)