Unrelated to this disclosure, I have acquired quite a lot of Fail2Ban attack reports generated by hosts around the world. I put together an initial set of charts showing attacks by country; I was wondering if anyone had any ideas for interesting things that should be done to analyze/visualize the data.
The data set contains the IP addresses for attacker and victim, the date, and the service name for each of a couple million attack reports.
«DShield is a community-based collaborative firewall log correlation system. It receives logs from volunteers world wide and uses them to analyze attack trends.»
Note that Debian release do not install sudo by default, unlike Ubuntu.
So if you have not installed sudo, or do not have permission to use it, then you should use "su - " to become root, then run "apt-get update;apt-get upgrade".
Fail2ban is fundamentally a wrong answer to the problem. If you're taking the time to install such things, you should instead either be turning off password authentication (relying only on keys) or shorewall (to default deny ssh access except to authorized subnets) or both.
Waiting for multiple authentication failures to mark a host/net as "bad" is fundamentally a bad idea. Stop using passwords.
I've used Fail2Ban even with SSH password authentication turned off. It's still helpful for preventing huge error logs full of pointless password guesses.
It's also useful for applications where you have to use passwords or where you need to allow anonymous requests but you don't want attackers using up all of your resources. VoIP (public-facing and handset-facing SIP) servers are my example of both of those situations.
I'm in no danger of ssh ever being brute'd, but having logs fill up with piles of attempts is annoying and some hosts can be very persistent.
In my case, I also use it to prevent other abuse: bans for ssh abuse protect other hosted services, and fail2ban has been modified to act on other forms of abuse too (like spam).
I agree even though I'm not sure the problem is on the "using passwords" side. There are more fundamental problems with fail2ban & Co.
Using regular expressions on logs and triggering such massive actions like IP-based bans isn't a wise thing to do. Logs are not the trivial lines of text they were 10 years before. In reality you'd have to deal with chaotic stuff often even including user-entered data - this is why there are so many classical Big-Data programs and algorithms are joining the party here. It's very hard to write solid regular expressions on those without overseeing a tiny DoS possibility because of injection. There are numerous examples in the CVE database for exactly this when you look at fail2ban and other programs using the same scheme...
If you really need to keep your logs clean and there is no way in detecting "unusual behaviour" by filtering out all those background noise you see on SSH usually, why not use iptables with it's extensions instead? You can easily count the attempts to connect to defined ports and block any IP address reaching a threshold for a limited amount of time. This is a way more solid implementation of the same thing as IP packages are well-defined structures and there is simply no way in injecting something in the IP source field of them.
About the hardware consumption by SSH login attempts: you guys aren't honestly arguing that a cronjob-like grep of log-files and performing regular expressions on hundreds of lines is a way better to do the same, right? This is actually highly inefficient too...
For me, even plain iptables filtering isn't a nice thing. I use port-knocking to get access to my SSH services. Every server has it's unique sequence of packages to tell it to open access to SSH for a IP transmitting the right packages with the right flags and payload in the correct ordering.
I'm not saying "get rid of fail2ban", but people should know what they are doing and what problems come with those programs. The convenience has it's price...
I also use log monitoring and banning to ban http connections to my web server (driven by scanning apache logs), as well as SMTP connections (scanning Exim's rejectlog).
The rationale "do not use password authentication" does not apply to unauthenticated HTTP and SMTP requests.
You can do both and use denyhosts instead of fail2ban. That one doesn't even require multiple authentication failures. Known-bad hosts are blocked because enough other people blocked them: http://stats.denyhosts.net/stats.html
How do you manage ssh keys once deployed? From personal experience using both systems, my thinking is that once you start deploying ssh keys, in a larger user/server environment, you have a massive rats nest. There is little to no life cycle management.
Fail2ban works and should be deployed regardless of whether you are using usernames/passwords or keys. I just checked one of our bastion hosts, and there are 384 banned IPs (on a rolling 30 day basis). This info is useful, and can be fed into other systems, like edge network devices.
It is possible to use brute-force via timing attacks to determine whether accounts exists on hosts, and then either focus different attacks on those accounts or spam them.
It is amazing how much this needs to be repeated, but security is not an absolute. Fail2ban adds another layer of security, thus it is a good thing.
that's assuming your use case for fail2ban is only blocking failed ssh login attempts. it's pretty handy for blocking malware scanners, x number of failed logins to wordpress/drupal/etc, and all that jazz.
I'm not sure why this comment is getting downvoted, especially since the users who replied to this comment correctly understand that Fail2Ban is a log prettifier and use it as intended.
[+] [-] knyt|11 years ago|reply
The data set contains the IP addresses for attacker and victim, the date, and the service name for each of a couple million attack reports.
edit: https://int80k.com/ftb/
[+] [-] gioele|11 years ago|reply
«DShield is a community-based collaborative firewall log correlation system. It receives logs from volunteers world wide and uses them to analyze attack trends.»
[+] [-] thaumaturgy|11 years ago|reply
Plotting your data on a world map would be kinda neat. D3.js is a good tool for that. Animate it over time for even more bonus points. :-)
Or, scrub your information from the data and post it for others to mess with.
I have several thousand entries for ssh, ssh-root, and spam abuse in my badhosts table.
[+] [-] castorio|11 years ago|reply
check the Country-Page
[+] [-] ForHackernews|11 years ago|reply
[+] [-] NickSharp|11 years ago|reply
sudo apt-get update
sudo apt-get install --only-upgrade fail2ban
Although it's considered best practice to upgrade all packages, like this:
sudo apt-get update
sudo apt-get upgrade
[+] [-] stevekemp|11 years ago|reply
So if you have not installed sudo, or do not have permission to use it, then you should use "su - " to become root, then run "apt-get update;apt-get upgrade".
[+] [-] daviddede|11 years ago|reply
http://dcid.me/texts/attacking-log-analysis-tools.html
It had a similar vuln many years ago.
[+] [-] kolev|11 years ago|reply
[+] [-] TimWolla|11 years ago|reply
[+] [-] stonogo|11 years ago|reply
[+] [-] whiteagle|11 years ago|reply
[+] [-] sneak|11 years ago|reply
Waiting for multiple authentication failures to mark a host/net as "bad" is fundamentally a bad idea. Stop using passwords.
[+] [-] knyt|11 years ago|reply
It's also useful for applications where you have to use passwords or where you need to allow anonymous requests but you don't want attackers using up all of your resources. VoIP (public-facing and handset-facing SIP) servers are my example of both of those situations.
[+] [-] thaumaturgy|11 years ago|reply
In my case, I also use it to prevent other abuse: bans for ssh abuse protect other hosted services, and fail2ban has been modified to act on other forms of abuse too (like spam).
[+] [-] ckoepp|11 years ago|reply
Using regular expressions on logs and triggering such massive actions like IP-based bans isn't a wise thing to do. Logs are not the trivial lines of text they were 10 years before. In reality you'd have to deal with chaotic stuff often even including user-entered data - this is why there are so many classical Big-Data programs and algorithms are joining the party here. It's very hard to write solid regular expressions on those without overseeing a tiny DoS possibility because of injection. There are numerous examples in the CVE database for exactly this when you look at fail2ban and other programs using the same scheme...
If you really need to keep your logs clean and there is no way in detecting "unusual behaviour" by filtering out all those background noise you see on SSH usually, why not use iptables with it's extensions instead? You can easily count the attempts to connect to defined ports and block any IP address reaching a threshold for a limited amount of time. This is a way more solid implementation of the same thing as IP packages are well-defined structures and there is simply no way in injecting something in the IP source field of them.
About the hardware consumption by SSH login attempts: you guys aren't honestly arguing that a cronjob-like grep of log-files and performing regular expressions on hundreds of lines is a way better to do the same, right? This is actually highly inefficient too...
For me, even plain iptables filtering isn't a nice thing. I use port-knocking to get access to my SSH services. Every server has it's unique sequence of packages to tell it to open access to SSH for a IP transmitting the right packages with the right flags and payload in the correct ordering.
I'm not saying "get rid of fail2ban", but people should know what they are doing and what problems come with those programs. The convenience has it's price...
[+] [-] kazinator|11 years ago|reply
tools like fail2ban are not only for SSH!
I also use log monitoring and banning to ban http connections to my web server (driven by scanning apache logs), as well as SMTP connections (scanning Exim's rejectlog).
The rationale "do not use password authentication" does not apply to unauthenticated HTTP and SMTP requests.
[+] [-] viraptor|11 years ago|reply
[+] [-] WestCoastJustin|11 years ago|reply
Fail2ban works and should be deployed regardless of whether you are using usernames/passwords or keys. I just checked one of our bastion hosts, and there are 384 banned IPs (on a rolling 30 day basis). This info is useful, and can be fed into other systems, like edge network devices.
There is actually a good article about this @ http://www.networkworld.com/article/2164048/tech-primers/ssh...
[+] [-] zorbo|11 years ago|reply
It is amazing how much this needs to be repeated, but security is not an absolute. Fail2ban adds another layer of security, thus it is a good thing.
[+] [-] stonogo|11 years ago|reply
[+] [-] thomaslutz|11 years ago|reply
[+] [-] atmosx|11 years ago|reply
[+] [-] houk|11 years ago|reply
[+] [-] kbar13|11 years ago|reply