SSH Brute Force – The 10 Year Old Attack That Still Persists(blog.sucuri.net)
blog.sucuri.net
SSH Brute Force – The 10 Year Old Attack That Still Persists
http://blog.sucuri.net/2013/07/ssh-brute-force-the-10-year-old-attack-that-still-persists.html
65 comments
Yes, SSH keys is the recommended option. The post has been updated there to state that.
thanks,
thanks,
When you're traveling or just for convenience's sake. You might not always have your key on hand, or may use devices/apps or computers that make it difficult to load one.
If your password is strong enough and isn't reused, most attacks are prevented. You could in theory eyeball a really long complex password but it would be quite difficult. Keys are mostly useful for multiple-device identity management and passwordless login using an agent.
If your password is strong enough and isn't reused, most attacks are prevented. You could in theory eyeball a really long complex password but it would be quite difficult. Keys are mostly useful for multiple-device identity management and passwordless login using an agent.
>Using keys is ever-so-slightly less convenient at setup
And it's easier once it's setup. I don't have punch in a password any time I need to make a server change.
And it's easier once it's setup. I don't have punch in a password any time I need to make a server change.
More importantly, it is far easier to revoke on a person-to-person basis even on the same account.
If you use password auth and some accounts have a password that is shared knowledge (root, anyone?), you have to change all passwords once one persons permissions change.
In practice, revoking public keys is a problem for many companies already as I found out when my private key was stolen a while ago. (on an encrypted drive, but paranoia is paranoia)
If you use password auth and some accounts have a password that is shared knowledge (root, anyone?), you have to change all passwords once one persons permissions change.
In practice, revoking public keys is a problem for many companies already as I found out when my private key was stolen a while ago. (on an encrypted drive, but paranoia is paranoia)
Don't forget that makes you more vulnerable to malware on the client, which now has easy access to your keys, and your known_hosts file (which gives the attacker a convenient list of servers to attempt to log into using those keys.)
Yes, this is why having a passphrase on your key is recommended. The idea is not to use keys instead of passwords, but in addition to: the password protects the private key locally, which is used to authenticate to the server. However that seems to be uncommon practice.
Still, any password that isn't very weak will do just fine, especially if root can't log in directly.
There are lots of good guides online on how to protect yourself from this (disable password-based auth, or use 2-factor auth, Fail2ban etc.), but there's one piece of advice that crops up a lot that is probably a bad idea, and that's changing the SSH port. This can give (at best) a false sense of security and (at worst) actually reduce security if you choose a port number above 1024, because then malware could pose as SSH without requiring root access and then steal your password once you've entered it.
Just thought it was worth mentioning because I recently went through all this myself to try and secure a Raspberry Pi and it was news to me...
Just thought it was worth mentioning because I recently went through all this myself to try and secure a Raspberry Pi and it was news to me...
I agree that changing the port is not a great security measure in theory and I think it should be last of a large number of things a person does to secure SSH.
BUT- I worked at a company a while back that had fail2ban notify us when by E-mail when it banned people. Changing the SSH ports dropped the bans per 6 month from 80-60 to 1-2. So while its certainly not the only thing you should do, there is something to be said for obscurity (in addition to existing real security of course).
BUT- I worked at a company a while back that had fail2ban notify us when by E-mail when it banned people. Changing the SSH ports dropped the bans per 6 month from 80-60 to 1-2. So while its certainly not the only thing you should do, there is something to be said for obscurity (in addition to existing real security of course).
Not to mention the hardware resources. Prior to changing our SSH port, the fail2ban process was #3-5 in terms of CPU time on many of our boxes. Changing the SSH port puts it out of contention for even the top 10.
If you must then just limit connections per minute that can be opened per address to the sshd port in the host firewall instead. Sshd should be configured to close the connection after a number of failed logins. Now you get the same effect without a daemon parsing logfiles.
I recently added UFW configuration to my provisioning scripts, so we now use UFW's 'limit' rules to do that on the port we use for SSH. We could probably get away with moving it back to the default with this in place.
I still think that it's a good idea to change the default port for sshd, even if you don't let root to login and use public key auth and fail2ban. Why? Because this "obscurity" helps keeping false possitives to a reasonable low number, so you can actually see if there's a tageted attack or not.
If I see a auth failure "root"-"123456" I instantly know that's not a targeted attack, it will unnecessary fill my log files that will add with time and become a burden to audit my systems, which at some point I either don't do it thoroughly or any at all.
SELinux takes care of controling which application could open outbound ports, so if your box is properly configured, there are other ways to reduce the impact if the box is exploited
If I see a auth failure "root"-"123456" I instantly know that's not a targeted attack, it will unnecessary fill my log files that will add with time and become a burden to audit my systems, which at some point I either don't do it thoroughly or any at all.
SELinux takes care of controling which application could open outbound ports, so if your box is properly configured, there are other ways to reduce the impact if the box is exploited
do (can?) auth failures display passwords? if so, is that a good idea? people (ie me) sometimes enter the wrong, but valid password. having that logged somewhere seems like a bad idea.
The standard sshd does not log failed passwords. A few years ago I was seeing a lot of brute force attempts and I was curious to see the passwords. The only way I could do it was to edit the source code of sshd to add my own logging line and recompile the whole thing.
If you use public-key it only shows the ip. I don't know if you use password, 2-factor auth or Kerberos it shows any additional information.
I would guess that if you enable debugging it will show some identifiable information, I never had to debug it.
I would guess that if you enable debugging it will show some identifiable information, I never had to debug it.
steal your password
i had always assumed that the protocol somehow protected your password, but reading the spec at http://www.ietf.org/rfc/rfc4252.txt that's not the case (and i've pretty much convinced myself it's not possible to do better).
learn something new every day...
[apart from the reference above, someone else here claims to be logging these things.]
i had always assumed that the protocol somehow protected your password, but reading the spec at http://www.ietf.org/rfc/rfc4252.txt that's not the case (and i've pretty much convinced myself it's not possible to do better).
learn something new every day...
[apart from the reference above, someone else here claims to be logging these things.]
There exist ways for a client and server to convince each other they share a secret (such as a password) without giving an adversary enough information to infer the secret. Here is one such: http://en.wikipedia.org/wiki/Secure_Remote_Password_protocol
does that work when it's a system password that exists in a hashed (scrypt) database? it seems like it would leave things open to someone stealing the hashed system password.
if this can be made to work for ssh when using system passwords that would be very cool. but to my (inexpert) eye it seems unlikely (without relying on a trusted client). do you have a description for that case?
the problem, in short, is that the password is not shared. the server does not know it - it only knows the hash. and the hash is not sufficient as a shared secret.
if this can be made to work for ssh when using system passwords that would be very cool. but to my (inexpert) eye it seems unlikely (without relying on a trusted client). do you have a description for that case?
the problem, in short, is that the password is not shared. the server does not know it - it only knows the hash. and the hash is not sufficient as a shared secret.
I'm not an expert, but your question has interested me for quite a while as well, so I did some research. It seems that SRP does not require the plaintext password to be stored by the server.
> SRP does not expose passwords to either passive or active network intruders, and it stores passwords as a "non-plaintext-equivalent" one-way hash on the server.
Source: http://srp.stanford.edu/analysis.html
> SRP does not expose passwords to either passive or active network intruders, and it stores passwords as a "non-plaintext-equivalent" one-way hash on the server.
Source: http://srp.stanford.edu/analysis.html
interesting, thanks. turns out there are even patches for ssh! http://srp.stanford.edu/links.html
but i get the impression that still doesn't mean that ssh can be made to work with system passwords without transmitting them (over an encrypted channel). i don't understand why - maybe srp requires a particular kind of hash?
ah yes - this answer http://security.stackexchange.com/questions/23821/srp-can-th... gives some details.
but i get the impression that still doesn't mean that ssh can be made to work with system passwords without transmitting them (over an encrypted channel). i don't understand why - maybe srp requires a particular kind of hash?
ah yes - this answer http://security.stackexchange.com/questions/23821/srp-can-th... gives some details.
Even if it's not possible to improve the security of a single-server system, there is definite room for improvement in the real world.
Assume for a moment that you have to transmit the password to the server instead of using proof of knowledge algorithms. This makes the server a potential point for stealing the password. This is of negligible risk in theory, because the server has access to itself by definition. But in the real world people reuse passwords. So in the real world you can improve security by hashing what the user types with server-specific data (public key?) and using that as the server-specific password. Then someone that can capture the password can only log on to that specific server.
Assume for a moment that you have to transmit the password to the server instead of using proof of knowledge algorithms. This makes the server a potential point for stealing the password. This is of negligible risk in theory, because the server has access to itself by definition. But in the real world people reuse passwords. So in the real world you can improve security by hashing what the user types with server-specific data (public key?) and using that as the server-specific password. Then someone that can capture the password can only log on to that specific server.
With "password" authentication method, the password is not transmitted to the remote server. (AFAIK)
However, all modern sshds (AFAIK, at least OpenSSH) default to (publickey, keyboard-interactive). As the name says, keyboard-interactive sends your keystrokes, one at a time (bar buffering) to the remote server.
This is required, for example, when using PAM, as PAM requires the actual authentication key.
However, all modern sshds (AFAIK, at least OpenSSH) default to (publickey, keyboard-interactive). As the name says, keyboard-interactive sends your keystrokes, one at a time (bar buffering) to the remote server.
This is required, for example, when using PAM, as PAM requires the actual authentication key.
[deleted]
about password auth - you're wrong (AFAIK). i even gave a reference in my original post. didn't you bother to check it?
I am wrong. No.
Huh. I hadn't actually thought that the protocol itself could protect you (I only left my comment based on what I'd read elsewhere; I don't know anything about it otherwise).
So hypothetically speaking, if the protocol did protect your password from being stolen, what would be the actual threat of a piece of malware specifically pretending to be SSH?
So hypothetically speaking, if the protocol did protect your password from being stolen, what would be the actual threat of a piece of malware specifically pretending to be SSH?
If you use "sudo foo" in the remote shell, and then type your password, it could steal that. Or even if you somehow obfuscated your sudo password, it could change the command presented to sudo, so after you type your password, the new substituted command would be executed. Exaple: you type "sudo less /var/log/messages" but the fake-ssh tells the remote shell that you really typed "sudo email-me-all-your-data". Then the real sudo authenticates you, and executes the command you didn't type.
Changing the port is still a worthwhile thing to do in many cases.
Military tanks aren't painted green because green paint is more bulletproof, they use green because hot pink tanks get shot at more.
Changing the port isn't armor, it's camouflage. Camouflage is still a useful thing.
Your point about not using a port north of 1024 is solid advice.
Military tanks aren't painted green because green paint is more bulletproof, they use green because hot pink tanks get shot at more.
Changing the port isn't armor, it's camouflage. Camouflage is still a useful thing.
Your point about not using a port north of 1024 is solid advice.
Changing the port number can boost security if it makes it easier to scan the log for attempts.
I think it is a good idea.
It is the expectation that this is enough that is the bad idea.
I think it is a good idea.
It is the expectation that this is enough that is the bad idea.
[deleted]
Additionally you could also use something like fwknop[1]. It opens the ssh port for your ip when you need it and closes it after a preset time has passed.
[1]http://www.cipherdyne.org/fwknop/docs/faq.html#fwknop
[1]http://www.cipherdyne.org/fwknop/docs/faq.html#fwknop
> above port 1024 malware...
Im using this https://github.com/stealth/sshttp to run it on port 443.
The other port it actually runs on is firewalled off from outside.
Im using this https://github.com/stealth/sshttp to run it on port 443.
The other port it actually runs on is firewalled off from outside.
What sane person would allow password authentication on their servers? Turn it off.
I am very surprised that the OP, and nobody in this thread, has mentioned port knocking.
You are absolutely better off / more secure, with port knocking enabled. The scans never touch your sshd, because your server does not even answer port 22. As far as the outside world is concerned, you don't run an sshd.
On my own systems, I set up port knocking, I delete the allow rule for the IPs I knocked from every 24 hours (so fresh knocking is needed every day, and I don't leave a trail of "open" IPs as I travel the world) and my .login script spits back at me the current days list of "knocking IPs" so I can immediately note if someone else is knocking.
You are absolutely better off / more secure, with port knocking enabled. The scans never touch your sshd, because your server does not even answer port 22. As far as the outside world is concerned, you don't run an sshd.
On my own systems, I set up port knocking, I delete the allow rule for the IPs I knocked from every 24 hours (so fresh knocking is needed every day, and I don't leave a trail of "open" IPs as I travel the world) and my .login script spits back at me the current days list of "knocking IPs" so I can immediately note if someone else is knocking.
The problem is that it's harder to setup and is less convenient to use. Can you point to any guides that make it simple to set up port knocking?
Would it fairly secure to setup a VPN service, connect via VPN, and then SSH in over the VPN connection?
Would it fairly secure to setup a VPN service, connect via VPN, and then SSH in over the VPN connection?
Its easier to go the VPN route than port knocking, but the vpn then ssh is too tinfoil hat for even my taste.
As someone else mentioned, just run it on another port than 22, like 53 or 109, and youll get 99% less brute forcing attempts, and dont use passwords as those in the list, and disable user root to authenticate at all.
If you're still paranoid about the minor brute forcing attempts left then add the iptables rule to limit connection attempts to 1/min. You're still paranoid? For what really? Port knocking wont help out at this level of paranoia. Hrm, security awareness. Its just not worth the hassle.
If you already have OpenVPN setup, then its easy to put the ssh to listen only on the tun interface the vpn server has opened.
As someone else mentioned, just run it on another port than 22, like 53 or 109, and youll get 99% less brute forcing attempts, and dont use passwords as those in the list, and disable user root to authenticate at all.
If you're still paranoid about the minor brute forcing attempts left then add the iptables rule to limit connection attempts to 1/min. You're still paranoid? For what really? Port knocking wont help out at this level of paranoia. Hrm, security awareness. Its just not worth the hassle.
If you already have OpenVPN setup, then its easy to put the ssh to listen only on the tun interface the vpn server has opened.
Port knocking: sending a stream of special packets to verify that the user is trusted.
Sounds familiar.
Sounds familiar.
I usually just limit SSH connections to one of my own static IP addresses, as well as disable password authentication and implement PSK.
At first I was shocked by the amount of intrusion attempts my servers were logging, but as time goes on I see that no attempts have yet been successful, so these sorts of intrusion attempts have probably made me complacent. (never a good thing)
That said, I found the article to be lacking in listing mitigation alternatives, such as those posted here in the comments.
At first I was shocked by the amount of intrusion attempts my servers were logging, but as time goes on I see that no attempts have yet been successful, so these sorts of intrusion attempts have probably made me complacent. (never a good thing)
That said, I found the article to be lacking in listing mitigation alternatives, such as those posted here in the comments.
It's amazing how quickly the brute force attempts start rolling in if you turn on SSH with password authentication. Last year I had it on for a bit on my home computer and logged thousands of attempts with the username 'initech'. Thank god I went to work for Intertrode! http://i.imgur.com/6lqXa.png
seriously, password-based auth... have ppl not heard of public key authentication?
allowing password-based auth in sshd is plain stupid. _always_ use pubkey auth, it's a 1-line change in /etc/ssh/sshd_config.
allowing password-based auth in sshd is plain stupid. _always_ use pubkey auth, it's a 1-line change in /etc/ssh/sshd_config.
Can they really bruteforce a 12 characters password? I don't think so...
What happen when you lose your key? How do you backup your key?
What happen when you lose your key? How do you backup your key?
The same way you back up all your other important and private files? That is, frequently and securely.
Also note that you can have multiple keys, potentially one per device (useful in case a device could get stolen).
Also note that you can have multiple keys, potentially one per device (useful in case a device could get stolen).
For most cases, gpg encrypt it and stick it in Dropbox.
If you're super paranoid, copy it to some physical media that you can store securely.
If you're super paranoid, copy it to some physical media that you can store securely.
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh limit: up to 1/min burst 2 mode srcip
(Oh, and PasswordAuthentication is off too.)
What sane person exposes SSH externally?
- OpenVPN takes only minutes to set up
- There are easy to use GUI clients for all major desktops (Windows, Mac OS X, Linux) and phones (Android, iOS)
- It supports running with zero privileges (chroot+setuid) since it only needs to forward packets to /dev/(tun|tap), which does not require privileges after the device has been opened.
- It limits your compromise exposure to only one edge gateway (you can put any number of machines behind a single dedicated gateway). If the gateway is pwned, you still have the defense-in-depth of internal SSH, internal encrypted comms, etc.
- Supports HMAC validation of incoming data to avoid any complex processing (and thus exposure of bugs) of packets from users that don't have the shared HMAC key. This means that only the HMAC code is exposed to untrusted users.
- Supports public key +/ password authentication.
There's no reason to expose SSH to the public internet when you can run a gateway that handles only VPN traffic and greatly limits your attack surface. Defense in depth!
- OpenVPN takes only minutes to set up
- There are easy to use GUI clients for all major desktops (Windows, Mac OS X, Linux) and phones (Android, iOS)
- It supports running with zero privileges (chroot+setuid) since it only needs to forward packets to /dev/(tun|tap), which does not require privileges after the device has been opened.
- It limits your compromise exposure to only one edge gateway (you can put any number of machines behind a single dedicated gateway). If the gateway is pwned, you still have the defense-in-depth of internal SSH, internal encrypted comms, etc.
- Supports HMAC validation of incoming data to avoid any complex processing (and thus exposure of bugs) of packets from users that don't have the shared HMAC key. This means that only the HMAC code is exposed to untrusted users.
- Supports public key +/ password authentication.
There's no reason to expose SSH to the public internet when you can run a gateway that handles only VPN traffic and greatly limits your attack surface. Defense in depth!
You would be surprised. But to say the truth, I trust the OpenSSH code a lot more than any VPN software that you can install to prevent direct access to it.
It's fairly common actually, at least around 12,860,698 devices have their SSH open to the public (http://www.shodanhq.com/?q=port:22).
The difference is that exposing OpenVPN code allows you to separate remote access from your production services, both reducing the total attack surface and providing defense-in-depth.
On top of which, OpenVPN has actually had fewer security vulnerabilities released than OpenSSH, and HMAC validation enormously restricts the surface area of exposed code as compared to OpenSSH.
On top of which, OpenVPN has actually had fewer security vulnerabilities released than OpenSSH, and HMAC validation enormously restricts the surface area of exposed code as compared to OpenSSH.
I don't understand how OpenVPN/PKI is any better than running SSH externally on a different port and strictly using key based auth.
OpenVPN:
- Exposes less code to attack
- Can be run on a single machine, distinct from your production systems, where compromise will only result in a compromise of VPN communications, not of an actual production machine on which your production data/processes exist.
- Exposes less code to attack
- Can be run on a single machine, distinct from your production systems, where compromise will only result in a compromise of VPN communications, not of an actual production machine on which your production data/processes exist.
A few people use spiped[1] in similar fashion.
[1]: http://www.daemonology.net/blog/2012-08-30-protecting-sshd-u...
[1]: http://www.daemonology.net/blog/2012-08-30-protecting-sshd-u...
Can't you just run an ssh gateway and accomplish the last goal, though?
You can (and probably should) setup ssh on one edge gateway and then ssh from that machine to further internal machines. ssh config files make it easy to make multiple steps transparent.
I expose ssh externally on the edge gateway (but can't provide evidence for sanity). The major attractions over a vpn is a limit in what traffic gets sent on the ssh connection - by default just the shell, and whatever port forwards are needed. vpns end up forwarding DNS traffic and a bunch of other gunk. They can also expose the client machines to unintended traffic from the remote end.
I expose ssh externally on the edge gateway (but can't provide evidence for sanity). The major attractions over a vpn is a limit in what traffic gets sent on the ssh connection - by default just the shell, and whatever port forwards are needed. vpns end up forwarding DNS traffic and a bunch of other gunk. They can also expose the client machines to unintended traffic from the remote end.
One thing you should keep in mind when doing this is all traffic will be routed through the gateway, so if you say had a home desktop connected with a VPS running as the OpenVPN server, connections routed to the home desktop will go through the VPS first. This can be a significant bandwidth/latency addition, depending on your needs. Obviously much less significant if everything will be physically colocated.
OpenVPN can be set to route only certain IP blocks to the tunnel. I sometimes do prefer to route everything over the vpn though, especially at coffee shops that filter specific ports.
I have considerably less faith in OpenVPN than you do. The "bastion host" concept is fine, but you can do this with SSH, near-transparently even (using ProxyCommand).
Can you expand on why? OpenVPN has long been an example of solid development (and crypto) practices.
As far as the bastion host, you need an actual VPN to do more than forward SSH connections. For instance, connectivity to your LDAP server, internal web services, etc.
Port forwarding gets you part of the way there, but it's inconvenient to use with SSL, requires using weird local port numbers when connecting to services, etc.
SSH does have VPN support via tun/tap, but it's not nearly as functionally complete as OpenVPN, and you still have the entire SSH authentication path exposed to the world.
As far as the bastion host, you need an actual VPN to do more than forward SSH connections. For instance, connectivity to your LDAP server, internal web services, etc.
Port forwarding gets you part of the way there, but it's inconvenient to use with SSL, requires using weird local port numbers when connecting to services, etc.
SSH does have VPN support via tun/tap, but it's not nearly as functionally complete as OpenVPN, and you still have the entire SSH authentication path exposed to the world.
OpenSSH has long been an example of solid development (and crypto) practices too, and has had a few less issues. Search for "Use constant time memcmp when comparing HMACs in openvpn_decrypt." (the fix is my colleague's, but the bug was mine.)
That said, both are very good, and if you do want a full VPN there's a lot to recommend OpenVPN.
That said, both are very good, and if you do want a full VPN there's a lot to recommend OpenVPN.
Is OpenVPN, both the clients and the server, fully open source? Can we look at each and every line of code?
I went back to the top to see when this was posted...today. I'm no expert on securing a server, but I thought the common thinking now was to just turn off password auth. The post itself says that all it takes is one weak password to be compromised.
Using keys is ever-so-slightly less convenient at setup, but negligibly so. Works on any device I have. Is there a reason one would continue to use passwords?