Sharkey: a service for managing certificates for use by OpenSSH(github.com)
github.com
Sharkey: a service for managing certificates for use by OpenSSH
https://github.com/square/sharkey
6 comments
It seems like you are reinventing the wheel here.
Did you look at RFC4255?
https://tools.ietf.org/html/rfc4255
Did you look at RFC4255?
https://tools.ietf.org/html/rfc4255
OpenSSH already supports certificates. This is a set of tools for issuing and managing those certificates.
We could use RFC4255 instead, but we'd still have the same problems around issuing them, and we could set up sharkey to support it if we wanted to.
We could use RFC4255 instead, but we'd still have the same problems around issuing them, and we could set up sharkey to support it if we wanted to.
That only works if you have control over the DNS and are using it.
DNS based protocols are quite flaky not every server/provider allows you to store text records and some even limit the length of the A/AAAA record.
It also becomes more complicated when you are running multiple ssh services on a single host or using proxies/forwarders.
I use a similar method to secure my own ssh services before that I used a tool that converts the fingerprint into a phrase but it wasn't collision resistant enough.
It also becomes more complicated when you are running multiple ssh services on a single host or using proxies/forwarders.
I use a similar method to secure my own ssh services before that I used a tool that converts the fingerprint into a phrase but it wasn't collision resistant enough.
Nice work - this solves a common problem.
You might consider adding a recommended ssh_config to ensure that host names are canonicalised. This allows FQDNs in certificates to match successfully when users only supply a hostname. More details at http://blog.djm.net.au/2014/01/hostname-canonicalisation-in-...
You might consider adding a recommended ssh_config to ensure that host names are canonicalised. This allows FQDNs in certificates to match successfully when users only supply a hostname. More details at http://blog.djm.net.au/2014/01/hostname-canonicalisation-in-...
Adding recommended ssh configs is a good idea. I'll file a ticket.
This is interesting. I wrote a service for issuing user ssh certs but it doesn't manage host certs.
https://github.com/nsheridan/cashier
https://github.com/nsheridan/cashier
This looks very interesting, but I'm not quite sure I see what it really adds to the mix. This is a client/server service, that bootstraps ssh host certificates, from long-lived TLS x509 client certs for the SSH servers?
And users still log in using bare keys, not certificates? (Ie: no real key revocation, expiry for user keys?)
What is the use-case here, that isn't better served by simply automating the signing of ssh server certs (and user certs)? Is it moving the authorization from install/key-gen time to run-time, and from the individual ssh servers to the central "ssh authorization service"?
I was initially hoping this is what I've been mulling over making for myself (a tool to make it easier to maintain ssh user and server certificates, along with an issuer log and revocation list (I take it the life time of these server certs are set so that a revocation list is only needed for the TLS certificates involved) -- but it seems like it might be a bit of a "horizontal abstraction" -- shifting work around without really simplifying the work flow all that much.
I'm sorry to come off negative, I love to see tools that help leverage the possibilities that's already within ssh to make something better than "TOFU" and a mess of expired, forgotten, compromised and lost public keys in "authorized_keys" -- but I really think the only sane way to do that is to use full ssh certs for both hosts and users... And for that to scale, one would need some scripts/a "ssh CA" utility or something.
I suppose this project might be useful if one already has a TLS/x509 CA in-house that is used for authentication and authorization -- and one doesn't want to patch ssh to use x509 certs directly?
And users still log in using bare keys, not certificates? (Ie: no real key revocation, expiry for user keys?)
What is the use-case here, that isn't better served by simply automating the signing of ssh server certs (and user certs)? Is it moving the authorization from install/key-gen time to run-time, and from the individual ssh servers to the central "ssh authorization service"?
I was initially hoping this is what I've been mulling over making for myself (a tool to make it easier to maintain ssh user and server certificates, along with an issuer log and revocation list (I take it the life time of these server certs are set so that a revocation list is only needed for the TLS certificates involved) -- but it seems like it might be a bit of a "horizontal abstraction" -- shifting work around without really simplifying the work flow all that much.
I'm sorry to come off negative, I love to see tools that help leverage the possibilities that's already within ssh to make something better than "TOFU" and a mess of expired, forgotten, compromised and lost public keys in "authorized_keys" -- but I really think the only sane way to do that is to use full ssh certs for both hosts and users... And for that to scale, one would need some scripts/a "ssh CA" utility or something.
I suppose this project might be useful if one already has a TLS/x509 CA in-house that is used for authentication and authorization -- and one doesn't want to patch ssh to use x509 certs directly?
This doesn't deal with user keys at all -- we use Kerberos for users to log in. This is just an SSH CA for server certificates. We could extend it to users, if we had a way to identify them.
We have an existing x509 trust infrastructure, where keys are distributed with Keywhiz (https://github.com/square/keywhiz). That's why we started with having hosts identify themselves with x509 client certs. We will deploy sharkey-client to all hosts, and it's automatically issued an x509 client cert.
The primary purpose of this service is to allow us to move to short-lived SSH Host certificates. Instead of issuing a single permanent certificate at host imaging time, we have a client running on every server that's fetching fresh host certificates all the time.
When you say "simply automating the signing of ssh server certs", what differences do you mean from this? Because that's what this is -- automating the signing of ssh server certs.
Your last sentence is pretty much spot on: We use x509 everywhere, but want to use stock SSH. We also didn't use our existing secrets distribution because we wanted to decouple SSH from the rest of our infrastructure
We have an existing x509 trust infrastructure, where keys are distributed with Keywhiz (https://github.com/square/keywhiz). That's why we started with having hosts identify themselves with x509 client certs. We will deploy sharkey-client to all hosts, and it's automatically issued an x509 client cert.
The primary purpose of this service is to allow us to move to short-lived SSH Host certificates. Instead of issuing a single permanent certificate at host imaging time, we have a client running on every server that's fetching fresh host certificates all the time.
When you say "simply automating the signing of ssh server certs", what differences do you mean from this? Because that's what this is -- automating the signing of ssh server certs.
Your last sentence is pretty much spot on: We use x509 everywhere, but want to use stock SSH. We also didn't use our existing secrets distribution because we wanted to decouple SSH from the rest of our infrastructure
Ok, thank you both. Makes a lot of sense. It's not what I need, but that's fine :-)
I've not really looked hard at (open)ssh and kerberos before - as far as I can tell, (open)ssh doesn't support kerberos fully - that is - it's not possible to simply use kerberos to authenticate both client and server to each other?
> When you say "simply automating the signing of ssh server certs", what differences do you mean from this? Because that's what this is -- automating the signing of ssh server certs.
I suppose the fact that you already have an x509 (and kerberos) infrastructure was one thing I didn't "expect" - ideally either one of those should be enough to identify server/services and users to each other -- I absolutely see the sense in leveraging that to automate ssh server cert generation - but it strikes me as a roundabout/complex way of solving the trust issue. Then again, composing a few simple tools might be better than trying to secure/get something like https://tools.ietf.org/html/rfc6187 to work...
In essence this places trust in the x509 infrastructure - I'm always a bit skeptical of schemes (most of which I must admit I've thought up myself at one time or another) that end up being "ad-hoc kerberos with public keys". They all feel straightforward on paper, but then I remember how many issues there were with getting kerberos to the place it is today, that is to say create an actually secure system...
I suppose strictly speaking, this is automating the signing of server certs - continuously. Which is a bit more than I meant - I was more precisely thinking of automating the setup of ssh server certificates, I guess. Perhaps with the possibility to push updates from time to time.
Anyway, maybe a sentence at the top of the readme along the lines of "Automating short-lived ssh server certificates on top of existing x509 infrastructure" would've made things more obvious.
Now that I have a handle on what this is, and the context - I suppose a similar infrastructure might be used for generating client certificates too. Perhaps semi-manually via a web-service that uses x509 client certificates for authentication, and perhaps ldap as a backend for authorization data.
It does appear that it's difficult to set up a fleet of ssh server so that there's no need to do a dynamic look-up to something like ldap, and still have sensible limits on who gets to log in to which server. I suppose one could use several CAs for signing user certs - one per group of servers - but that sounds a bit messy.
I suppose the other alternative is to do something with AuthorizedPrincipalsCommand[1] - but that also feels a bit messy...
For others contemplating playing with plain ssh certificates, I came across this, which is about as good a guide as I've read: https://ef.gy/hardening-ssh
[1] http://man.openbsd.org/sshd_config
I've not really looked hard at (open)ssh and kerberos before - as far as I can tell, (open)ssh doesn't support kerberos fully - that is - it's not possible to simply use kerberos to authenticate both client and server to each other?
> When you say "simply automating the signing of ssh server certs", what differences do you mean from this? Because that's what this is -- automating the signing of ssh server certs.
I suppose the fact that you already have an x509 (and kerberos) infrastructure was one thing I didn't "expect" - ideally either one of those should be enough to identify server/services and users to each other -- I absolutely see the sense in leveraging that to automate ssh server cert generation - but it strikes me as a roundabout/complex way of solving the trust issue. Then again, composing a few simple tools might be better than trying to secure/get something like https://tools.ietf.org/html/rfc6187 to work...
In essence this places trust in the x509 infrastructure - I'm always a bit skeptical of schemes (most of which I must admit I've thought up myself at one time or another) that end up being "ad-hoc kerberos with public keys". They all feel straightforward on paper, but then I remember how many issues there were with getting kerberos to the place it is today, that is to say create an actually secure system...
I suppose strictly speaking, this is automating the signing of server certs - continuously. Which is a bit more than I meant - I was more precisely thinking of automating the setup of ssh server certificates, I guess. Perhaps with the possibility to push updates from time to time.
Anyway, maybe a sentence at the top of the readme along the lines of "Automating short-lived ssh server certificates on top of existing x509 infrastructure" would've made things more obvious.
Now that I have a handle on what this is, and the context - I suppose a similar infrastructure might be used for generating client certificates too. Perhaps semi-manually via a web-service that uses x509 client certificates for authentication, and perhaps ldap as a backend for authorization data.
It does appear that it's difficult to set up a fleet of ssh server so that there's no need to do a dynamic look-up to something like ldap, and still have sensible limits on who gets to log in to which server. I suppose one could use several CAs for signing user certs - one per group of servers - but that sounds a bit messy.
I suppose the other alternative is to do something with AuthorizedPrincipalsCommand[1] - but that also feels a bit messy...
For others contemplating playing with plain ssh certificates, I came across this, which is about as good a guide as I've read: https://ef.gy/hardening-ssh
[1] http://man.openbsd.org/sshd_config
What is the use-case here, that isn't better served by simply automating the signing of ssh server certs (and user certs)?
This is exactly what this tool does. We leverage our existing PKI to request the signed certificates. mcpherrinm is planning to publish the design doc, it might help clarify things.
[...]shifting work around without really simplifying the work flow all that much.
Sharkey currently solves one problem and does it well: removing the trust on first use prompt. When you have thousands of servers, you run into the prompt on a daily basis. In addition, some virtual machines get re-imaged on a regular basis. As a result, you train engineers to ignore the TOFU prompt and also ignore the fingerprint changed warning in some instances.
The certs can be short lived. It's not strictly equivalent to a revocation system, but in practice it seems short lived certs work well.
[...]expired, forgotten, compromised and lost public keys in "authorized_keys"
Sharkey does not address any of these. We already have automation around our authorized_key management.
This is exactly what this tool does. We leverage our existing PKI to request the signed certificates. mcpherrinm is planning to publish the design doc, it might help clarify things.
[...]shifting work around without really simplifying the work flow all that much.
Sharkey currently solves one problem and does it well: removing the trust on first use prompt. When you have thousands of servers, you run into the prompt on a daily basis. In addition, some virtual machines get re-imaged on a regular basis. As a result, you train engineers to ignore the TOFU prompt and also ignore the fingerprint changed warning in some instances.
The certs can be short lived. It's not strictly equivalent to a revocation system, but in practice it seems short lived certs work well.
[...]expired, forgotten, compromised and lost public keys in "authorized_keys"
Sharkey does not address any of these. We already have automation around our authorized_key management.
What would cause someone to use this instead of deploying a known-hosts file with configuration management tools?
Using a CA means all users instantly trust a host, without needing to update their known hosts file.
If you are frequently spinning up or reimaging hosts, you don't have to worry about constant updates to known hosts files everywhere.
If you are frequently spinning up or reimaging hosts, you don't have to worry about constant updates to known hosts files everywhere.
you can put host keys in DNS, which can/should be secured. doesn't that avoid the need for this new infrastructure?
We answered this question in a sibling thread, see: https://news.ycombinator.com/item?id=12098202
In the enterprise world, engineers often ssh into hosts for the first time on a daily basis. This trains them to acknowledge the trust on first use prompt on a daily basis.
Sharkey removes the prompt by signing the openssh certificates.
Feel free to post questions here and I'll relay them.