Critique My Plan: API Key for Authentication(dev.to)
dev.to
Critique My Plan: API Key for Authentication
https://dev.to/imthedeveloper/critique-my-plan-api-key-for-authentication-26i5
8 comments
Advocate-of-the-deviling re: don't bother encrypting them. Your argument is sound (the reasons why password encryption matters mostly don't apply to API keys), but there is a subtle advantage here: the amount of time it takes you to detect compromise isn't zero. Got a backup on a public S3 bucket? That's bad, but now at least you know you don't have to audit every user action, too. Since they're already high-entropy they can't be enumerated so you don't have to use an expensive KDF like scrypt or bcrypt, and you can get away with just a hash or a regular KDF. (Doesn't hurt to use scrypt though.)
Just to keep myself honest: I'm aware that HMACing with the API key (a suggestion I defended in a different comment) and storing the API key with a KDF mutually exclusive and that might seem like I'm giving contradictory advice. My specific recommendation is still to just use API keys stored plaintext server side just like 'tptacek is. I'm just saying that these alternative suggestions aren't silly.
TL;DR: plaintext API keys are fine and you should use them but you're not a bad person for wanting to hash them :)
Just to keep myself honest: I'm aware that HMACing with the API key (a suggestion I defended in a different comment) and storing the API key with a KDF mutually exclusive and that might seem like I'm giving contradictory advice. My specific recommendation is still to just use API keys stored plaintext server side just like 'tptacek is. I'm just saying that these alternative suggestions aren't silly.
TL;DR: plaintext API keys are fine and you should use them but you're not a bad person for wanting to hash them :)
I would think that hashing the API keys for storage, like a password, is worth it. It's extremely cheap in both development and CPU time because you can just use SHA-256 (high entropy is sufficient to deter brute force), and avoids unauthorized accesses in case of a database leak.
If you've lost the database, you've lost accountability for all user actions anyways. Everyone should be forced to reset their credentials.
Meanwhile: storing authenticators of API keys rather than the keys themselves precludes you from doing "request signing" with the keys in the future, since validating a MAC will require the preimage of the hash.
Generally: if you're concerned about survivability of systems after compromise, a better strategy for storing API secrets would be to compartmentalize them in a simple authentication server with a tiny attack surface (you could implement it with minimal state by doing a challenge-response protocol internally, so that the only interface exposed by the authentication server would be "does this message have a valid MAC").
Meanwhile: storing authenticators of API keys rather than the keys themselves precludes you from doing "request signing" with the keys in the future, since validating a MAC will require the preimage of the hash.
Generally: if you're concerned about survivability of systems after compromise, a better strategy for storing API secrets would be to compartmentalize them in a simple authentication server with a tiny attack surface (you could implement it with minimal state by doing a challenge-response protocol internally, so that the only interface exposed by the authentication server would be "does this message have a valid MAC").
> If you've lost the database, you've lost accountability for all user actions anyways.
Not sure I understand why. Is it because the (hashed) passwords are there too, or did you have an attack against the API keys in mind? Just to be clear, I'm thinking of an unsecured backup scenario, not full blown database compromise.
Regarding signing, that's a good point I hadn't considered, and maybe reason enough not to hash the API keys.
Not sure I understand why. Is it because the (hashed) passwords are there too, or did you have an attack against the API keys in mind? Just to be clear, I'm thinking of an unsecured backup scenario, not full blown database compromise.
Regarding signing, that's a good point I hadn't considered, and maybe reason enough not to hash the API keys.
In my mental model, you lose the database to appsec flaws, not to opsec flaws with backups. Both things definitely can happen! I'd suggest though that losing your entire (auth) database to a compromised backup is a "reset all the API keys" moment anyways.
Remember: these services that provide API keys are invariably backed with some kind of login service that uses passwords. Even if you're hashing with scrypt, if you lose the password hashes, everyone's resetting creds.
Remember: these services that provide API keys are invariably backed with some kind of login service that uses passwords. Even if you're hashing with scrypt, if you lose the password hashes, everyone's resetting creds.
Isn’t the issue the time when you’ve lost your dB and the time you realize it? During that time keys could be used.
> > Key vs Secret
> An AWS_Access_Key is basically just a random username. If you understand how userids work in your system, you don't have to worry about this.
The big difference is actually that the aws secret key is never sent to the server, but rather used as the secret in an HMAC.
That means that if I make a request to S3 with a given secret key, S3 never has access to that secret key (rather it takes my HMAC'd signature, passes it on to the IAM service internally, and gets back a 'yes' or 'no' for if it's correct).
That means that S3's logs can't contain sensitive information (the HMAC includes a time, so it can't even be replayed after a short while), any networking dumping I do doens't contain the secret key, etc.
This is a useful property in general.
> An AWS_Access_Key is basically just a random username. If you understand how userids work in your system, you don't have to worry about this.
The big difference is actually that the aws secret key is never sent to the server, but rather used as the secret in an HMAC.
That means that if I make a request to S3 with a given secret key, S3 never has access to that secret key (rather it takes my HMAC'd signature, passes it on to the IAM service internally, and gets back a 'yes' or 'no' for if it's correct).
That means that S3's logs can't contain sensitive information (the HMAC includes a time, so it can't even be replayed after a short while), any networking dumping I do doens't contain the secret key, etc.
This is a useful property in general.
This is true of lots of API secret schemes that don't have an opaque random key ID.
If your secrets are opaque random tokens that only have meaning as an assertion to your servers, it's worth asking yourself what value you'd derive by trying to hide them from server components. And, obviously, something on the serverside will need enough information (almost always: the secret itself) to verify the message.
If your secrets are opaque random tokens that only have meaning as an assertion to your servers, it's worth asking yourself what value you'd derive by trying to hide them from server components. And, obviously, something on the serverside will need enough information (almost always: the secret itself) to verify the message.
Advocate-of-the-devling re: "something still needs the secret" (assuming you're symmetrically MAC'ing, not asymmetrically signing): yep, but that can be a tiny, separate component and that might be better than everything seeing the plaintext secret.
I wouldn't recommend such a scheme by default, because now e.g. your API clients are way hairier to write (e.g. serialization, possibly canonicalization) and it's pretty easy to mess this up (AWS did in their first attempt). But if you have resources to do it well, that is: audit the heck out of the protocol and the tiny component that verifies the MACs, it does prevent some problems mentioned such as unintentional disclosure of the key on the server side, replayability of requests. I don't think those are super valuable properties, but it's also not intrinsically a hare-brained idea.
TL;DR: API keys are fine and you should use them but you're not a bad person for wanting to HMAC things :)
I wouldn't recommend such a scheme by default, because now e.g. your API clients are way hairier to write (e.g. serialization, possibly canonicalization) and it's pretty easy to mess this up (AWS did in their first attempt). But if you have resources to do it well, that is: audit the heck out of the protocol and the tiny component that verifies the MACs, it does prevent some problems mentioned such as unintentional disclosure of the key on the server side, replayability of requests. I don't think those are super valuable properties, but it's also not intrinsically a hare-brained idea.
TL;DR: API keys are fine and you should use them but you're not a bad person for wanting to HMAC things :)
Also important to remember is that even when someone thinks JWT will allow avoiding database lookups they eventually still add them for one reason or another (e.g. for instantaneous token revocation).
Macaroons look nice and simple on the surface but have many edge cases that are not at all supported by the existing libraries or have bad/insecure defaults.
For random keys it's good to remember that plain lookup in the database is prone to time side channel attacks, see https://paragonie.com/blog/2017/02/split-tokens-token-based-...
Macaroons look nice and simple on the surface but have many edge cases that are not at all supported by the existing libraries or have bad/insecure defaults.
For random keys it's good to remember that plain lookup in the database is prone to time side channel attacks, see https://paragonie.com/blog/2017/02/split-tokens-token-based-...
Agree but be sure to configure the cookie correctly. Set it to http-only, "secure" (only transmit over https) and set the domain/path correctly so only your server/application can see it.
Use the "__Host-" name prefix. [0]
This prevents cookies from being set at the root domain by a subdomain. Which in turn prevents non-TLS MITM cookie injection [1] as long as the application uses HSTS (even without includeSubdomains from the root).
[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se...
[1] https://www.usenix.org/conference/usenixsecurity15/technical...
This prevents cookies from being set at the root domain by a subdomain. Which in turn prevents non-TLS MITM cookie injection [1] as long as the application uses HSTS (even without includeSubdomains from the root).
[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se...
[1] https://www.usenix.org/conference/usenixsecurity15/technical...
I like seeing the "you're struggling toward reinventing JWT; consider just using JWT" discussion play out in the comments. (Spoiler: the dev ends up deciding to just use JWT.)
JWT is a great idea, but in practice the actual specification is broken in dumb ways. You may as well lob off the entire header portion of the token since there's no sense in listening to what it says.
It is broken but at the same time, if you don't send the header, you're not using a real JWT according to RFC 7519.
If you're going to use JWTs - even with all their flaws - please use them properly.
If you're going to use JWTs - even with all their flaws - please use them properly.
Sure, but if I'm not actually setting out to reinvent auth, I'm still better off going with a suboptimal but basically functional standard like JWT than I am trying to roll my own.
I find it hard to recommend JWT unless you're already a security expert.
This is unfortunate, since JWT was meant to bring client-side tokens to the masses, but the reality is that:
1. JWT has amazing cross-language support, but most of the libraries out there are unaudited, and even the ones that are audited can change and, let's say, make the 'None' signer the default one: https://github.com/lcobucci/jwt/commit/6507ac39be5a5e06457c8...
2. JWT can be used somewhat responsibly, but the JWT spec does n't enforce any security measure and doesn't even contain proper recommendations like:
a. Never use the None algorithm. b. Never use the RSA algorithm. c. Basically, only use HMAC (or Ed25519 if supported). d. Always include an issue timestamp ('iat' claim) and possibly a unique nonce ('tid') to make tokens unique. e. Always do key rotation using 'kid'. f. Implement a blacklist-based token revocation mechanism (or use short-lived access tokens backed by DB-based refresh tokens). g. Include the token expiry claim! Seriously, JWT doesn't even require token expiry! h. Don't use JWE. h. Associate your keys with signature algorithms, and _make sure that token is signed with the right alorithm_: https://auth0.com/blog/critical-vulnerabilities-in-json-web-...
3. I'm yet to encounter a library which enforces good defaults on your JWT usage. Some libraries validate expiry by default (if the claim exists), but from my experience most of them don't set it by default. Almost no library contains an convenience method for setting the issue time or a nonce, let alone setting them by default.
4. No JWT library I know of provides help for implementing basic features like key rotation/revocation and token revocation. Without these features JWT security model becomes vastly inferior to database-backed tokens.
The reality is that developers implementing JWT choose it exactly because they don't want to delve into all these complications of using tokens, so they just pick up a library and hack something around it, expecting the JWT library to take care of all the security issues for them.
1. JWT has amazing cross-language support, but most of the libraries out there are unaudited, and even the ones that are audited can change and, let's say, make the 'None' signer the default one: https://github.com/lcobucci/jwt/commit/6507ac39be5a5e06457c8...
2. JWT can be used somewhat responsibly, but the JWT spec does n't enforce any security measure and doesn't even contain proper recommendations like:
a. Never use the None algorithm. b. Never use the RSA algorithm. c. Basically, only use HMAC (or Ed25519 if supported). d. Always include an issue timestamp ('iat' claim) and possibly a unique nonce ('tid') to make tokens unique. e. Always do key rotation using 'kid'. f. Implement a blacklist-based token revocation mechanism (or use short-lived access tokens backed by DB-based refresh tokens). g. Include the token expiry claim! Seriously, JWT doesn't even require token expiry! h. Don't use JWE. h. Associate your keys with signature algorithms, and _make sure that token is signed with the right alorithm_: https://auth0.com/blog/critical-vulnerabilities-in-json-web-...
3. I'm yet to encounter a library which enforces good defaults on your JWT usage. Some libraries validate expiry by default (if the claim exists), but from my experience most of them don't set it by default. Almost no library contains an convenience method for setting the issue time or a nonce, let alone setting them by default.
4. No JWT library I know of provides help for implementing basic features like key rotation/revocation and token revocation. Without these features JWT security model becomes vastly inferior to database-backed tokens.
The reality is that developers implementing JWT choose it exactly because they don't want to delve into all these complications of using tokens, so they just pick up a library and hack something around it, expecting the JWT library to take care of all the security issues for them.
If you would've used this logic just a little while ago and slapped an off-the-shelf JWT library into your app, you'd have a trivially exploitable design flaw that could be used to easily forge tokens.
In that case, reinventing the wheel properly probably would've been more secure, if not perfect.
I personally use JWT, but unfortunately like so many other things you can't just drop it in and forget; you have to have some level of care if you really want it to work. All cryptography has its caveats.
OTOH, going with simple, server-issued tokens that don't carry data actually is pretty proven already, so it's not really that bad of an idea anyways, if you are only ever going to have a few servers in the same region anyways.
In that case, reinventing the wheel properly probably would've been more secure, if not perfect.
I personally use JWT, but unfortunately like so many other things you can't just drop it in and forget; you have to have some level of care if you really want it to work. All cryptography has its caveats.
OTOH, going with simple, server-issued tokens that don't carry data actually is pretty proven already, so it's not really that bad of an idea anyways, if you are only ever going to have a few servers in the same region anyways.
All else equal, I'm more likely to find out about a dumb gotcha in a commonly used standard before it bites me than I am in my own sui generis design that nobody else uses. You're right that crypto of any sort requires care in how we use it, but preferring a more widely used method lets us leverage the care of others as well as our own. And if we don't bother, we're probably about as hosed either way.
And, yeah, if it's practical to just generate and issue a secret from the server side and authenticate it at a single point when the client presents it back to us, we may as well do that. I'm not sure how practicable an option that is in OP's case, though; the description of the problem suggests there's something rather strange going on, and I don't grasp it well enough to speak to what scheme is really most useful in the context. But it does sound like having the client bear its own claims is useful, and JWT would be the obvious go-to for the use case.
And, yeah, if it's practical to just generate and issue a secret from the server side and authenticate it at a single point when the client presents it back to us, we may as well do that. I'm not sure how practicable an option that is in OP's case, though; the description of the problem suggests there's something rather strange going on, and I don't grasp it well enough to speak to what scheme is really most useful in the context. But it does sound like having the client bear its own claims is useful, and JWT would be the obvious go-to for the use case.
What terrible standard could you ever avoid given this logic? I could just as easily convince you to use raw XML DSIG with this argument; after all, in the dichotomy you propose, your only alternative would be a DIY reimplementation of XML DSIG. Why not find a way to fit IKE into your design as well? What, are you going to redesign your own IKE?
Of course this argument is rather useless when taken as the sole metric by which to decide whether and which standards to use, or implementations to prefer. But why would anyone do such a nonsensical thing as that?
Yeah, JWT, like YAML, was one of those dead-simple, great ideas that ended up overcomplicated and error-prone.
There’s so much FUD around JWT that I feel compelled to answer this everytime it comes up.
Just to get it out of the way, you’re correct that the JWT spec doesn’t do a lot to prevent implementors from doing stupid things, particularly bad was downgrade attacks from asymmetric to symmetric keys.
Here’s the thing, these issues don’t exist anymore. Amazon’s Cognito relies heavily on JWT and I tend to trust Amazon’s security folks.
Just to get it out of the way, you’re correct that the JWT spec doesn’t do a lot to prevent implementors from doing stupid things, particularly bad was downgrade attacks from asymmetric to symmetric keys.
Here’s the thing, these issues don’t exist anymore. Amazon’s Cognito relies heavily on JWT and I tend to trust Amazon’s security folks.
"Amazon uses JWT" and "Amazon knows what they're doing" are not unreasonable statements, but it doesn't follow that "therefore JWT is now fine for everyone".
You can make that argument for a lot of different footgun specs. I'm not saying JWT implies broken. I'm saying that JWT plus sub-billion-dollar-company-security-budget often leads to disaster. I'm also saying it's an unforced error, in that people do that in order to solve a problem they most likely don't have. It's clear that some people can do JWT correctly, just like e.g. some people can do OAuth2 correctly (to add another example to your list: Google/GSuite-as-an-IdP). That doesn't make it a safe and well-designed spec, and it doesn't make it great general advice.
You can make that argument for a lot of different footgun specs. I'm not saying JWT implies broken. I'm saying that JWT plus sub-billion-dollar-company-security-budget often leads to disaster. I'm also saying it's an unforced error, in that people do that in order to solve a problem they most likely don't have. It's clear that some people can do JWT correctly, just like e.g. some people can do OAuth2 correctly (to add another example to your list: Google/GSuite-as-an-IdP). That doesn't make it a safe and well-designed spec, and it doesn't make it great general advice.
This is not FUD. It is a design flaw.
It is now widely recognized that the header component of the JWT token cannot be used except to reject a token, making it pretty close to absolutely useless (except for debugging.)
HOWEVER. Before this was recognized, most JWT implementations were broken and were easily susceptible to the most basic of downgrade attacks.
Downgrading from asymmetric to symmetric was a slap in the face for obvious reasons, but that's just the last round of problems. The early problems were even more ridiculous; many libraries would readily downgrade to 'none' and turn off protection altogether.
Not just Amazon Cognito, but Google's OIDC implementation, and indeed, all OIDC implementations, use JWT. JWT implementations today are hopefully no longer susceptible to basic lapses in security.
But there's good reason for its reputation: the way it's designed lead to these trivial downgrade attacks. Having the token specify the algorithm was a bad decision, and it lead to bad implementations. Importantly, someone implementing JWT today could easily make some of the same mistakes if they aren't careful.
I hope a future JWT release entirely removes the information from the token and just forces the client/server to agree statically.
It is now widely recognized that the header component of the JWT token cannot be used except to reject a token, making it pretty close to absolutely useless (except for debugging.)
HOWEVER. Before this was recognized, most JWT implementations were broken and were easily susceptible to the most basic of downgrade attacks.
Downgrading from asymmetric to symmetric was a slap in the face for obvious reasons, but that's just the last round of problems. The early problems were even more ridiculous; many libraries would readily downgrade to 'none' and turn off protection altogether.
Not just Amazon Cognito, but Google's OIDC implementation, and indeed, all OIDC implementations, use JWT. JWT implementations today are hopefully no longer susceptible to basic lapses in security.
But there's good reason for its reputation: the way it's designed lead to these trivial downgrade attacks. Having the token specify the algorithm was a bad decision, and it lead to bad implementations. Importantly, someone implementing JWT today could easily make some of the same mistakes if they aren't careful.
I hope a future JWT release entirely removes the information from the token and just forces the client/server to agree statically.
I agree strongly that JWT is badly flawed, and that criticisms of JWT don't constitute "FUD". JWT's uptake has been alarming given how little cryptographic engineering input the format seems to have received, versus how complex it is under the hood.
That said:
Asymmetric crypto is a crypto code smell. You use it when you absolutely have to because there's no other way to express what you're trying to accomplish. It is much harder to get public key crypto right than it is to safely use a "Seal/Unseal" AEAD interface. One of the things that alarms me about JWT is that it's a format that presumes developers might want to effortlessly switch between symmetric and asymmetric crypto, as if they were just two different ways of solving the same underlying problem.
That said:
Asymmetric crypto is a crypto code smell. You use it when you absolutely have to because there's no other way to express what you're trying to accomplish. It is much harder to get public key crypto right than it is to safely use a "Seal/Unseal" AEAD interface. One of the things that alarms me about JWT is that it's a format that presumes developers might want to effortlessly switch between symmetric and asymmetric crypto, as if they were just two different ways of solving the same underlying problem.
Amazon are the same folks who store their infra status page on their own infra, which is fine when it's up, and pretty useless when it's down. Even the smartest people are still just people - it's worth trusting several groups of smart people over just one.
What's treacherous about JWT is that the programming interface is extremely simple and pretty closely approaches the ideal of what developers would want from secure clientside storage. It's a great interface, but terrible crypto.
https://news.ycombinator.com/item?id=14292223
https://news.ycombinator.com/item?id=14292223
You're right! You need a strong engineering culture around its use to avoid its pitfalls, and without that, the results are often regrettable. And I agree, too, that this is a failing of the standard, not of those who use it.
But it's also true that JWT has legitimate use cases; I'm working with it right now in my professional capacity, and we are using it precisely because "twenty bytes of hexified /dev/urandom" isn't a terribly efficient option at our scale - we could do it, but we won't, because it doesn't add value to the business.
As you say, though, the crypto sucks. You know a great deal, both in the absolute and relative to my own knowledge, about such things. What, in your estimation, is the means most likely to produce an outcome where we have something of roughly similar utility and ease of use, but with solid crypto and few or no bear traps for the unwary engineer? (Beyond just removing the algorithm negotiation, or at least the "none" option.) Or does such a thing exist and I'm just one of today's lucky 10,000?
But it's also true that JWT has legitimate use cases; I'm working with it right now in my professional capacity, and we are using it precisely because "twenty bytes of hexified /dev/urandom" isn't a terribly efficient option at our scale - we could do it, but we won't, because it doesn't add value to the business.
As you say, though, the crypto sucks. You know a great deal, both in the absolute and relative to my own knowledge, about such things. What, in your estimation, is the means most likely to produce an outcome where we have something of roughly similar utility and ease of use, but with solid crypto and few or no bear traps for the unwary engineer? (Beyond just removing the algorithm negotiation, or at least the "none" option.) Or does such a thing exist and I'm just one of today's lucky 10,000?
You can start with running your JSON data (or whatever serialization format that you choose) through NaCl/libsodium's crypto_secretbox() and you already get more than what JWT offers by default.
It still doesn't include everything you need of course: at the very least you'll still need to implement expiry validation and some token revocation mechanism. But you actually get more value out of the box than you get with a JWT library: your token content is encrypted, your using a solid crypto library and you don't have to mess around with any knob to secure your implementation.
It doesn't offer support for hot-swapping crypto algorithms, but for what it's worth, cryptographic agility is the root cause of many issues we had with TLS in the past[1].
If you end up needing both symmetric and asymmetric cryptography in your tokens, you better treat them as two different types of tokens, because they usually are. I think tptacek already said, but asymmetric crypto is rarely interchangeable with symmetric crypto - you usually use them in very different cases.
[1] https://www.imperialviolet.org/2016/05/16/agility.html
It still doesn't include everything you need of course: at the very least you'll still need to implement expiry validation and some token revocation mechanism. But you actually get more value out of the box than you get with a JWT library: your token content is encrypted, your using a solid crypto library and you don't have to mess around with any knob to secure your implementation.
It doesn't offer support for hot-swapping crypto algorithms, but for what it's worth, cryptographic agility is the root cause of many issues we had with TLS in the past[1].
If you end up needing both symmetric and asymmetric cryptography in your tokens, you better treat them as two different types of tokens, because they usually are. I think tptacek already said, but asymmetric crypto is rarely interchangeable with symmetric crypto - you usually use them in very different cases.
[1] https://www.imperialviolet.org/2016/05/16/agility.html
Rails applications have been doing stateless auth in encrypted cookies for over a decade, and the constructions they use to do it are safer than JWT's, and orders of magnitude less complex. I would want to push back on the idea that a good use case for JWT is "we need to do stateless auth".
Stateless auth is a bad place to start. But, when you need it, there are better ways to do it than JWT.
Stateless auth is a bad place to start. But, when you need it, there are better ways to do it than JWT.
I wish the author went deeper into the topic. For example JWTs have a pretty straightforward structure and if you use just a subset of the spec they look like a clean and simple solution. But did the author check Macaroons? Did they consider SSL Client Certificates?
What is JWT?
The first Google search result for "jwt" is https://jwt.io/ which explains it pretty well for you. https://en.wikipedia.org/wiki/JSON_Web_Token is pretty good, too.
fleitz(1)
migth want to check out https://github.com/yahoo/athenz
An auth/security lib from the company that leaked a billion users PII?
First off, I'm not an expert at security, but I do have a background in it and in development of microservices. So fair warning.
API keys provide reasonably good security, but there is a reason why companies have moved to OAuth. Using https, your payloads are protected by SSL, but the URI and the request and response headers aren't. That means no matter how you intend to pass along your API key, it is in the open and thus can be intercepted.
So, bad, right? Well, maybe not. If whatever you are securing with an API key is not that critical to anyone and you just want to prevent outsiders from calling your API or calling it repeatedly and abusing it, then API keys are fine. I would make it hard to guess, but I wouldn't go crazy with that, either, because everything is "easy to guess" for anyone who is really working hard at gaining access. If you are just working on a game, I would do the basics to secure your services and then move on to the more critical matters.
API keys provide reasonably good security, but there is a reason why companies have moved to OAuth. Using https, your payloads are protected by SSL, but the URI and the request and response headers aren't. That means no matter how you intend to pass along your API key, it is in the open and thus can be intercepted.
So, bad, right? Well, maybe not. If whatever you are securing with an API key is not that critical to anyone and you just want to prevent outsiders from calling your API or calling it repeatedly and abusing it, then API keys are fine. I would make it hard to guess, but I wouldn't go crazy with that, either, because everything is "easy to guess" for anyone who is really working hard at gaining access. If you are just working on a game, I would do the basics to secure your services and then move on to the more critical matters.
You appear to be shadowbanned for no immediately obvious reason. I vouched this comment to point out that it contains a factual error, to wit, the claim that the headers and URI of a request made via TLS are sent in the clear. They are not; the socket is encrypted before any protocol messages are exchanged. Perhaps you're thinking of SMTP STARTTLS?
Companies have moved to OAuth, and to JWT as the payload, because it provides a lightweight, shared-little authentication/authorization framework that's well suited to the needs of a distributed microservice architecture and to third-party service integration - not because it overcomes a nonexistent problem with HTTPS not encrypting headers.
Companies have moved to OAuth, and to JWT as the payload, because it provides a lightweight, shared-little authentication/authorization framework that's well suited to the needs of a distributed microservice architecture and to third-party service integration - not because it overcomes a nonexistent problem with HTTPS not encrypting headers.
A properly configured SMTP STARTTLS would not really send anything substantial in cleartext either, it would issue the STARTTLS command as the first command and the rest would be wrapped in encrypted TLS. You can think of the unencrypted STARTTLS preamble as a set of static ascii bytes that are the same for every connection.
It's the closest common parallel I can find for what the prior commenter is describing, but yeah, it isn't really all that close. I've seen poor implementations leak, but not for many years now.
I don't think the op had something else in mind. IME it's a strangely widespread misunderstanding that https encrypts just parts of the http request. I think it's because of the advice to not put sensitive data in urls. The reason for that is due to logging at the endpoint, but it's misunderstood that it's not encrypted. It's funny because it's so easy to test the hypothesis, and it's hard to even propose a reason it would be that way.
Headers and URI are encrypyed aswell, in a HTTPS request. Only the domain is exposed (when SNI is supported)
Not trying to be pendantic but the entire hostname is exposed, not just the domain name. I’m assuming that’s what you meant.
The hostname is a domain name. Some domain names may not be hostnames, but all hostnames are domain names. So the entire domain name, which is also a hostname, is exposed.
These definitions are highly context-dependent, though.
Even more pedantic, but sometimes surprising: SNI reveals the full text of the final lookup query that the requester used to obtain an IP address to open a TCP connection to the server.
Neither the text nor the address are necessarily "correct", and the text might be formally a nodename, a hostname, a domain name, a fully-qualified domain name, or a text representation of an IP address (which, again, is not necessarily correct).
In practice, certificate authorities constrain the possibilities of the lookup text (for a successful connection using the CA-signed cert), but that is not a technical limitation. And of course, a self-signed certificate has no such constraints.
With cooperation between the server owner and users, an SNI-sensitive publisher could make their site available at https://fbi.gov/. But it's probably easier just to use a meaningless domain instead. :)
Even more pedantic, but sometimes surprising: SNI reveals the full text of the final lookup query that the requester used to obtain an IP address to open a TCP connection to the server.
Neither the text nor the address are necessarily "correct", and the text might be formally a nodename, a hostname, a domain name, a fully-qualified domain name, or a text representation of an IP address (which, again, is not necessarily correct).
In practice, certificate authorities constrain the possibilities of the lookup text (for a successful connection using the CA-signed cert), but that is not a technical limitation. And of course, a self-signed certificate has no such constraints.
With cooperation between the server owner and users, an SNI-sensitive publisher could make their site available at https://fbi.gov/. But it's probably easier just to use a meaningless domain instead. :)
> Neither the text nor the address are necessarily "correct", and the text might be formally a nodename, a hostname, a domain name, a fully-qualified domain name, or a text representation of an IP address (which, again, is not necessarily correct).
Please don't make claims about standards that you haven't read. SNI supports only DNS FQDNs.
Please don't make claims about standards that you haven't read. SNI supports only DNS FQDNs.
You are incorrrect.
That's not quite what the spec says, and furthermore it is not true in practice.
Giving you the benefit of the doubt, I reread the spec, and I tested in nginx.
Works fine.
That's not quite what the spec says, and furthermore it is not true in practice.
Giving you the benefit of the doubt, I reread the spec, and I tested in nginx.
Works fine.
What spec?
RFC6066 (https://tools.ietf.org/html/rfc6066#section-3) says "Currently, the only server names supported are DNS hostnames; however, this does not imply any dependency of TLS on DNS, and other name types may be added in the future (by an RFC that updates this document)." [snip] ""HostName" contains the fully qualified DNS hostname of the server, as understood by the client." [snip] "Literal IPv4 and IPv6 addresses are not permitted in "HostName"."
RFC6066 (https://tools.ietf.org/html/rfc6066#section-3) says "Currently, the only server names supported are DNS hostnames; however, this does not imply any dependency of TLS on DNS, and other name types may be added in the future (by an RFC that updates this document)." [snip] ""HostName" contains the fully qualified DNS hostname of the server, as understood by the client." [snip] "Literal IPv4 and IPv6 addresses are not permitted in "HostName"."
RFC6066, correct.
The "as understood by the client" is very important, apparently.
Furthermore, the name does not need to be in DNS. There is no single source of true DNS anyway (though of course the ~whole world uses the same root servers).
Try it. I just set up a remote server for "snitest", added the IP address to /etc/hosts only, generated a cert for that name only, and got the correct cert via SNI in a local browser.
The same process (minus /etc/hosts modification) also worked for a bare (textual representation of an) IP address.
The "as understood by the client" is very important, apparently.
Furthermore, the name does not need to be in DNS. There is no single source of true DNS anyway (though of course the ~whole world uses the same root servers).
Try it. I just set up a remote server for "snitest", added the IP address to /etc/hosts only, generated a cert for that name only, and got the correct cert via SNI in a local browser.
The same process (minus /etc/hosts modification) also worked for a bare (textual representation of an) IP address.
> Furthermore, the name does not need to be in DNS. There is no single source of true DNS anyway (though of course the ~whole world uses the same root servers).
That is not "furthermore", that is what "as understood by the client" means.
> Try it. I just set up a remote server for "snitest", added the IP address to /etc/hosts only, generated a cert for that name only, and got the correct cert via SNI in a local browser.
That is a fully qualified DNS hostname, by definition. You told both your server that it was a DNS FQDN it should serve and your client that it was a DNS FQDN that it should request, and so they obviously were matched exactly as required by the standard. Whether DNS is involved in the resolution process is irrelevant, especially so given that the standard nowhere specifies the DNS root to use.
> The same process (minus /etc/hosts modification) also worked for a bare (textual representation of an) IP address.
What was the name type in the certificate? That sounds like a bug in the client, and possibly in the server as well.
That is not "furthermore", that is what "as understood by the client" means.
> Try it. I just set up a remote server for "snitest", added the IP address to /etc/hosts only, generated a cert for that name only, and got the correct cert via SNI in a local browser.
That is a fully qualified DNS hostname, by definition. You told both your server that it was a DNS FQDN it should serve and your client that it was a DNS FQDN that it should request, and so they obviously were matched exactly as required by the standard. Whether DNS is involved in the resolution process is irrelevant, especially so given that the standard nowhere specifies the DNS root to use.
> The same process (minus /etc/hosts modification) also worked for a bare (textual representation of an) IP address.
What was the name type in the certificate? That sounds like a bug in the client, and possibly in the server as well.
Is your argument that a name can be a DNS FQDN even if it is not in DNS?
A non-DNS hostname can be treated as fully-qualified since there's no organizational structure to qualify it in.
> What was the name type in the certificate?
The CN? Is the IP address, as specified at cert generation.
> That sounds like a bug in the client, and possibly in the server as well.
Perhaps. I'm not a standards adjudicator.
Nevertheless, it works in the web we have. Tested in nginx, OpenSSL, curl, Firefox.
A non-DNS hostname can be treated as fully-qualified since there's no organizational structure to qualify it in.
> What was the name type in the certificate?
The CN? Is the IP address, as specified at cert generation.
> That sounds like a bug in the client, and possibly in the server as well.
Perhaps. I'm not a standards adjudicator.
Nevertheless, it works in the web we have. Tested in nginx, OpenSSL, curl, Firefox.
> Is your argument that a name can be a DNS FQDN even if it is not in DNS?
In the sense of the SNI spec, yes. It would be pointless to require the name to be "in DNS", for the simple reason that there is a potentially infinite number of DNS roots, and protocol standards naturally don't specify instances, only mechanism, so any TLS client could just implement a DNS server that uses the hosts file as its data source for a private DNS root and uses that for resolution, which would be indistinguishable from an implementation that simply skips the pointless encoding and decoding of DNS messages.
The point is to (a) distinguish host names from IP addresses, (b) require fully qualified names, (c) specify the syntax of the host names. Whether the names are actually in any DNS is irrelevant.
> A non-DNS hostname can be treated as fully-qualified since there's no organizational structure to qualify it in.
Sure, and that is fine. The point of the FQDN requirement is that if you send "foobar" as the SNI hostname, you cannot expect the server to match it to a certificate for "foobar.example.com". If both server and client agree on a namespace(/DNS root) where "foobar" is a fully qualified name, then you can expect everything to work just fine. It's not a requirement on how the name is to be verified, only a guarantee on how it may be transformed by protocol participants.
> The CN? Is the IP address, as specified at cert generation.
The CN is essentially deprecated, but matching an IP address in SNI against the CN should be fine. I am just not aware of any way to encode an IP address in SNI. If you had an alternate name IP address in the cert, matching an SNI host name against that would be a bug. If the browser actually accepted a certificate for an IP address when matching against a hostname, that would be a vulnerability.
> Nevertheless, it works in the web we have. Tested in nginx, OpenSSL, curl, Firefox.
Nah, if stuff that contradicts the standard happens, that means that there are things that the standard guarantees to work, but that are broken with the non-compliant implementation. Whether that buggy behaviour is useful to you in some way doesn't change that it's not "working" in any meaningful sense.
Oh, and if by OpenSSL you mean the command line tools: Yes, with those it does work correctly. If you specify the SNI name 1.2.3.4, that is the DNS hostname 1.2.3.4, which should indeed be correctly matched by the server against any certs for alternate DNS name 1.2.3.4, or possibly against certs for CN 1.2.3.4, but not certs for alternate name IP address 1.2.3.4.
In the sense of the SNI spec, yes. It would be pointless to require the name to be "in DNS", for the simple reason that there is a potentially infinite number of DNS roots, and protocol standards naturally don't specify instances, only mechanism, so any TLS client could just implement a DNS server that uses the hosts file as its data source for a private DNS root and uses that for resolution, which would be indistinguishable from an implementation that simply skips the pointless encoding and decoding of DNS messages.
The point is to (a) distinguish host names from IP addresses, (b) require fully qualified names, (c) specify the syntax of the host names. Whether the names are actually in any DNS is irrelevant.
> A non-DNS hostname can be treated as fully-qualified since there's no organizational structure to qualify it in.
Sure, and that is fine. The point of the FQDN requirement is that if you send "foobar" as the SNI hostname, you cannot expect the server to match it to a certificate for "foobar.example.com". If both server and client agree on a namespace(/DNS root) where "foobar" is a fully qualified name, then you can expect everything to work just fine. It's not a requirement on how the name is to be verified, only a guarantee on how it may be transformed by protocol participants.
> The CN? Is the IP address, as specified at cert generation.
The CN is essentially deprecated, but matching an IP address in SNI against the CN should be fine. I am just not aware of any way to encode an IP address in SNI. If you had an alternate name IP address in the cert, matching an SNI host name against that would be a bug. If the browser actually accepted a certificate for an IP address when matching against a hostname, that would be a vulnerability.
> Nevertheless, it works in the web we have. Tested in nginx, OpenSSL, curl, Firefox.
Nah, if stuff that contradicts the standard happens, that means that there are things that the standard guarantees to work, but that are broken with the non-compliant implementation. Whether that buggy behaviour is useful to you in some way doesn't change that it's not "working" in any meaningful sense.
Oh, and if by OpenSSL you mean the command line tools: Yes, with those it does work correctly. If you specify the SNI name 1.2.3.4, that is the DNS hostname 1.2.3.4, which should indeed be correctly matched by the server against any certs for alternate DNS name 1.2.3.4, or possibly against certs for CN 1.2.3.4, but not certs for alternate name IP address 1.2.3.4.
What works fine that is not syntactically a fully qualified DNS hostname, and what could you even test about that?
> SNI reveals the full text of the final lookup query that the requester used to obtain an IP address to open a TCP connection to the server.
Are you talking about something other than DNS lookups done at the client side?
Are you talking about something other than DNS lookups done at the client side?
Yes, that is how I tested it, but there's nothing magic about DNS either. Remember that this depends on the site owner and the users cooperating to put misleading data into the SNI, while still functioning properly. It isn't often useful, but it is sometimes convenient.
Basically, whatever the client does that results in a successful map of name to address (including the textual representation of the address, to the address) will cause the name to be sent in SNI, and will be used to select a matching cert on the server side.
If you do use DNS, your lookup might have a suffix appended automatically, depending on local nameserver config.
Basically, whatever the client does that results in a successful map of name to address (including the textual representation of the address, to the address) will cause the name to be sent in SNI, and will be used to select a matching cert on the server side.
If you do use DNS, your lookup might have a suffix appended automatically, depending on local nameserver config.
[deleted]
Key Generation
generate-safe-id is fine. For a secret API key, UUIDs are less fine. Also fine: directly reading 16-32 bytes from crypto.getRandomBytes and converting to hex.
Key vs Secret
An AWS_Access_Key is basically just a random username. If you understand how userids work in your system, you don't have to worry about this.
Key Usage
Don't use JWT; JWT is a mess. The real distinction here isn't what a secret key can get you, but whether you attempt to track clients without serverside state. What JWT ostensibly gets you is an arbitrary per-client state store without requiring database lookups on the serverside. In reality, you almost never get that, and inherit all the downsides of JWT --- including extreme, scary complexity --- for very little practical benefit. If you're already using a database to handle client request (like most applications), keep it simple.
Key Storage (Clientside)
Cookies are safer than localStorage; modern browsers are bristling with defenses for cookies, but not for localStorage access.
Key Storage (Serverside)
We hash passwords because passwords are valuable across sites; it's a big deal to compromise someone's password, even on a random low-value application. That's not true of API keys. If your database is compromised, the API keys don't matter anymore. Don't bother encrypting them.
Authorization
Authorization is a big topic. API keys are an authentication concern. AuthN != AuthZ. Keep it simple with AuthN.
Don't:
* Bother with TLS client certificates. Client certs are useful internally, when you have an ensemble of services that need to talk to each other securely. They're much less useful when you have tens of thousands of semi-anonymous clients. They're a huge pain in the ass to deal with on both the client and the server side, and have extremely bad UX.
* Use JWT or Macaroons. If you can get away with opaque random strings and a serverside database, then get away with that, for as long as you possibly can. We work on applications running at "popular Internet app" scales and they manage this just fine. Trying to push client state to the client costs security; it's something you get away with, not something you do to shore up your defenses.
* Bother with OAuth. The only "interesting" thing OAuth provides is a UX to allow you to delegate authentication to third-party services --- think, every app that wants to see your Twitter timeline. If you don't have that problem, OAuth doesn't buy you anything except complexity.