Everything you always wanted to know about JSON Web Tokens
What are JSON Web Tokens (JWTs) and how do we use them?
I’m starting a series of articles about API authorization and security. This is the first article in the series. It explains what JSON Web Tokens (JWTs) are and how we use them. I hope you find this article and the rest of the series useful!
When working with APIs, you’ve probably seen the requirement to include authorization tokens in the request headers, typically under the Authorization
header. We call these tokens JSON Web Tokens (JWTs), since they represent JSON documents that contain information about the user sending the request. An example of such tokens is this:
eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL2F1dGgucHlqb2JzLndvcmtzIiwic3ViIjoiZjFiYTRlODEtNzkzMi00YjNiLWFiMzEtZDFhZTFkNWU5OGNhIiwiYXVkIjoiaHR0cHM6Ly9weWpvYnMud29ya3Mvam9icyIsImlhdCI6MTY2NzYzODI2NSwiZXhwIjoxNjY3NzI0NjY1fQ.ZKJPOtwYzRxQYmlTZsQcUa5FFvf3-pNNcHz5WgsOlK1xEHhW6d4hpA_0Y9-m_V0rSycRX1Ln5qr7XAvbY8pRoscAIf7R5SxDDwwILcPXUD1DVrBbUIDukeHDmUOQ6wcEqNOwSCB3EJMrTDBVqOwS-g-llnir82vUS3OCIB2iYRxRsj0TgxD3ekwnkKLnuNXJo9bfG2-XWccrBBLdFt3IAl120B0gciqpMa-fQwSjVaWXLZ2QvQRa8eSwVvN3DBhtqx9Jx1Rdnw9rKe1aorGfuHE4Syr7MRPK4UtaEEaOX5AklWNilmR26sApP7UFlOS_iXwiWxm2qf67m4d08xibyw
Over the past years, I’ve helped dozens of companies implement secure JWT-based authorization for their APIs, and over the course of my work, I’ve accumulated a number of recurrent questions and common mistakes. The goal of this article is to address some of those questions and mistakes.
TL;DR
JWTs are JSON documents that contain claims about a user. JWTs are base64url encoded.
We distinguish between ID tokens and access tokens. ID tokens identify a user while access tokens authorize their access to an API.
Tokens are issued by an authorization service. The API validates that the token is valid.
JWTs have three parts: header (how the token is signed), payload (the claims), and signature. The signature proves that the token is legit.
The most popular signing algorithms are HS256 and RS256. HS256 uses a shared secret while RS256 uses a public/private key pair.
The client includes the access token in every request, under the
Authorization
header, and the API server validates the token in every request.To validate a token, we verify its signature and check that all the claims are correct, e.g. we verify the audience is correct and the token isn’t expired.
jwt.io is a very useful tool to inspect tokens and validate their signatures.
What are JWTs?
Essentially, a JWT is a JSON document that contains information about a user. We call the properties in a JWT claims. JWTs are encoded using base64url encoding, which is a form of base64 encoding that uses only safe characters for URLs (check out RFC-4648 for more details).
JWTs are defined in RFC-7519, while RFC-8725 describes current best practices. I recommend you take a look at both documents.
We distinguish between two main types of JWTs:
ID tokens: they contain identifying information about a user, such as their name, email, date of birth, and so on.
Access tokens: they contain claims about the right of a user to access an API.
ID tokens are issued by an authorization server after a successful login. In a UI, we may use ID tokens to populate details about the user. We don’t use ID tokens to authorize user access. In fact, ID tokens should never be sent back to the server, and since they contain identifying and potentially sensitive information about users, we really want to restrict the circulation of those tokens. Typically, ID tokens are used in the context of an OpenID Connect (OIDC) integration.
To authorize access to an API, we use access tokens. Access tokens are issued by an authorization server in response to an authorization request. Before we can access an API, we need to obtain permission to do so. To ask for permission, we send an authorization request to the authorization server. If the request is successful, the authorization server issues an access token.
The type of tokens we discuss in this article are also known as JWS (JSON Web Signature), which are JWTs with a signature at the end (more on this below). As I mentioned earlier, JWTs are base64url encoded, which means anyone can inspect their contents. In some cases, this isn’t desirable - for example, when the token contains sensitive information. In those cases, we use another type of token known as JWE (JSON Web Encryption), which are encrypted tokens. Since JWEs are encrypted, their contents cannot be inspected and therefore we can use them to transfer sensitive data. For more information on this topic, check out this article by Prabath Siriwardena.
Before I continue, I’d like to make 2 important clarifications about access tokens:
It’s best practice to decouple token management from your application’s API. Use an authorization service to issue and manage access tokens. The authorization service manages user identities and credentials, and it knows which APIs each user has access to. Application APIs don’t issue tokens, they only receive and validate access tokens. Let’s say you have a job portal that exposes several APIs, like a jobs API and a candidates API. Access to those APIs will be controlled by an authorization service, i.e. neither the jobs nor the candidates APIs will issue their own tokens.
Access tokens are requested by a client who wants to access an API. The client could be anything from a browser application to a microservice. The client is responsible for caching the token, sending it with every request to the API, and refreshing it when it expires. A fairly common mistake I’ve seen in some APIs is that they cache the client’s token upon validating it for the first time, and then validate every subsequent request against the same cached token. Crazy stuff 🤯.
Structure of a JWT
As you can see in the figure below, JWT contains three parts:
Header: metadata about the JWT. For example, it identifies the document as a JWT, specifies its signature algorithm, and the key that was used to sign it.
Payload: the actual set of claims about the user. In the case of ID contains, these are the user identifying claims, while in the case of access tokens, these are the authorization claims.
Signature: the token’s signature. The signature proves that the token is valid.
When producing a token, each part of the JWT is base64url encoded separately and joined through periods. Consider the below token:
eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL2F1dGgucHlqb2JzLndvcmtzIiwic3ViIjoiZjFiYTRlODEtNzkzMi00YjNiLWFiMzEtZDFhZTFkNWU5OGNhIiwiYXVkIjoiaHR0cHM6Ly9weWpvYnMud29ya3Mvam9icyIsImlhdCI6MTY2NzYzODI2NSwiZXhwIjoxNjY3NzI0NjY1fQ.ZKJPOtwYzRxQYmlTZsQcUa5FFvf3-pNNcHz5WgsOlK1xEHhW6d4hpA_0Y9-m_V0rSycRX1Ln5qr7XAvbY8pRoscAIf7R5SxDDwwILcPXUD1DVrBbUIDukeHDmUOQ6wcEqNOwSCB3EJMrTDBVqOwS-g-llnir82vUS3OCIB2iYRxRsj0TgxD3ekwnkKLnuNXJo9bfG2-XWccrBBLdFt3IAl120B0gciqpMa-fQwSjVaWXLZ2QvQRa8eSwVvN3DBhtqx9Jx1Rdnw9rKe1aorGfuHE4Syr7MRPK4UtaEEaOX5AklWNilmR26sApP7UFlOS_iXwiWxm2qf67m4d08xibyw
If you look carefully, you’ll notice the token has three sequences joined by periods. Each sequence represents one of the parts of the JWT we discussed above: first the header, then the payload, and finally the signature.
Let’s now look at each JWT part in detail!
JWT header
Also known as JOSE (JSON Object Signing and Encryption) Header, the header contains metadata about the JWT, including information about the signature algorithm and the key that was used to sign it. We use this information to validate the token on the server.
The most commonly found claims in JWT headers are the following:
typ
(type): it tells us what type of document it is. Its value should be “JWT”.alg
(algorithm): it tells us what algorithm was used to sign the token.kid
(key ID): it tells us the ID of the key that was used to sign the token.
Below is an example of JWT header:
{
"typ": "JWT",
"alg": "RS256",
"kid": "1ee2ca16-a0a1-44ab-99d9-cf9b2bd50c6d"
}
Let’s analyze this header:
The
typ
claim tells us that this is a JWT.The
alg
claim tells us that this token was signed using the RS256 algorithm. More on signing algorithms below.The
kid
claim tells us that this token was signed using a key whose ID is 1ee2ca16-a0a1-44ab-99d9-cf9b2bd50c6d.
The information contained in the JWT header is fundamental for the JWT validation process. When we validate the JWT, we use the alg
claim to test the signature with the right algorithm, and we use the kid
claim to validate the signature against the right key.
JWT payload
The payload is the part of the JWT that contains the actual set of claims about the user. In the case of an ID token, it contains the claims that identify the user, and in the case of access tokens, it contains the claims about the right of the user to access an API.
The JWT specification (RFC-7519) defines the following list of reserved claims:
iss
(issuer): identifies the authorization server that issued the JWT. sub
(subject): identifies the subject of the JWT, i.e. the user sending the request to the server. The value of this claim must be a string. To avoid leaking user details through access tokens, the value of this claim should ideally be an opaque ID. aud
(audience): indicates the intended recipient for which the JWT was issued, i.e. the API to which the token gives access. exp
(expiration time): when the token expires. It must be a UTC timestamp. nbf
(not before): time before which the JWT must not be accepted. It must be a UTC timestamp. iat
(issued at time): when the JWT was issued. It must be a UTC timestamp. jti
(JWT ID): a unique identifier for the JWT. This can be used to prevent token reuse if it’s intercepted by a malicious agent.
None of these claims are required, however it’s recommended to include them for interoperability with third-party applications. In addition to the reserved claims, you can enrich your JWTs with custom claims according to your needs.
For a full list of additional standard claims that can be used in JWTs, check out https://www.iana.org/assignments/jwt/jwt.xhtml.
Below is an example of a typical payload:
{
"iss": "https://auth.coffeemesh.io",
"sub": "153dd667-6d97-4060-bf7a-365b94da4af8",
"aud": "https://coffeemesh.io/orders",
"iat": 1667550308,
"exp": 1667553960,
}
Let’s look at each claim to understand what’s going on here:
The
iss
claim tells us that this token was issued by the https://auth.coffeemesh.io authorization server.The
sub
claim tells us that the user’s ID is153dd667-6d97-4060-bf7a-365b94da4af8
.The
aud
claim tells us that this token gives access to the https://coffeemesh.io/orders API.The
iat
claim tells us that this token was issued on November 4, 2022, at 8:25 AM UTC.The
exp
claim tells us that the token expires on November 4, 2022, at 8:26 AM UTC.
When validating a token in the server, we must check each and every claim in the payload. We must not accept tokens that have expired, and equally we must not accept tokens whose audience isn’t our API.
JWT signature
The last part of the token is what tells us whether the token is legit or not. To produce the signature, we first encode the header and the payload using base64url encoding, then we join the header and the payload using a period as delimiter, and finally, we sign them using an algorithm of choice.
We can use different algorithms to sign the tokens, but the most common are HS256 and RS256. HS256 is a HMAC (Hash-based Message Authentication Code) algorithm. HMAC algorithms use a hashing function in combination with a secret to encrypt a value. If the token is signed with HS256, we validate the signature using the same secret that was used to sign it.
RS256 is an RSA (Rivest-Shamir-Adleman) signing algorithm. RSA uses a public/private key pair for encryption: we sign the token using the private key, and we validate its signature using the public key.
The preferred algorithm to sign JWTs is RS256. We prefer RS256 since it doesn’t require sharing a secret to validate the token’s signature - we only need the public key. The security concern here is more from a human error perspective: the secret we use to produce the signatures is highly sensitive, and sharing it around across different services, or having to use it for testing and validation, increases the risk of leaking it. With RS256, the private key stays in the authorization service and isn’t used anywhere else.
Token validation
The first piece of advice when it comes to validating JWTs is to use a good JWT library. In other words, don’t do it yourself. There’re plenty of good libraries, and this is a complex area where it’s damn easy to make mistakes. An excellent resource to learn about the available libraries and their features for different languages is https://jwt.io/libraries.
Once you’ve chosen a good library, there’re a few additional considerations which I discuss below.
First, you must validate access tokens on every request. When auditing APIs for my clients, I’ve seen cases in which the API only checks whether there’s a token in the request headers, and if the token is there, it accepts the request without further validation. This error seems to be more common among so-called internal APIs (APIs meant for internal consumption within an organization), and it’s a huge security hole. The point of including a token in the request is so that the API can validate whether the request is legit or not, and that can only be done by validating the access token.
To validate the token, we must validate both its signature and its payload. A token with valid payload claims but an invalid signature isn’t good, and neither is a token with a valid signature but with the wrong claims.
Lack of signature validation is one of the most common mistakes I’ve seen among my clients, and in fairness, even the biggest companies fall for this mistake. In January 2021, apisecurity.io reported how lack of validation of tokens’ signatures allowed hackers to access Microsoft’s Office 365 Outlook’s API with unsigned tokens. So pay extra care in this area.
When validating payload claims, make sure that:
The audience matches the API that’s receiving the token. If the audience is set for a different API, don’t accept the token.
The token isn’t expired. If the token is expired, reject it.
If the
nbf
claim is set, make sure its value is set in the past.If the
jti
claim is set, you can use it to check that tokens aren’t reused.
Inspecting tokens
When working with JWTs and debugging them, it’s useful to be able to inspect their contents. One of the most convenient tools for inspecting JWTs is jwt.io. The only thing you need to do is paste the token on the left-side input panel, and the header and the payload will show up on the left.
As you can see in the figure, you can also use jwt.io to validate the token’s signature. The token shown in the figure below was signed with RS256. jwt.io allows you to provide both the public and the private keys to validate the token’s signature, however only the public key is necessary.
You can inspect the token’s contents using the terminal’s base64
command. Using this approach, we decode the header and the payload separately. For example, to decode the header:
$ echo eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9 | base64 --decode
{"typ":"JWT","alg":"RS256"}
Other security considerations
When validating tokens, make sure the alg
claim in the header has an acceptable value. In April 2020, apisecurity.io reported that a vulnerability was discovered in OAuth0’s JWT validation, which allowed hackers to use unsigned tokens by setting the value of alg
to “nonE” (check out the details here).
A good JWT library should take care of this vulnerability by ensuring the value of alg
is restricted to a specific set of values. However, for security, make sure that alg
has a valid value. When you configure your authorization server, you decide which algorithms will be used to sign the JWTs, so in your APIs, you can check that only one of those algorithms is present in the header. [Bear in mind that, as per the JWT specification, “none” is an acceptable value for alg
. However, you don’t want to use unsigned tokens as JWTs]
Rotate your keys/secrets. As I mentioned earlier, one of the claims in the JWT header is kid
, which represents the ID of the key that was used to sign the token. It means that, at any given time, you can have a collection of keys to sign the tokens, and you can rotate those keys frequently. My recommendation is you do both. That way, if your secrets are leaked, you can simply rotate them again.
One recommendation I always give is don’t build your own authorization service. There’re very good Identity as a Service (IaaS) providers available nowadays and my recommendation is to use one of them. Securely managing identity and authorization are very complex tasks (if you think they’re not, you’re probably just unaware of the complexity). If you want to do this well, there’re complex security protocols that you need to master and implement correctly. Compared to the amount of effort it takes to build an authorization service correctly, using an IaaS is a cheaper option and allows you to focus on building your own product. Using JWTs to authorize requests To authorize a request using an access token, we include the token in a request header called Authorization
, and we precede the token by the “Bearer” keyword. This is also known as the bearer authentication scheme. For example:
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9…
The bearer authentication scheme was introduced by OAuth 2.0, and OAuth 2.1 has added additional best practices around its use. The idea of bearer tokens is to indicate that the user sending the request is the bearer of that token. Bearer tokens can be used also with opaque tokens and therefore are not exclusive to JWTs. As a side note, OpenAPI allows you to document bearer authentication schemes using the “bearer” scheme.
If you liked this article, you’ll like my book Microservice APIs. Microservice APIs teaches you everything you need to know to build microservices and APIs from the ground up, including design, implementation, best practices, principles and patterns, testing, and deployments.
You can download two chapters of the book for free from this link.
You can also use the following code to obtain a 40% discount when buying the book: slperalta.
I also run a very popular series of workshops all year round in which I teach how to build microservices and APIs, and how to write better software. The workshops range from introductory to advanced. An up to date list of workshops is always available here. Tickets usually sell fast, so if you’re interested in a workshop, I recommend you sign up as soon as possible.
One of the upcoming workshops is about API authorization. Use this link to register, and use the following code to get a 25% discount: securing-apis.
Finally, if you’re interested in my content, you may also want to subscribe to my occasional newsletter and check out my YouTube channel, in which I regularly upload coding tutorials.