APIs with bounded contexts
Better design with domain-driven APIs
APIs are becoming essential components of the business and technical strategy of modern organisations. Companies using APIs have a competitive edge, but, how far does the competitive advantage of APIs go? What determines the business and technical value of an API program? What decides its success or failure?
Two crucial ingredients of API success are 1) alignment with the organisation’s overall business strategy and 2) usability. Aligning an API with the business strategy means the API must support the business to achieve its goals, while usability means the API is easy to consume. There are many more elements to API success, of course, but in this article I want to focus on these two, and I’ll argue that domain-driven design is a great method for creating successful APIs.
When APIs don’t align with the business
A few years ago, I had the opportunity to work on a project to improve forecasting operations for a large organisation. We created tools to improve the process of forecasting sales, warehouse space requirements, or calculating the business impact of opening new stores. Many of these tools were backed by machine learning models, and we used APIs to expose the tools to our stakeholders.
What started out as a simple API modelling process quickly turned into an API hellscape. We ended up in this situation because we didn’t take a structured approach to API design from the beginning. Instead, we let the API grow “organically”, or rather, to sprawl. Whenever we needed to add a new feature, we added a random new endpoint, or we re-used an existing one. Meanwhile, our API schemas grew in complexity to accommodate different entities and states, to the point that they became useless for data validation or for API mocking.
A major driver of this strategy was the difficulty we encountered to model business operations. It became obvious that the organisation lacked specific terminology for many of its processes and flows. Stakeholders explained to us the steps to produce a sales forecast, but every step was loosely defined. And whenever we requested a new explanation, there were slight variations in the description of their process.
Lack of precision suited the business, since it meant they could easily change and adapt the process to their changing needs. But it also meant the process was very difficult to model with software. Furthermore, teams in different countries often used completely different processes, so what we modelled for one team didn’t quite work for another.
To get our APIs back on track, we decided to take a step back, spend more time talking to the business to understand their processes, and work harder on aligning our software with them. This exercise helped the stakeholders become more aware of their own processes and understand them better. We also figured out that, when modelling business processes with software, sometimes you need to have a bi-directional conversation: the business informs the design of the software, but the software can also influence improvements in the business process. All along, we found domain-driven design a very useful tool to guide our conversations and modelling efforts.
Aligning APIs with business
How do we align our API with the business while making it usable? We want to make sure the API models the processes and flows of the business, and that it talks the language of the business. Domain-driven design is the best tool to make this happen.
Domain-driven design is a software modelling methodology that helps us align software and business. When we say “business”, we mean the domain we’re trying to model with software. It applies to both technical and non-technical domains. The point is to make sure we model our software after the language our stakeholders use. If our stakeholders work in the hospitality industry, our software will use the hospitality industry’s language; if our stakeholders work in car manufacturing, our software will use the car manufacturing’s language.
One of the greatest challenges in domain-driven design is achieving precise definitions of each process, flow, and concept the business uses. The less technical the business (or perhaps the less “mechanical”), the more ambiguous its concepts can be. For example, concepts like “trust” or “liability” can be difficult to model, and often we need to resort to metaphors to build our mental models.
Over the years, I’ve helped many organisations in their API journey, and API modelling has been a constant source of friction. For one, API modelling is difficult. I’d go as far as to say that it’s slightly more difficult than software modelling (I’ll explain why). And secondly, most organisations prefer to “move fast” rather than spend time modelling their APIs.
However, failing to take API design and modelling seriously makes APIs prone to failure, makes them vulnerable, and makes them more difficult to manage and evolve in the future. It also makes them difficult to consume. Hence why I spend a significant amount of my time evangelising organisations on the importance of prioritising API design and modelling.
Applying DDD to APIs is challenging
As I mentioned before, it’s challenging to apply domain-driven design to APIs. APIs don’t have a concept of “flows” or “processes”. They expose endpoints and operations, and they help us exchange data with the server. A lot of the effort we spend designing APIs goes into ensuring that our schemas are sufficiently robust to validate incoming and outgoing data. Because of the way APIs work, we can’t add behaviour or additional functionality to our schemas. For this reason, sadly APIs are often seen as simple “data gateways” rather than as interfaces to service capabilities.
But it doesn’t have to be this way! Despite the limitations, we can use heuristics to model processes and flows in our APIs. Let’s consider the case of REST APIs. REST APIs are resource-oriented, which means every endpoint represents a resource. On the face of it, this constraint means we can only represent entities through REST endpoints. However, using heuristics, we can model operations through REST endpoints, and by combining those operations, we can represent processes and flows. Let’s see how we do that.
Applying DDD to API modelling: an example
Imagine we want to model the process of taking a payment through our API. Paying is an operation, not an object or resource, so how do we model a payment in REST? We can do it by creating a POST /pay endpoint. The semantics of a POST request indicate that we want to create a resource or send data for processing, hence it’s suitable to represent payment processing. Since it’s a POST endpoint, this operation will create a payment object in our server; it’ll process the payment, and it’ll return the outcome of this operation to the user. Using this heuristic, we can model nearly every business operation through the API.
What about flows? A flow represents the steps or processes that take place to produce a business outcome within a particular domain or bounded context. For example, we use flows to represent how a sale takes place in an e-commerce website. Let’s say the flow to purchase an item looks like this:
Customer adds items to the cart.
Customer proceeds to check out.
After applying coupons, discounts, and so on, the system calculates the total.
Customer selects a payment method and enters payment details.
System processes payment.
System confirms the order.
How do we model these operations through an API? How do we represent them as a flow? Since flows take place within a bounded context, the first thing we want to do is figure out a way to create a bounded context in our API. The simplest way to do that is using nested URL paths.
For example, we could represent the process of adding items to the cart and checking out with a POST /carts endpoint. When calling this endpoint, the customer sends the items they want to buy, plus any coupons or discount codes they want to apply. When the API receives this request, it creates a cart resource and it returns to the customer the cart ID plus the total price.
To pay, the user calls the POST /carts/{cartId}/pay endpoint. When the API receives this request, it creates a payment record and processes the payment. If payment is successful, the user receives a 201 response with the order confirmation.
From this design exercise, we obtain the following endpoints:
How do we represent the relationships between these operations to create a flow? We can use hypermedia links or OpenAPI links. OpenAPI links are more powerful, since they are natively supported by the OpenAPI specification and allow us to create relationships between a collections endpoints (e.g. POST /carts) and a singleton endpoints (e.g. GET /carts/{cartId}). Here’s how we describe the purchasing flow using OpenAPI links:
There’s an implicit sequence of steps in the links. We use the `id` property from the POST /carts response payload to form the /cats/{cartId}/pay URL, which means POST /carts is a necessary first step. This is a very simple example, but the same approach can be used to document complex processes and flows.
But you’re violating the design principles of REST!
Am I? A common critique to the heuristics I’ve shown in the previous sections is that they violate the design principles of REST and turn the API into a de facto RCP interface. Some argue that REST APIs aren’t suitable to model business processes and flows, and that if you want to do that, you need to use other types of APIs.
The main challenge here is that REST is resource-oriented. Through REST endpoints, you represent collections of entities or singleton entities. Clients use the API to create resources or alter their state, and the API gives you back the resource’s representation. Hence, the argument goes, REST is pretty much limited to exchanging data between the server and the client.
I think this is a very limited interpretation of REST. REST is resource-oriented, but it doesn’t have a strong definition of what a resource is. In the heuristics I’ve shown before, we create resources all the time: our payments endpoint creates a payment record/resource, and our checkout endpoint creates a cart record/resource. We create resources while enabling processes and flows.
APIs serve as interfaces for our services. We use APIs when we want to expose the capabilities of our services through the network. To do this well, we must use best practices in API design and abide by their constraints. Best practices and constraints exist for a reason: they’re the result of many years of trialing and experimentation, of making mistakes and finding out what works best. There’s good reason to use best practices and constraints. At the same time, we must also use heuristics to figure out how to apply those best practices to our use cases. It’s by no means easy, but it’s worth every bit.
Additional resources
I presented an earlier version of this paper in my presentation “APIs with Bounded Contexts” at API World 2022. Check out the slides for more details.
For an insightful and pioneering overview of how domain-driven design can help and empower the API design process, check out James Higginbotham’s Principles of Web API Design. Another great resource that encourages a DDD approach to API design is Patterns for API Design, by Olaf Zimmerman, Mirko Stocker, Daniel Lübke, Uwe Zdun, and Cesare Pautasso. Patterns for API Design contains a great collection of patterns for modelling processes and flows through APIs.
For a comprehensive overview of API design, check out Arnaud Lauret’s The Design of Web APIs. David Biesack’s newsletter API Design Matters is also a rich source of ideas and information about good API design.
If you want to learn more about API design heuristics for REST/OpenAPI, check out Josh Ponelat and Lukas Rosenstock’s Designing APIs with Swagger and OpenAPI. Make sure you check out Lukas’s talk “From Domain Model to API” for more examples of heuristics.
For ideas on modelling complex concepts like trust and liability, check out Martin Verraes and Rebecca Wirfs-Brock’s article “Models and Metaphors”.
If you liked this post, you’ll like my book Microservice APIs. Use the code slperalta to obtain a 40% Manning’s website. You can also get the book on Amazon.com.
I’m on a mission to help organisations deliver reliable and secure APIs and microservices. To this end, I run public webinars (check out my upcoming “Fundamentals of API Security” free webinar), I publish content regularly in this newsletter, and I maintain open source projects like fencer, an automated API security testing tool. I also streamline my consulting efforts through microapis.io. If you work with microservices and APIs, don't hesitate to get in touch directly or arrange a call!