The future of APIs is promising and bleak
APIs are everywhere. Organisations use APIs more and more to tap into new business opportunities, automate internal processes, and to drive integrations between internal services. The way modern applications are built means APIs are pervasive and inescapable. According to Postman’s 2023 State of API of the API Report, 65% of developers spend over 10 hours a week working with APIs. And yet, we haven’t really figured out how to work effectively with APIs.
This post is part of a series of publications on the future of APIs, hosted by Matthew Reinbold. Check out the amazing publications within this series on APIFutures.
The problem
A few years ago, I helped a client build a partner API. I got to work both on client and server-side code. I worked alongside five other developers, and what started as a fun project quickly turned into a mess. Documentation practices were poor, the time spent on API design was nearly nil, and proper API testing wasn’t even considered.
I want to believe that we’ve come a long way from those days. I certainly try to bring with me better workflows whenever I get to work with a new client. Like many other API practitioners, I advocate better practices in my talks and publications. And yet, when I look around, the majority of APIs are still being poorly developed. Joyce Stack put it eloquently in a recent publication: “FFS every conference feels like an overwhelming overview of all the things we are NOT doing.”
This has very practical consequences. Developers often struggle to deliver their APIs, the APIs fail to deliver business value, and eventually, they become a maintenance nightmare and a security liability. One of my clients recently confided how they struggle to keep their APIs correctly documented and how that’s costing them so much business - their customers can’t put up with wrong API documentation anymore, so they’re leaving in droves.
As we walk into 2024, APIs are more ubiquitous than ever, and the question is: what can we do to make our lives easier when we work with APIs? What are the biggest challenges organisations face in the API space in 2024?
There are many things we can do to improve our API development experience. But in my experience, most organisations will make major improvements if they focus on two areas: accountability and security. Let’s break it down.
Accountability
Accountability is probably a weird word to use in the context of APIs, but I couldn’t come up with something better. Here’s what I mean: when we decide we’re going to build an API, we have some expectations about the API. On a business level, we expect the API to deliver some value. At the technical level, we expect the API to satisfy a certain interface. I want to focus on the technical aspect here.
How many times have you set out to build an API and delivered exactly what was expected? If you say many, you’re probably lying, and your colleagues are exposing you on Reddit. No shame tho, APIs are difficult and it’s very difficult to get them right in the first pass.
When our frontend developers complain that backend developers don’t deliver the right APIs, two things are happening: 1) we didn’t have a clear spec, and 2) we didn’t test properly. If we can solve these two problems, most organisations will eliminate most of their API development issues. It’s a bold statement, but I work with many different kinds of organisations, and most of the API development bottlenecks I see revolve around this problem.
We can fix this problem in many different ways, but let’s go with the simplest. What we need is:
A clear idea of how the API must work
A formal API specification
Validate the server implementation against the API specification
Step (1) is a design problem - we’ve got to sit together and discuss how we want the API to work. Ideally, we discuss the expected user flows, the expected data exchanges, and so on. Keep it simple. At the core, we’re discussing business logic here and how to translate it into an interface. Also, these designs will evolve over time, so don’t overthink it.
Once we agree on a design, we must be able to build client and server against that design. For that, we need a formal spec (step 2). Why? Because a formal spec is the only way to accurately describe an API. You know the drill: REST APIs -> OpenAPI, GraphQL APIs -> Schema Definition Langauge, gRPC -> Protocol Buffers, etc.
Use the most convenient method for you to produce the spec. You don’t have to write YAML by hand if you don’t want to. I like to use self-documenting frameworks like FastAPI to produce a quick API spec and then edit it by hand. Once you have a spec, confirm with the team it looks good, make any desired changes, and put it somewhere where both client and server developers can reference it. Ideally, a Git repository where you can track all the changes.
Finally, it’s time to build. We build the client and the server following the agreed spec. At this point, we will find constraints in the original design, so we go to the spec’s repo, propose some changes, merge them, and make sure everybody is aware of them.
Now, how do we ensure the API is correctly implemented? And as well, how do we ensure that the server doesn’t change the API without anyone noticing? In other words, how do we hold the backend developers accountable? We test the implementation against the spec. I recommend using fuzzy testers like schemathesis to validate the server implementation.
If you follow these steps, you’ll reduce your API integration problems significantly. You may find it even a joy to work with APIs.
Now let’s talk about security.
Security
APIs are special beasts. The security profile of APIs is unlike that of traditional web applications, so much so that OWASP (Open World Application Project) created a list of specific API threats in 2019 (updated in 2023). A lot of API security problems revolve around authentication and authorisation, and our API designs can also expose vulnerabilities in ways we didn’t expect. I spent 2023 researching API design vulnerabilities - you can check out a summary of my findings here and I’ll soon publish a website dedicated to this topic under apivulnerabilities.com (under construction atm).
Sadly, API security is a very poorly understood topic, and most organisations expose vulnerable API designs, even in sensitive sectors such as fintech. Unsurprisingly, APIs are now the main vector of attack in web applications. If CEOs understood how easy it is to introduce vulnerabilities in APIs, APIs wouldn’t be as pervasive as they are today.
Does this mean there’s nothing we can do to protect our APIs, we should accept our fate, and keep CEOs in the dark about the risks? No, not really. The good news is, we can address a lot of API vulnerabilities by hardening the design. There is a lot we can do to improve the security of our APIs, and here I want to suggest three quick actions any organisation can take to improve their security posture:
Have very accurate API documentation
Test the security of your API design
Run fuzzy testers against the API
Step (1) should be nearly done if you followed my recommendations in the previous section. What I want you to do here is go a little further in the documentation and be as specific as you can in the shape and format of your data. Constrain user input as much as possible. Can that URL query parameter be an enumeration or a regular expression? Should your numerical parameters allow negative values? The more you can constrain user input, the fewer chances your users have to abuse your APIs.
Once you’ve gone through the previous step, it’s time to test how secure your API design is (step 2). You can accomplish this with open-source tools like spectral and its OWASP plugin. If you run spectral against your API specification, you’ll probably uncover dozens or even hundreds of vulnerabilities. Don’t fret - this is how it’s supposed to work. Run spectral early on in your API development cycle and you’ll provide nasty surprises.
Finally, it’s also a good idea to test how secure your implementation is, and that’s where step (3) comes in. I already recommended fuzzy testers in the previous section, and I just want to highlight that fuzzy testers can uncover a great deal of vulnerabilities and potential attack vectors in your API.
As a bonus, I’d recommend running a test suite with a proper API security testing tool. There are many API security testing tools available from vendors, and I’m currently working on an open-source API security testing tool called fencer.
Two more things:
Many API breaches happen due to authorization vulnerabilities. Hardening your API design will not fully protect you against this. You must follow best practices for authentication and authorization and apply the correct access controls across all your endpoints. Unfortunately, generic tools won’t go far in this area. If in doubt and seriously concerned, hire a penetration tester.
It’s great to harden our APIs, but despite our best efforts, hackers will always find ways around it. Security incidents are not a matter of if, but when. Be prepared for the worst, and continuously monitor your APIs. Observability is a key ingredient in good API security.
Conclusion
Last year I discussed a business partnership with a software consulting company. When they asked me to explain my value proposition, I emphasized my experience and expertise in the field of APIs, to which they countered that, in their view, “APIs are well-understood technologies” and hence didn’t see much value in my proposition. Their statement echoes a widespread sentiment in our industry - that APIs are simple technologies that need little thought and planning.
However, APIs are deceptively simple. Sure enough, anyone can build an API. But it takes a lot of effort to deliver a well-designed, well-implemented, and properly secured API. I think this is where we tend to get it wrong. This is what turns our APIs into liabilities instead of opportunities.
As we walk into 2024, I hope we can improve the quality and security of our APIs, and that working with APIs becomes a fun experience. It really isn’t that difficult, and if you’ve made it so far, you now know how to make it work.
I hope you found this article useful. If you have any questions or comments, I’d love to hear from you! You drop a comment here or write to me at jose@microapis.io.
I’m on a mission to help organisations deliver reliable and secure APIs and microservices. To this end, I run public webinars and workshops, teach courses, publish content regularly in this newsletter, and maintain open-source projects like fencer, an automated API security testing tool. I also streamline my consulting efforts through microapis.io. If you work with microservices and APIs, don't hesitate to get in touch directly or arrange a call!