Security and the API Journey

If you ask most developers “What’s your favorite API?” you’ll get a variety of answers that fall into a few categories. Most people think of APIs that let them accomplish something important or an API that made them feel powerful or an API that made it all click for them. When I think of my favorite API, I think of Amazon’s Sales Tax API.

To explain why, I need to step into the ancient Internet history of 2002.

Enter the Bezos Memo

Around that time, Jeff Bezos wrote what has since become known as the “Bezos Memo.” In it, he laid out the Vision for Amazon’s internal systems and infrastructure and laid the groundwork for what would become AWS. Here are the paraphrased key points:

• All teams will henceforth expose their data and functionality through service interfaces

• Teams must communicate with each other through these interfaces

• There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.

• It doesn’t matter what technology they use

• All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.

While these principles are obvious in 2018, remember this was 2002!

Setting aside the architecture and development implications, consider this communication aspect:

By designing for externalizability from the start, teams must favor simplicity and consistency in the services they deliver.

And the resulting business implication is that any currently-internal service today may become a revenue stream tomorrow.

Calculating sales tax is a stunningly complex and yet incredibly boring operations necessary for retailers. That’s why the sales tax API is my favorite. The ability to change it from a cost center to a revenue stream is mindblowing. But how do we get from there to here?

API Security Phase 0: Private APIs

The vast majority of APIs start the same way. A team has a data source or functionality they need to share. They design a simple API with the minimally necessary functionality and share it within the team. If it solves a useful problem, other teams learn about it and request access. And then the word spreads and it turns out that nearly every team has a few APIs.

As documentation improves and adoption increases, dependencies between teams and systems explode. At this stage, teams have to systematize their practices to protect themselves and each other.

Finally, because the APIs are simple and the consumers are exclusively employees, the security considerations are simple. All of the consumers are in the same datacenter, on the same network, and can be traced to specific teams and lines of business.

API Security Phase 1: Scaling APIs Internally

At this stage, teams will evaluate and start applying devops practices to ensure their APIs are deployed consistently and reliably locally or in their cloud. Regardless of the approach, this is error prone and challenging in the best of times so most teams will use automated deployment tools such as Jenkins or Travis CI.

The most advanced - or most risk averse - teams will take it a step further and automate deploying entire servers only hosting their APIs. This architecture is considered the beginning of microservices and allows teams to scale specific components/APIs as necessary depending on the needs of the moment.

Finally, the security policies and enforcement start to evolve. Our consumers are still internal but there are more of them and they’re deployed in more places. While the policies may be configured on a per-API basis, they’re more likely centrally managed and added to the services at deploy time by Jenkins or Travis CI.

API Security Phase 2: Partners & Customers

At this stage, we know our APIs are useful because other teams are using them and most likely our partners and customers could use them too. If we open our APIs to customers and partners, we have two immediate benefits.

First, we can distribute our costs over a larger and larger group of API users. In some organizations - such as AWS or Twilio - that directly translates to revenue but at minimum it means we become important to more users.

Second, we solve more of their problems. As we demonstrate value, they integrate more closely and never consider replacing us. In the API ecosystem, companies like Salesforce and Twilio have demonstrated this this through incredibly low churn and strong growth.

Now security radically changes but most companies don’t realize it.

When we deployed APIs before, we knew the other teams using them. Whether it was a team down the hall or across campus, as employees, there was a shared understanding of proper security practices and policies. We could review their use cases and potentially even their code. In the worst case scenarios, we could call their boss to resolve the problem.

Now when we deploy APIs, we can’t simply whitelist IP addresses. We can’t assume our users are trusted or we even know who they are. We can’t assume their use cases are acceptable or something we want to allow. We can’t assume their security and data policies are compatible with our own.

Reevaluate your API Security

The dirty secret is that our security requirements never changed.

When we built our API, we believed it would be used as we planned by the people we chose for the use cases we identified.

In reality, the interfaces we design will have strengths and weaknesses we can’t predict. Some of those weaknesses will become vulnerabilities while others cause performance issues. The worst will do both. And unfortunately, the wider our API is used and the more use cases it addresses, the more likely malicious users will discover and exploit those problems.

In reality, users should only be trusted for certain actions in certain contexts for specific times. Further, API users can be malicious, unaware, or sloppy whether we know who they are or note. For example, the difference between a poorly written loop and a Denial of Service attack are in intent, not in function.

In reality, we can never predict all the use cases for our API. If we could, we would build and market those directly. For example, ten years ago no one would have predicted that you’d push a button on the super computer in your pocket and have a car appear minutes later.

API Security from Day One

The only reasonable approach for API Security is to consider it from Day One:

• Plan for logging and rate limits - your users can be malicious or sloppy, even if they’re employees on your team.
• Protect your endpoints - your API expects specific inputs and you should reject everything else. You can always expand on your use cases later but it’s nearly impossible to reduce them.
• Plan on granular permissions - a simple is_admin check is never sufficient. A vendor-neutral authorization framework such as OAuth is robust, extensible, and well understood from wide use in the API ecosystem.
• Have a plan for responding to problems - assume you will make mistakes building your API. You may leak internal data or cause performance issues that put your team, customers, or systems at risk and a consistent and predictable response is key to rebuild trust.
• Have a plan for communicating with users - assume people will need to hear from you. You will need to inform them of rate limits, policy changes, or if their keys are compromised.

For a more detailed but strategic view, check out our new book on API Security written by our Developer Relations and Product Teams. It covers everything from describing Denial of Service attacks, various authentication and authorization methods, the basics of OAuth 2.0, and an overview of API gateways.