How to design Powerful APIs via Nolan’s The Prestige !
In this blog, I am attempting to provide an incremental design guide to designing powerful APIs depending upon the stage where you or your organization is in; utilizing Christopher Nolan’s movie; The Prestige as a conduit.
What I meant by powerful API is the one that unlocks the value for the end consumer/customer, easier to manage, understand, and consume across multiple layers by the consumers of the API product.
I have divided this blog into three sections.
- The Pledge — Pragmatic API Design unlocks the real value for your end customers.
- The Turn — Well, Pragmatic API Design alone cannot unlock the real value to your end customers. It makes it even worse by letting you live in the illusion that you have unlocked the value.
- The Prestige!
The Pledge
In this section, I am going to explain — Why we still believe Pragamatic designed REST API will unlock the real value for the end customer.
Well, What the hell is Pragmatic REST API ?
The simple answer is to approach API design from the ‘outside-in’ perspective. This means we start by asking — what are we trying to achieve with an API?
As per this approach, The API’s job is to make the consumer of the API as successful as possible. The orientation for APIs is to think about design choices from the API consumer’s point of view.
You may be thinking, Isn’t this obvious ? Why are we boasting about Pragmatic REST API?
In order for us to answer this question we will have to understand a little bit of history behind REST APIs
I also believe in order to design a better present or even future, it is critical to understand the past.
Rewind to a decade back!!
In the early days of web APIs, people spent a lot of time and effort figuring out how to implement the features of previous-generation distributed technologies like CORBA and DCOM on top of HTTP. This led to technologies like SOAP and WSDL. these technologies were more complex, heavyweight, brittle, and kill the agility/speed to market.
The idea that replaced SOAP and WSDL was that you could use HTTP more directly with much less technology layered on top. Most modern web APIs are much simpler than SOAP or WSDL APIs, implemented much more lightly on top of HTTP. These APIs have come to be known as RESTful APIs.
Why RESTful API?
The HTTP specifications are some of the best-written, best-designed, and most universally accepted standards that the industry has ever seen, the chances of you inventing an alternative that will serve your users better are low.
Not all your users will have detailed knowledge of the HTTP standards, but making them learn the standard HTTP mechanisms to use your API will be a better investment for them and for you than teaching them an alternative you invented
Why HTTP as a Platform for API?
“Perfection is achieved not when there is nothing more to add, but when there is nothing more to take away”. — Antoine de Saint-Exupery,
There are other reasons than conceptual minimalism for limiting ourselves to HTTP concepts for REST APIs. One of the most important characteristics that designers of distributed APIs strive for is minimizing coupling between the client and the server. The measure of coupling is how easily either side can be changed without breaking the other.
What was the recommended Architectural Style for REST ?
In the initial days, REST was more of a Standard/Specification. Hence we followed the Dogmatic REST approach.
What is the Dogmatic REST approach?
The text book definition is
As per the Dogmatic REST approach, REST is a set of architectural constraints chosen for the properties they induce on a selected candidate architecture.
It was all about constraints. It does not place any importance on the developers or teams consuming the API.
It was not all bad; it did not scale well.
Hence Pragmatic REST approach came to the rescue.
Pragmatic approach is all about identifying Resources and rely on HTTP Methods to perform CRUD operation on that resource. They are also about well defined, well documented, standardized APIs catered to the need of API consumers and hence it was believed to be the silver bullet to unlock the value for end consumers.
The Turn
As API developers, we all go through Fake Motion Of Pragmatism by developing well-defined, documented, standard APIs catered to the need of API consumers without understanding the relationship between the resources that translated into direct value for the end customers.
For example, e-Commerce checkout is a function that acts on Cart resource and Payments resource when a checkout function is invoked. It can be poorly placed under:
instead of
These are supposed to be well-defined pragmatic APIs but fail to express ‘Resource Relationship as a Resource’ hence unlocking the underlying value of the resource.
In the above example, since we failed to express checkout function as a checkout manager, the value of checkout manager resource is shadowed under Cart and Payments; thereby creating cohesive coupling between Cart and Checkout or Payments and Checkout.
The worst thing is it has created an illusion that we have pragmatically decoupled API without solving the underlying problem.
Don’t get me wrong this is very hard to identify even for the experienced folks !
The Prestige
How to unlock the value to the customers through API design?
Well, if Pragmatic API Design is not the answer, what else is the answer ?
Pragmatic API Design is just one aspect of it. — 1
I suggest approaching from the top down within the organization or outside in. Depending upon your Organization structure, it has to be driven from Business Owners to corresponding Technical Owners to Architects/Cross-Functional Engineers to the team of engineers who own the set of APIs backed by Resources.
I will provide some pointers that worked for us:
Resources and Resource Relationship-based Design
- Determine what qualifies to be a Resource in your organization
- Determine whether the relationships between these Resources must be accessible via API.
“Usually, they are counterintuitive and often driven by business!”
- Attach a minimum set of HTTP methods to these Resources and Resource Relationship expressed as Resources
then comes the easy part
- Determine the resource name schemes based on types and relationships.
- Determine the resource schemas.
- Attach a minimum set of HTTP methods to resources.
Resources As First-Class Citizen grouped into domains!
For example, these resources can be:
It also helps us keep in check to not to get lost in details around Resources such as Actions and Extensions
HTTP API Design
- Standard Resource Representation — This includes the definition of the fields in the resources and the links to related resources.
- Standard URLs and URI templates — They define the query interface of your API for locating resources based on their data.
- HTTP Standards — The use of standard (and occasionally custom) HTTP headers.
- Client Behaviors — for example, DNS caching behaviors, retry behaviors, tolerance of fields that were not previously present in resources, and so on.
HTTP API Design is another aspect ! — 2
Try to stay away from ..
- Copy Cat Approach
- Build-Up Approach
- Standard Committee Approach
They are slow, bureaucratic, inefficient and the probability of getting us to a spaghetti state is 99%. — Aspect 3
Fourth and Final Aspect
I realized Onion Architecture is the best way to depict this where the inner layers are guided by the outer layers definition.
Layer/Tier
The outermost layer of the Onion is the Layer/Tier. This layer usually represents the Infrastructure Layer, Common Services/Platform Layer, etc. This comes from the Outer Architecture or the way an organization is set up. You will need to have these bare bones in place before proceeding further.
In case, if you don’t have this setup, you will need to identify and qualify what constitutes to be the layer.
One obvious Tip here is: Layers run as Orthognals / Horizontal within an Organizations
Domain-Driven Boundaries Layer
The layer underneath this is Domain-Driven Boundaries fronted by API Gateways utilizing API Facade Pattern. /orders, /payments/, /movies/ can be Domains.I recommend Architects or Cross-Functional Engineers at various levels need work with Product Owners or Businesses across the organization to arrive at high-level Domains. These Domains are usually the logical grouping of the Resources, discussed in Section 2.
Services /Microservices Layer
This layer can be Services/Microservices layer depending on the stage of the organization. The idea here is, they are governed by the Domain boundaries.
Business Function Layer
The layer underneath Services/Microservices layer is Business Function Layer this layer would unlock the collective value that the business needs interacting with multiple domains.
This can be visualized as follows
Final Takeaway
In summary, even Pragmatically approached APIs cannot unlock the true potential/value to the end customer. Instead, it fools us with the illusion of Pragmatism (These words are inspired by Cutter’s expressions in Prestige). I have personally found the 4 aspects of API design helped me unlock the potential/value to the end customer by organizing the relationship of resources and designing Pragmatic API around them, thereby exhibiting The Prestige.
