As APIs are becoming the building blocks of modern software, being able to rely on them is critical. Some might even say that APIs are the next big SaaS wave. But the problem is that as APIs become more universal, integrating them is not getting any easier, on the contrary. After interviewing over 50 engineering organizations, we’ve compiled a list of the best practices we came across and the challenges faced while building API integrations.
12. Summary
Company per Stage Do you consume third party APIs?
To build your core product? Who doesn’t add new features through APIs?To add new features?
16. Don’t know when and why APIs break
Lack of proper monitoring
Managing credentials & rotation
Technical debt & dependencies
APIs are too diverse (protocol, format, etc.)
Feature disparity (sdk, webhook, etc.)
Authentication flow (OAuth 1, 1a, 2.0, etc.)
Lack of async/event-driven mechanism
Unclear potential use cases
Data leak
20. ● Thinking about API Client & SDK usage
○ Is there one? Would I use it?
■ Impact on your technical debt & dependencies
■ Features such as network retry, request identifier, timeout configuration, etc.
● Working with OAuth
○ Versions, grant types
○ Dance, token refresh, token revocation, scope change & token impact
● Using Webhook
○ Are there any?
■ Retry? log? signature?
Building API Integrations means
23. ● Gathering metrics
○ Instantly know which APIs you use
■ Monitor latency, error rate, rpm, etc.
● Producing logs
○ Log every API call request, response, headers, status code, payload, etc
○ Filter sensitive data
● Setup alerting
○ Receive alerts using custom rules based on:
■ Error rate, latency deviation, certain types of error, rate limits.
Monitoring API Integrations means
25. ● Storing credentials
○ Outside your codebase
○ Use different credentials per environment
○ Manage visibility between Developer, SRE & Security teams
● Rotating credentials
○ One-click rotation, every 6 months
○ Log trail of when & who they were updated by
● Preventing data leaks
○ Monitor critical data sent through APIs (PII etc.)
○ Setup automated alerts
Secure API Integrations means
27. ● Shielding your App from API failure
○ Perform critical API call as async
○ Setup retry logic using status code & circuit breaker algorithm
● Setup caching
○ Circuit breaker cache pattern for API failure or latency issues
○ Reduce API call cost
● Optimizing for latency
○ Optimize API network call, especially on mobile.
● Polling API
○ Poll APIs through a job/queue manager, deal with delta & retry
Optimizing API Integrations means
28. As APIs are becoming the building
blocks of modern software, being
able to rely on them is critical.
29. Takeaway
s
API Integrations are everywhere and it’s just the beginning
Building is critical but it’s only one part of the overall equation
Integration goes through the entire engineering organization
Enforce best practices and invest in tooling as soon as possible
Guillaume, CEO of Bearer, tool to help developers build & manage integration
CTO for over 10y… sorry I crossed the rubycon :D
A pain shared across the engineering organization
Sad reality, is that we lacking of best practices in this field
First thing first!
What is an API Integration
Integration = consume a bunch of APIs
Today, we are mostly going to talk about third party APIs!
Reminder
API is a tale of two worlds
Hey
..
Software is eating the world - Andreessen Horowitz
API is now the fuel
Building and consuming API are two very different things with unique challenges and perspective.
Industry focused on API Provider for too long
Time to shift FOCUS
Short answer, all of you!
48 Companies
Across all stages
Those that don’t are actually Open Source Software
Including Gitlab
Follow two key trends
API first product
Integration is the new partnership!
Integration Marketplace (Front, Intercom, Zendesk etc.)
On top of the 95%, all consume API to build poduct
79% accelerate their Growth!!
On top of the 95%, all consume API to build poduct
79% accelerate their Growth!!
As for API Provider, only company starting are not using APIs to accelerate growth
New trend with startups doing so right away
All simple standalone product have been built
Frontier is getting blurred
-> Integration starts from day-0 and gets strategic as you grow to support business
Because obviously, there are
Best Practices to help you build & manage Integrations more efficiently
Because obviously, there are
Integration has key implication for dev, ops and sec team!
We build more and more everyday, but still lacking the tools to build and manage them at scale.
Call an API is easy, building Integration is hard.