This talk will showcase how to use pyramid_openapi3 for building robust RESTful APIs. With only a few lines of code you get automatic validation of requests and responses against an OpenAPI v3 schema. You also get automatically generated “try-it-out” style documentation for your API.
It is a nice walkthrough of pyramid_openapi3. With a defense of design decisions and a few tales from the battleground.
31. Generation Validation
1. Define views and
models in Python
2. Generate
openapi.yaml by
introspecting Python
code
1. Write opeanapi.yaml
2. Define views and
models in Python
3. Validate against
openapi.yaml
“API-first”
vs
“cool” “precise”
32. Validation approach or “API-
first”
1. [API-first] incentivizes you to write the specification
so that all of your developers can understand what
your API does, even before you write a single line
of code. (source: zalando/connexion)
2. Provides a clear separation between the intent and
the implementation.
33. Validation approach or “API-
first”
3. Generation/Validation is imperfect. Much easier
escape hatch with Validation approach.
4. Several services running the same API is easy with
API-first approach.
34.
35. Validation approach or “API-
first”
3. Generation/Validation is imperfect. Much easier
escape hatch with Validation approach.
4. Several services running the same API is easy with
API-first approach
42. pyramid_openapi3
• API documentation never
out-of-date
• Try-it-out examples
• API spec is always valid
• Payload validation &
sanitation
• Responses must match
schema
• A single source of truth
Bonus karma points:
49. “See how the exact same Medium.com clone (called Conduit) is
built using any of our “supported frontends and backends. Yes,
you can mix and match them, because they all adhere to the
same API spec 😮😎”
"The mother of all demo apps"
52. Why → What →
How
Robust APIs:
Nejc Zupan / @nzupan /
github.com/pylons/pyramid_openapi3
Notas del editor
This talk is about building Robust APIs with Python.
We’re gonna look at what drives me to spend so much time and energy on designing and building robust APIs.
What Robust APIs even are
And how you can do them as well
It all started back when I was in college and Niteo was a young consulting agency.
I read about a famous slovenian businessman, how he spends half a year in Slovenia, working, and the remaining half a year kitesurfing in Brasil. One day, I thought to myself, one day …
I couldn’t get that thought out of my head for months. Then I came to a realization: There is not really anything preventing me to do it! We have international clients in Niteo, they don’t care where I am located as long as work gets done!
And so I did. I spent my first winter in Spain. Sorry, I meant Catalonia :)
It was a total blast, spending my mornings working from a sailing club in Barcelona and afternoons on the water. When it was time to go home I was convinced I need to do this again.
And sure enough, when it got properly depressing in the coming winter, as it does in our part of Europe (I don’t mind cold and snow, but I hate it when it’s cloudy, foggy and slush instead of snow), well I packed up my car and headed South.
This time to Valencia.
And I have been escaping winter ever since.
You know what sucks though? You come to the beach, the waves are pumping, the wind is howling, and then THIS
Getting called in because we have a production issue:
One of our VIP customers is claiming that our API suddenly started returning different results.
Or the app is crashing because the view code is trying to process invalid data.
AARGHHH
This is my WHY. I hate “stupid” bugs. Preventable bugs. Sure, some bugs will happen, nothing is perfect. But I hate it when they are of the kind that is totally preventable. This is why I want to build Robust APIs.
Now you know WHY. So people don’t break your schedule.
My first proper dive into the world of APIs was 7 years ago, on a PloneKonferenz.
We started the “plone.api” project during the after-conference sprints. I got to experience first-hand everything that is hard about designing APIs.
The plone.api project started because back then, Plone was already 10 years old working with Plone required memorizing a buuuunch of boilerplate code.
For example,
What we did, essentially, is survey a bunch of Plone code and identified 20% the tasks that people did 80% of the time.
Then we put them down on a piece of paper and asked ourselves how would it make the most sense to name them.
We wanted the API to be easy to remember, and easy to guess.
Then we wrote documentation on how people will use the API.
By writing the docs we really polished the naming of API methods. It might sound trivial, but naming things is one of the hardest things in programming. Especially when designing APIs, it’s really hard to change your API once you have users, without causing pain and suffering among your users.
BTW: these snippets were (and still are) tests. These bits of code are being run on every commit and asserted that they do what they claim. Fantastic for preventing regressions in a project with a great number of contributors.
the API can’t change without also changing the docs
over 60 contributors, people have given talks about it
nowadays ships with Plone by default
I haven’t touched it in years, completely taken over by the community
It turns out, if you’re building RESTful webservices API these days, you don’t have to look far. Use Swagger.
The defacto tool for describing and documenting REST APIs is Swagger.
It’s a YAML specification for your API, that generates into nice HTML documentation with an integrated test client.
And the ecosystem around it built a number of tools such as client generation tools. I.e. you can generate a React client for your API, no need to manually write API connectors.
Actually, Swagger was recently renamed to OpenAPI. If you are used to Swagger, from now on when I say OpenAPI, you translater to Swagger in your heads :)
Swagger/OpenAPI has been in in refinement for almost a decade so it’s pretty solid and not something that’s just fashionable this month.
Now you know What makes an API robust:
automatically up-to-date documentation
signature that does not change without intent
Last summer we started working on our new project WooCart.
It’s a complete-autopilot hosting for webshops built on Wordpress and WooCommerce. I was in charge of building the API that will glue together the client-facing control panel built in React (first time!) and the deployment machinery on Kubernetes.
I.e. when a client wants to create a new store, they click a button, wait about a minute, and see the new store in their WooCart Control Panel.
I knew I want to use OpenAPI specification to declare how the API will work and to have runnable documentation. So I went to swagger’s website to check what Python integration are available. And found this:
Ugh.
Almost like looking for a JavaScript packages. A bunch of overlapping packages, hard to decide which to choose.
But not really. I was quite sure I want to use Pyramid. I want to build a robust API, remember? So I can go surfing?
Yes, and Pyramid is a really mature web framework. It allows me to start small, even in a single file, and scale easily from there.
Moreover, the people that rewrote and launched the new PyPI.org last year, used Pyramid to do it. They are one of the most knowledgeable people in our community, and if Pyramid is their choice, it’s probably a good choice for me too.
Still, six packages.
Growing up, I was a huge fan of Mythbusters. I loved watching Adam and Jamie bust one myth after the other. I consider myself a tinkerer and a maker, just dying to find time to do some DIY projects.
You can imagine my joy when Adam’s first book came out recently: Every Tool’s A Hammer. Sort of an autobiography, mixed with a collection of tips on how to be a maker, how to be creative.
One of the main chapters of the book is called “Use More Cooling Liquid”. It’s also an answer to “what would you tell to 25 year old self“ that Adam gave during a recent interview. Use More Cooling Liquid.
Such a strange advice, isn’t it? Use More Cooling Liquid.
Why would Adam tell his 25-year-old self that? It’s not about the act of cutting or drilling, it’s about preparation. About using the right tools for the job and not risking damaging the tool or the piece. Of cutting in the right place, at correct dimension.
I realized that advice (next slide) applies to my use case as well.
next slide
There are two distinct approaches to doing OpenAPI with web frameworks.
2. -> code reviews are MUCH easier -> if openapi.yaml did not change, API did not change
3. NEXT SLIDE
basically the same problem, but if you are generating openapi spec from code you need to monkeypatch or fork the framework to wrestle it to produce the api spec that you want
with validation, you make a big # TODO: fix this upstream, and write your own validation in your view code, and move on
4. In WooCart we had to rewrite some of our endpoints in GoLang to get better performance & concurrency. If we would generate openapi spec with python code we would have to keep “dead” python code around, just to generate and server the spec that we need.
Going back to list of python integrations for swagger, and removing those that are not written in API-first approach I get …
… these five, three done on top of flask and two done on top of pyramid
and since I prefer Pyramid, I’m down to two
One with documentation in Japanese, the other only supporting the old Swagger 2.0, not OpenAPIv3 spec.