These presentation slides are what will be presented at Pycon Korea 2018. They explain what microservices architecture is and Why company 'mymusictaste' moved from monolithic application to microservices architecture. The slides also explains how we built microservices with Python and utilized AsyncIo and Sanic. We built our own framework to add features on Sanic to meet our needs. This is to share our experiences transforming to micro services architecture as a startup company with everyone at Pycon Korea 2018.
6. Microservices
-
-
-
-
A collection of loosely coupled services
Parallelizes development by small autonomous
teams to development
scale their respective services independently
polyglot
developed by Python
developed by Go
more resilient to
architecture erosion
easier to understand,
develop, test
14. AsyncIO
I/O bound
CPU bound
A B A
single-threaded concurrent code
B
cooperative multitasking
A B
A
what if B is not cooperative?
15. “So microservices use a distributed system to improve modularity. But distributed
software has a major disadvantage, the fact that it's distributed.
The second mitigation is to use asynchrony. If make six asynchronous calls in parallel
you're now only as slow as the slowest call instead of the sum of their latencies. This can
be a big performance gain, but comes at another cognitive cost. Asynchronous
programming is hard: hard to get right, and much harder to debug. But most microservice
stories I've heard need asynchrony in order to get acceptable performance.”
Martin Fowler
16. SANIC
- Sanic is a Flask-like Python 3.5+ web server
https://github.com/channelcat/sanic
from sanic import Sanic
from sanic.response import json
app = Sanic()
@app.route('/')
async def test(request):
return json({'hello': 'world'})
if __name__ == '__main__':
app.run(host='0.0.0.0', port=8000)
20. Settings
Settings for Insanic differ a lot from sanic's config pattern. While implementing, I found the need to access the settings
from modules where the sanic application was not accessable. Thus the settings object in insanic takes a lot from django's
settings configuration where the settings object is a single instantiated config object that can be imported anywhere.
In addition to sanic's config object, where it is usually instantiated and attached to sanic's application instance, the
settings object in insanic is a lazy loaded with dependencies on the service name.
Settings Loading
There are several steps/places the settings object loads settings from.
1. Loads the global_settings.py from within insanic. These are the default settings that insanic needs to run.
2. Then with the SERVICE_VARIABLE, tries to load config.py from within the project.
3. common settings are loaded from VAULT
4. service settings are loaded from VAULT
5. Any environment variables are loaded. Must be prefixed with set prefix. (default its INSANIC)
INSANICINSANIC
21. View Authentication and Permission Handling
Insanic takes the original views from sanic and modifies them to handle authentication and permission handling. A lot of
patterns were taken from Django Rest Framework. The bulk of the updates is through the dispatch_request method. To
register authentication and permissions, we must first create or use the general authentication and permission provided
by insanic
Views
# views.py
from sanic.response import json
from insanic import permissions, authentication
from insanic.views import InsanicView
class GottaGoFastView(InsanicView):
permission_classes = (permissions.AllowAny,)
authentication_classes = (authentication.JSONWebTokenAuthentication, )
async def get(self, request, *args, **kwargs):
return json({"how fast?": "insanely fast"})
Permissions
- AllowAny
- IsAuthenticated
- IsAdminUser
- IsAuthenticatedOrReadOnly
- IsOwnerOrAdmin
- IsServiceOnly
INSANIC
22. Request Object
Apart from the request attributes provided by the sanic request object, insanic creates additional attributes with a lot of
inspiration from Django-REST-Framework but with async compatibilities.
Request Parsing Authentication
.data
.query_params
.user
user = await request.user
.service
service = await request.service
INSANIC
23. Inter Service Communications
One of the core additional features of insanic, that differentiates it from other frameworks, is the Service object.
Service Features
• Resolve address and endpoint construction
• Header preparation and injections for https requests
• Error handling
• Circuit Breaker patching
• Response handling
from insanic.loading import get_service
ArtistService = get_service('artist')
response, status_code = await ArtistService.http_dispatch('GET', '/api/v1/artists/',
query_params={"query": "insanic"},
include_status_code=True)
INSANIC
24. Pytest helpers
Public Facing API Gateway Registration
Tracing with AWS X-Ray service
Addition of these features are ongoing
- Contract test
- OpenAPI 3.0 specification document generation
- inter service communication with RPC option
INSANIC
26. In… series / wrapper for asyncio
aiohttp
Asynchronous HTTP client/server for asyncio
aioelasticsearch
elasticsearch-py wrapper for asyncio
asyncpg
A fast PostgreSQL database client library
aioredis
Asyncio Redis support
InPynamoDB
This transforms PynamoDB’s basic methods working
asynchronously used aiobotocore.
Infuse
This is heavily based on pybreaker. For full
documentation refer to pybreaker. What is different from
pybreaker is that it includes asynchronous storage
options.
async-python-mailchimp-api
A straighforward python asynchronous client for v3 of
MailChimp API using aiohttp >= 3.0.0. This Project
forked from python-mailchimp
CHALLENGES
28. PLAN
1. Insanic
(2018. 8. 6. Release)
2. Insanic Open source
Main developer for Insanic
Kwang Jin Kim
https://github.com/MyMusicTaste/recruit
We are hiring!