Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Java-Jersey 到 Python-Flask 服務不中斷重構之旅

937 visualizaciones

Publicado el

PyCon Taiwan 2018
Code: https://github.com/cclai999/pycontw2018-demo
Demo: http://bit.ly/2kAIgMo

Publicado en: Tecnología

Java-Jersey 到 Python-Flask 服務不中斷重構之旅

  1. 1. Java-Jersey Python-Flask Max Lai 2018/06/01 PyCon Taiwan 2018
  2. 2. About Me • Taichung.py • Agile Taichung • Agile Tour Taichung 2016~2018 • (Jarvish Inc.) PyCon Taiwan 2018
  3. 3. Agenda • • RESTful Framework • Flask vs Django REST framework • SQLAlchemy MySQL DB • API Documentation by Swagger • RESTful API • Tavern RESTful API • (Canary Release) API A/B test • Q&A PyCon Taiwan 2018
  4. 4. • http://bit.ly/pycontw2018-flaskSlides • https://github.com/cclai999/pycontw2018-demoCode • http://bit.ly/2kAIgMoDemo PyCon Taiwan 2018
  5. 5. • Jarvish AI • back end lead Azure DevOps • back end Java-Jersey, Scala-Akka Project PyCon Taiwan 2018
  6. 6. ) ( PyCon Taiwan 2018 X
  7. 7. Before: PyCon Taiwan 2018 Nginx Java/Jersey Scala/Akka Redis MySQL RESTful API API Set-A API Set-B
  8. 8. Ongoing: PyCon Taiwan 2018 Nginx Scala/Akka Redis MySQL RESTful API API Set-A API Set-B Python/Flask API Set-C Python/Flask
  9. 9. After: ( ) PyCon Taiwan 2018 Nginx Redis MySQL Python/Flask Python/Flask Docker RESTful API Python/Flask
  10. 10. RESTful Framework Flask vs. Django • Flask is light, with extensions • Django is large, batteries included • Both are good • Which one do I choose? • depend on the context PyCon Taiwan 2018
  11. 11. Hello World in Flask PyCon Taiwan 2018 from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return 'Hello World!' $ FLASK_APP=hello.py flask run #start flask server #web server in a file
  12. 12. Hello World in Django PyCon Taiwan 2018 $ django-admin startproject mytsite $ cd mysite $ python manage.py startapp hello # edit mysite/settings.py INSTALLED_APPS = [ ... 'hello', ] # creating project hello
  13. 13. Hello World in Django PyCon Taiwan 2018 # edit hello/views.py from django.http import HttpResponse def index(request): return HttpResponse("Hello World!")
  14. 14. Hello World in Django PyCon Taiwan 2018 $ python manage.py runserver # edit mysite/urls.py from django.conf.urls import include, url from hello import views urlpatterns = [ url(r'^$', views.hello), ] #start web server
  15. 15. RESTful API PyCon Taiwan 2018 HTTP Method Action Examples GET Obtain information about a resource http://[hostname]/todo/api/v1.0/tasks (Retrieve list of tasks) GET Obtain information about a resource http://[hostname]/todo/api/v1.0/tasks/1 (Retrieve task #1) POST Create a new resource http://[hostname]/todo/api/v1.0/tasks (Create a new task, from data provided with the request) PUT Update a resource http://[hostname]/todo/api/v1.0/tasks/1 (update task #1, from data provided with the request) DELETE Delete a resource http://[hostname]/todo/api/v1.0/tasks/1 (delete task #1)
  16. 16. API in Flask PyCon Taiwan 2018 from flask import jsonify @app.route('/todo/api/v1.0/tasks/<int:task_id>', methods = ['GET']) def get_task(task_id): task = filter(lambda t: t['id'] == task_id, tasks) if len(task) == 0: abort(404) return jsonify( { 'task': make_public_task(task[0]) } ) http://localhost:5000/todo/api/v1.0/tasks/1 Serialization: Flask-Marshmallow with SQLAlchemy, MongoEngine, etc
  17. 17. API in Django • Django REST Framework is great! • Browsable API: documents API with a human-friendly HTML output • Auth Support: rich support for various authentication protocols along with permissions and throttling policies • Serializers: converting querysets/instances to native Python datatypes that can be easily rendered into JSON and XML • Throttling: generally used for rate limiting API requests from a single user • It is subject to same restrictions as Django PyCon Taiwan 2018
  18. 18. RESTful API API biz logic Template, Admin PyCon Taiwan 2018
  19. 19. Flask MySQL DB PyCon Taiwan 2018
  20. 20. Flask MySQL Schema PyCon Taiwan 2018 mysql_db_url="mysql://maxlai:pycontw2018@127.0.0.1/demodb" engine = create_engine(mysql_db_url, echo=False, pool_recycle=3600) Base = declarative_base() Base.metadata.reflect(engine) db_session = scoped_session(sessionmaker(bind=engine)) class Todo_Tasks(Base): __table__ = Base.metadata.tables['todo_tasks']
  21. 21. Flask-SQLAlchemy (OperationalError) (2006, 'MySQL server has gone away') PyCon Taiwan 2018 MySQL features an automatic connection close behavior, for connections that have been idle for a fixed period of time, defaulting to eight hours. source: http://docs.sqlalchemy.org/en/latest/dialects/mysql.html?highlight=mysql
  22. 22. 1 • pool recycle . ex. 3600 sec. PyCon Taiwan 2018 mysql_db_url="mysql://maxlai:pycontw2018@127.0.0.1/demodb" engine = create_engine(mysql_db_url, pool_recycle=3600) source: http://docs.sqlalchemy.org/en/latest/dialects/mysql.html?highlight=mysql ‘MySQL server has gone away’
  23. 23. 2 connection test, “SELECT 1” PyCon Taiwan 2018 source: http://docs.sqlalchemy.org/en/latest/core/pooling.html#pool-disconnects @event.listens_for(engine, "engine_connect") def ping_connection(connection, branch): if branch: return save_should_close_with_result = connection.should_close_with_result connection.should_close_with_result = False try: connection.scalar(select([1])) except exc.DBAPIError as err: if err.connection_invalidated: connection.scalar(select([1])) else: raise finally: connection.should_close_with_result = save_should_close_with_result
  24. 24. 2(cont.) • SQLAlchemy v1.2 PyCon Taiwan 2018 source: http://docs.sqlalchemy.org/en/latest/core/pooling.html#pool-disconnects mysql_db_url="mysql://maxlai:pycontw2018@127.0.0.1/demodb" engine = create_engine(mysql_db_url, pool_recycle=3600, pool_pre_ping=True)
  25. 25. API documentation by Swagger • API • PyCon Taiwan 2018
  26. 26. API documentation by Flasgger(Swagger) • Swagger API HTML Javascript • OpenAPI Specification ( Swagger Specification) RESTful API • JSON or YAML API spec Flasgger SwaggerUI • Marshmallow APISpec Flasgger specs base template • Demo http://flasgger.pythonanywhere.com/ PyCon Taiwan 2018
  27. 27. • Martin Fowler • infrastructure level PyCon Taiwan 2018
  28. 28. • ( ) • • • API • • PyCon Taiwan 2018 Nginx Java/API Set-ARedis MySQL API Flask/API Set-A API
  29. 29. Why Tavern • Why not Postman, Insomnia? • focus on automated testing of APIs • test syntax is more intuitive (YAML format) • easy to write for most people • Why not pyresttest? • it is no longer actively developed. PyCon Taiwan 2018
  30. 30. API returns the double of a number PyCon Taiwan 2018 @app.route("/double", methods=["POST"]) def double_number(): r = request.get_json() try: number = r["number"] except (KeyError, TypeError): return jsonify({"error": "no number passed"}), 400 try: double = int(number)*2 except ValueError: return jsonify({"error": "a number was not passed"}), 400 return jsonify({"double": double}), 200
  31. 31. Tavern RESTful API PyCon Taiwan 2018 test_name: Make sure server doubles number properly stages: - name: Make sure number is returned correctly request: url: http://localhost:5000/double json: number: 5 method: POST headers: content-type: application/json response: status_code: 200 body: double: 10
  32. 32. Tavern RESTful API PyCon Taiwan 2018
  33. 33. (Canary Release)1 • Canary • • 17 • PyCon Taiwan 2018
  34. 34. (Canary Release)2 PyCon Taiwan 2018 source: http://bit.ly/2seHrwg
  35. 35. (Canary Release)3 PyCon Taiwan 2018 source: http://bit.ly/2seHrwg
  36. 36. (Canary Release)4 PyCon Taiwan 2018 source: http://bit.ly/2seHrwg
  37. 37. Nginx Upstream PyCon Taiwan 2018 Nginx Java/API Set-ARedis MySQL Flask/API Set-A 80% 20%
  38. 38. Nginx PyCon Taiwan 2018 http { upstream demoweb { server web1:8000; # demo web server1 server web2:8000 weight=4; # demo web server2 } location ~ ^/demo/api/v1.0/servername/(.*)$ { proxy_pass http://demoweb/demo/api/v1.0/servername/$1; # Redefine the header fields that NGINX sends to the upstream server proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; …… } }
  39. 39. Demo PyCon Taiwan 2018
  40. 40. Takeaways PyCon Taiwan 2018 Flask
  41. 41. Taichung.py Agile.Taichung PyCon Taiwan 2018

×