Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3

138 visualizaciones

Publicado el

Доповідь Максима Пугача, Team Lead/Software Engineer at LITSLINK, на Elixir Evening Club 3, Kyiv, 13.12.2018
Наступна конференція - http://www.elixirkyiv.club/

A boss of mine once told me "Just see, my poorly written Vert.x app outperforms my poorly written Elixir app". Now it is time to take up the gauntlet.

Cлідкуйте за нами у соцмережах @ElixirClubUA та #ElixirClubUA
Анонси та матеріали конференцій - https://www.fb.me/ElixirClubUA
Новини - https://twitter.com/ElixirClubUA
Фото та невимушена атмосфера - https://www.instagram.com/ElixirClubUA
*Канал організаторів мітапа - https://t.me/incredevly

Publicado en: Software
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Performance measurement methodology — Maksym Pugach | Elixir Evening Club 3

  1. 1. Performance measurement methodology
  2. 2. Maksym Pugach
  3. 3. A couple of years ago in a company, somewhere in the city….
  4. 4. Eclipse Vert.x is a polyglot event-driven application framework that runs on the Java Virtual Machine
  5. 5. Vert.x promises for our case ● Just wrap your existing code into a Vert.x and it works (JS and Ruby in our case) ● More performant than other solutions
  6. 6. The Test Environment Google Cloud Platform 1. K8S services - 1 vCPU 3.75GB Ram 2. MongoDB - 1 vCPU 3.75GB Ram 3. Target(elixir-cowboy, elixir-elli, vertx-ruby) - 2 vCPUs 7.5GB Ram 4. Tester(ab, wrk) - 4 vCPUs 15GB Ram vCPU implemented as a single hardware hyper-thread on Intel Xenon Skylake 2.7GHz
  7. 7. The Test Environment Vert.x 3.5.4 (official Docker image) Elixir 1.7.4 (official Docker image) Cowboy 2.6.0 Elli 3.1.0 Jiffy 0.15.2 (NIF JSON parser) Plug … not used
  8. 8. Tools and Methodology ab - ApacheBench 2.3 Pick concurrency number which results better requests per second Iterate same test while last two tries are better than any other Pick the best
  9. 9. GET /static serve a static JSON string 1000 000 times Concurr ency Req/sec Time (s) CPU load (%) RAM (Mb) Comment Elixir Cowboy 10 4889.53 204.519 200 490 4807.77 rps on concurrency 30 Elixir Elli 30 9198.34 108.715 200 490 Not stable, +/- 500 req/sec Vertx Ruby 30 9667.33 103.441 170 820 Not stable, +/- 500 req/sec; CPU cores are not fully loaded even if performance start to degrade on high concurrency. Vertx Ruby 30 11176.91 89.470 180 820 “/blocking_static” endpoint
  10. 10. GET /mongo get 20 documents, serialize them and server JSON string 100 000 times Concurr ency Req/sec Time (s) CPU load (%) RAM (Mb) Comment Elixir Cowboy 20 1724.53 57.987 200 500 1656.11 rps on concurrency 30 Elixir Elli 20 2155.92 46.384 200 500 2058.08 rps on concurrency 30 Vertx Ruby 30 2135.98 46.817 150 920 Vertx Ruby 30 2200.38 45.447 150 920 “/blocking_mongo” endpoint
  11. 11. POST /sum post ‘{"a":10,"b":20,"iterations_count":1}’, calculate the sum and return serialized JSON string 100 000 times Concurr ency Req/sec Time (s) CPU load (%) RAM (Mb) Comment Elixir Cowboy 42 4521.85 22.115 200 480 Elixir Elli 42 8302.44 12.045 200 500 Vertx Ruby 36 6725.28 14.869 150 1320 6574.72 rps on concurrency 42 Vertx Ruby 42 10884.58 9.653 195 1075 “/blocking_sum” endpoint
  12. 12. I never heard of Vert.x before, but seeing that it’s based on non-blocking event-based approach and runs on Java, I’d guess that it suffers from issues such as cooperative scheduling, anonymous implicit activities, and callback hell. Here’s one way how we could compare Elixir (or any other BEAM language) against that. Make a simple server which has two requests: short and infinite. Make short do something trivial like return “Hello World!”. Make infinite run an infinite tight CPU bound loop. Saša Jurić https://elixirforum.com/t/elixir-synthetic-performance-test/3558/6
  13. 13. Vert.x POST /sum 1 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1}' http://vertx/sum real 0m0.016s 2 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1000000000}' http://vertx/sum real 0m35.825s 1 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1}' http://vertx/sum real 0m34.992s
  14. 14. Vert.x POST /blocking_sum 1 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1}' http://vertx/blocking_sum real 0m0.016s 2 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1000000000}' http://vertx/blocking_sum real 0m31.166s 1 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1}' http://vertx/blocking_sum real 0m30.396s
  15. 15. Elixir Elli POST /sum --erl “+S 1” so only one Erlang scheduler is launched 1 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1}' http://elixir-elli/sum real 0m0.013s 2 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1000000000}' http://elixir-elli/sum real 0m32.924s 1 > time curl -X POST -d '{"a": 1, "b": 20, "iterations_count": 1}' http://elixir-elli/sum real 0m0.013s
  16. 16. Scheduling types Preemptive: A preemptive scheduler does context switching among running tasks and has the power to preempt (interrupt) tasks and resume them at a later time without the cooperation of the preempted tasks. This is done based on some factors like their priority, time slice or reductions. Cooperative: A Cooperative scheduler needs tasks’ cooperation for context switching. This way the scheduler simply lets running tasks to voluntarily release control periodically or when idle, then starts a new task and again waits for it to return the control back voluntarily. Now the question is what scheduling mechanism is suitable for real-time systems which must response within a specified time. Cooperative Scheduling system cannot satisfy a real-time system because a running task in such system might never return control back or returns late after a deadline. So real-time systems commonly use Preemptive Scheduling. https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-sche duler-details.html
  17. 17. “Just wrap your existing code into a Vert.x” Real project feedback: ● “console.log” to debug ● Objects are not always JS objects, for example they do not always have “toString” method to print it to the log ● Not all Node.js libs are usable ● It took a week to rewrite JS promises to the Vert.x type of async execution ● Elasticsearch client for Vert.x is a piece of crap (same thing performs faster on Rails and has less error rate)
  18. 18. They do have some good abstractions to reduce callback hell. I played with it for awhile but in the end it was a lonely affair, with very little activity on their Google group. Mandemus https://elixirforum.com/t/elixir-synthetic-performance-test/3558/11?u=mpugach
  19. 19. Questions 1. Vert.x does not utilize the CPU fully in our test, is it a bottleneck? 2. Does Vert.x language translation affects performance? 3. Do they have :observer.start ? 4. Does the community active? 5. What about documentation? 6. How about other ecosystem, can I reuse libraries?
  20. 20. Repo https://github.com/litslink/elixir_vertx_benchmark/tree/round_1
  21. 21. Links https://gist.github.com/Gazler/c539b7ef443a6ea5a182 https://elixirforum.com/t/elixir-synthetic-performance-test/3558 https://elixirforum.com/t/plug-post-performance/473 https://elixirforum.com/t/evaluating-elixir-phoenix-for-a-web-scale-performance-critical-application/832 https://www.theerlangelist.com/article/phoenix_latency https://crypt.codemancers.com/posts/2017-11-22-elixir-remote-debugging/ https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-schedul er-details.html https://en.wikipedia.org/wiki/Vert.x https://stackoverflow.com/questions/40709931/vertx-scaling-the-number-of-instances-per-thread/4071632 1#40716321
  22. 22. Presentation sponsor https://jobs.dou.ua/companies/litslink/vacancies

×