About code quality & production monitoring

1. Code Quality to Monitoring production:
use cases and best practices
David Melamed
Pyweb meetup - September 21, 2014

2. Agenda
What is good code?
How to measure quality code?
Continuous Integration
How to efficiently monitor production?

3. About me
Full-stack developer at Cloudlock
Fond of technology & automation
Code Quality Evangelist
Geek on my free time

4. About Cloudlock
Founded: 2011
Headquarters: Waltham, Mass. (U.S.)
R&D Headquarters: Tel Aviv
Employees: 80+
Funding: $28M+
Investors Selected customers
700+ Customers 9,600 Unique
Apps Discovered
Native Cloud Security Solutions
for SaaS Applications
5 million Users
under Management
13 billion objects
processed daily

5. What is good code?

6. Good code is…
Easy to read & understand
Easy to extend
Easy to maintain
Testable

7. How to write good code?

8. from xkcd

9. Good code requires constant
REFACTORING

10. The Code Quality Star
Coding
Standards
Code
Review
Automated
Tests
Code
Complexity
Architecture
Design
Source

11. Code should be easy to read

13. Code should be readable
Avoid comments but avoid obvious ones (the code
should be self-explanatory)
Avoid comments too verbose
Comments need to be maintained

14. Coding Standards

15. The Zen of Python (PEP20)
Beautiful is better than ugly
Explicit is better than implicit
Simple is better than complex
Complex is better than complicated
Readability counts

16. Coding Style: Readability Counts
“Programs must be written for people to read, and
only incidentally for machines to execute.”
Abelson & Sussman, Structure and Interpretation of Computer Programs

17. PEP8: Style guide for Python
Indentation
Maximum line length
Blank lines
Redundant white spaces
Imports
Comments
Naming conventions
…

18. How to check /enforce PEP8
Pycharm (Tools > Flake8)
Git pre-commit hook (http://goo.gl/bteYkq)
Jenkins job for continuous integration

19. Naming is key

22. Naming best practises
Adopt a convention and stick with it (fetch vs. retrieve vs. get)
Use intention-revealing names
Avoid misleading readers
Make meaningful distinctions
Use searchable names
Avoid mental mapping
Use domain names
From Clean code, R. Martin

23. Code Architecture

24. Best Practices in Code Architecture
OOP
DRY
SOLID
Loose Coupling

25. Don’t Repeat Yourself (aka DRY)
http://www.sonarqube.org/manage-duplicated-code-with-sonar/

26. SOLID principles
• SRP: Single Responsibility Principle
• OCP: Open/closed principle
• LSP: Liskov substitution principle
• ISP: Interface segregation principle
• DIP: dependency inversion
principle

27. http://www.codeproject.com/Articles/93369/How-I-explained-OOD-to-my-wife

28. http://www.codeproject.com/Articles/93369/How-I-explained-OOD-to-my-wife

29. http://www.codeproject.com/Articles/93369/How-I-explained-OOD-to-my-wife

30. http://www.codeproject.com/Articles/93369/How-I-explained-OOD-to-my-wife

31. http://www.codeproject.com/Articles/93369/How-I-explained-OOD-to-my-wife

32. Loosely Coupled Components

33. Main principles
Split components into repositories
Web app: use clean layers
view / endpoint (presentation)
service (business logic)
DAO (model)
Use message bus for communication inter-components (i.e. RabbitMQ)
Use daemons / workers for long-running tasks
Use events / signals to decouple unrelated actions
Microservices

35. Architecture at Cloudlock
Nginx
AWS VPC
Web
Server
API
Server
Message Bus (RabbitMQ)
RDS
PostgreSQL
ElastiCache
Redis
Authentication Gateway
Worker Worker Worker
Logs

36. Code Reviews

38. Tools for code review
Github Pull Requests
Review Board
Gerrit
Crucible

39. Code documentation

41. Code Testing

42. Different types of tests
Manual Test
Unit Test
Integration Test
System Test
Acceptance/Regression Test (end-to-end)
Smoke Test

43. Unit Test
Test if small, distinct and isolated portions of code
are working as expected, provided some input
Mock your dependencies (database, logger, service)
Should be very fast
No need to test EVERYTHING (i.e. private methods)
Frameworks: unittest, pytest and nose

45. Pytest
No need for base class (works both for classes and methods)
More pythonic than unittest
Fixtures (conftest.py)
Parametrized tests
Markers
Various plugins for coverage, bdd, django, twisted…
Used for unit tests, integration tests and end to end tests

46. Integration Tests
Test communication between services
You may mock some of the dependencies depending on
what you test

47. System Tests
Global test, more functional of the whole system
TestApp (Pyramid app) using a test database
Call the API and test the expected response code

48. End-to-end tests
Splinter: web driver abstraction (firefox, selenium)
splinter-pytest: fixtures for browser
Plugin for rerun flaky tests, tests with steps
Use Selenium Grid to test several browsers

49. Smoke Tests
Make sure the system is alive
Tests that the components are working
Tests that the components can communicate
Health check / status page

50. Measuring code quality

52. Code Quality Metrics
Code Standards Violations (pylint /flake8/pyflakes)
LOC (Lines Of Code)
Cyclomatic Complexity
Unit Tests Coverage

53. Code Coverage
Makes sense only for unit tests
Calculates the coverage based on the paths run by
the tests
100% coverage is not a goal on its own
May be misleading regarding the code quality

54. Unit Test Diff Coverage

55. Mutation Analysis
From Mutation analysis vs. code coverage in automated assessment
of students’ Testing Skills (P. Ihantola)

56. Pylint
Report
======
85 statements analysed. !
Duplication
----------- !
+-------------------------+------+---------+-----------+
| |now |previous |difference |
+=========================+======+=========+===========+
|nb duplicated lines |0 |0 |= |
+-------------------------+------+---------+-----------+
|percent duplicated lines |0.000 |0.000 |= |
+-------------------------+------+---------+-----------+ !!!
Messages by category
-------------------- !
+-----------+-------+---------+-----------+
|type |number |previous |difference |
+===========+=======+=========+===========+
|convention |18 |18 |= |
+-----------+-------+---------+-----------+
|refactor |2 |2 |= |
+-----------+-------+---------+-----------+
|warning |1 |1 |= |
+-----------+-------+---------+-----------+
|error |4 |4 |= |
+-----------+-------+---------+-----------+ !!!
Messages
-------- !
+-----------------------------+------------+
|message id |occurrences |
+=============================+============+
|line-too-long |10 |
+-----------------------------+------------+
|no-name-in-module |4 |
+-----------------------------+------------+
|missing-docstring |4 |
+-----------------------------+------------+
Global evaluation
-----------------
Your code has been rated at 5.18/10 (previous run: 5.18/10, +0.00) !
Raw metrics
----------- !
+----------+-------+------+---------+-----------+
|type |number |% |previous |difference |
+==========+=======+======+=========+===========+
|code |90 |63.83 |90 |= |
+----------+-------+------+---------+-----------+
|docstring |30 |21.28 |30 |= |
+----------+-------+------+---------+-----------+
|comment |5 |3.55 |5 |= |
+----------+-------+------+---------+-----------+
|empty |16 |11.35 |16 |= |
+----------+-------+------+---------+-----------+ !!!
Statistics by type
------------------ !
+---------+-------+-----------+-----------+------------+---------+
|type |number |old number |difference |%documented |%badname |
+=========+=======+===========+===========+============+=========+
|module |1 |1 |= |0.00 |0.00 |
+---------+-------+-----------+-----------+------------+---------+
|class |2 |2 |= |100.00 |0.00 |
+---------+-------+-----------+-----------+------------+---------+
|method |1 |1 |= |100.00 |0.00 |
+---------+-------+-----------+-----------+------------+---------+
|function |6 |6 |= |50.00 |33.33 |
+————+-------+-----------+-----------+------------+---------+ !!!!!!!!!!

57. Test Automation

58. Why do you need automated tests?
!
Optimise speed, efficiency and quality
Catch the bugs upfront
Safety net for refactoring

60. On the Road to
Continuous Integration

61. Several “Continuous” processes
Continuous Integration
Continuous Deployment
Continuous Delivery

62. Several platforms for CI
Jenkins
Travis CI
Circle CI
Shippable
BuildBot

63. Continuous Integration Flow
Feature Branch
Develop Branch
Git Push
Updates
Github PR
Several jobs:
- pep8 compliance
- unit test
- integration test
- end to end test
Merge PR
Pull Request
Tag
Deployment
Code Review

64. Jenkins dashboard

67. Automated Deployment

68. Pick the right tool to deploy
AWS ElasticBeanstalk
AWS OpsWorks
Chef
Puppet
SaltStack
Ansible

69. Ansible
http://terry.im/wiki/terry/attachments/29786135/37552129.png

70. It works only on my machine

71. The Docker Project
Run everywhere
Run everything
Advantages:
• Pricing
• Speed
• Unifying environments
• Testability / reproducibility
• Easily add/replace components

72. Docker on your Mac
http://www.warski.org/blog/2014/05/spray-server-docker-container/docker-stack-5/

73. Production Monitoring

75. Production Monitoring
AWS CloudWatch
End-to-end tests running in Jenkins
StackDriver
Log analysis (streams)

76. Questions?

77. Python developers WANTED
Contact: dmelamed@cloudlock.com