http://www.hakkalabs.co/articles/performant-django-best-practices

1. Performant Django
Ara Anjargolian
Co-Founder & CTO, YCharts

2. There are two distinct kinds of
performance issues
Predictably, they are: front-end and back-
end.
Handling them effectively, requires very
different approaches.

3. First, a quick note about frontend
performance
80-90% of the end-user response time is
spent on the frontend.
Start there.
-Steve Souders

4. Front-End Performance Work
• Can be universally applied
• Requires systems/tooling changes
• Often has clear, system-independent best
practices

5. Best Practice: Cache static assets
forever (as long as they don’t change)
Why: Download assets as infrequently as
possible
Solution: Already done! (As long as you use
CachedStaticFilesStorage or
CachedFilesMixin with your own storage)

6. Best practice: Bundle/minify/
compress static assets
Why: Reduce # of requests, download time
Solution: Use a static-asset-manager. 2 good
ones: django-pipeline, webassets.
Bonus points: Lower number of requests by
using data URIs for images (which pipeline
supports)

7. Best Practice: Serve static
files via a CDN.
Why: Less latency
Solution:
Good: Store in filesystem, point
STATIC_URL to CDN with an origin to your
URL.
Better: Use django-storages/
STATICFILES_STORAGE storage setting to
store in cloud file storage (i.e. S3) and point
CDN to it.

8. Best Practice: Serve more
stuff as static assets.
Why: Static assets can be served faster,
more efficiently than dynamic assets.
Solution: Front-end templates, static-y data
structures that can be served as JSON.
All that’s required are some custom
management commands.

9. Back-End Performance Work
• Can really only be done on a case by case
basis.
• Often only requires code changes.
• Is very site-specific.

10. OK, I lied, there are some global back-
end performance to-dos.
• Use cached sessions
(contrib.sessions.backends.cache or
contrib.sessions.backends.cached_db)
• Use cached template loader
• If you’re starting a new project, or do a ton
heavy weight templates, consider using
jinja2 as your template engine.
But on to the real stuff!

11. OK, I lied, first a disclaimer
DO NOT try to “optimize” every view.
• This is an utter waste of time, as there will
be diminishing returns.
• Optimizing on the backend often means
adding complexity. And in a multi-
programmer environment, complexity is
expensive!

12. Backend performance work starts with
a profile of the “problem” view
Use a profiler middleware!
(A good one:
https://gist.github.com/Miserlou/3649773)

13. What does a profile look like?

14. Understanding a profile
Things to look for:
• Tons of time spent in SQL?
• Particular functions being called to where
the function call is taking longer than you
would expect, or, the function is being
called way too much?

15. What if the problem is SQL?
First use django-debug-toolbar, or, django-
devserver to identify the problem queries.
Is the issue one slow query? Too many
queries?

16. SQL Tricks, Part 1
• select_related(): Helps avoid extra queries
to grab objects referenced by foreign
keys/one to one relationships
• values/values_list(): Avoid Python object
creation overhead when dicts/lists are
good enough
• db_index=True: if you are referencing
objects by field that’s not it’s primary/
foreign key and does not have a
uniqueness constraint on it, you might
need this

17. SQL Tricks, Part 2
• prefetch_related(): Like select related
except the “join” is done in Python and
thus works for M2M
• only(): Only grab fields in the model you
need (USE WITH CAUTION!)
• defer(): Get all fields except those stated
in defer()
• bulk_create(): When writing lots of rows
to same table

18. What if the problem is SQL and none
of the above helps?
• raw(): -Roll your own SQL that can
perhaps use stuff specific to the DB, or
fancier queries.
• Denormalization: Less joins, precomputed
data
• No SQL: Maybe the data you are storing in
a relational database doesn’t map well to a
relational database.

19. What if the problem is in the Python?
Common issues:
• Algorithmic issues like n^2 paths that don’t
need to be n^2
• Doing extra work like constantly re-evaluating
a loop invariant inside a loop
• Using if/else for error controls where
exceptions will do (again, most problematic
inside a loop)
In general: People doing bad stuff inside loops.

20. What if you optimized your Python/
SQL and you’re still slow?
Cache.
Then cache some more.

21. • View cache
• Template fragment cache
• Function level cache (via package like
django-cache-utils, django-cache-helper)
• Query cache (django-cache-machine,
django-cacheops)
Many types of caching

22. The End
Questions?
ara@ycharts.com
http://github.com/ara818
Like solving complex performance
problems?
YCharts is hiring!