What does it take to build a web application that can serve millions of concurrent users? This talk will dissect Open edX, the open source online learning platform that powers edX.org and hundreds of other sites around the world. Open edX is written in Django but relies on a slew of other software to work at scale. We’ll break down the components of the entire software architecture, and explain how each component is used. This will give attendees some insights into how they might architect their web application if they’re building software to serve a large audience. We’ll discuss how Open edX utilizes both MySQL and MongoDB as data stores and why it uses MySQL for student data and MongoDB for course data. We’ll describe the queing architecture of Celery and RabbitMQ to process background jobs, and Hadoop for processing terabytes of learner analytics data. DevOps enthusiasts will not be disappointed as we’ll go into the use of Ansible and Terraform for deploying the entire suite of software in a consistent and repeatable way up to popular cloud providers such as AWS, Google Cloud and Azure. What will attendees learn? The attendees will get a close look at a modern Django-based web application to see how it’s architected, and come away with a better understanding of design decisions when building a web application that needs to serve 35M users from every country in the world. Attendees will also be equipped with the knowledge to run Open edX on their laptop or in the cloud, if they want to explore the software further.

1. The Anatomy of
Open edX
A modern open source online learning
platform for delivering education at scale
Nate Aune (@natea)
DjangoCon - San Diego
October 17, 2018
www.appsembler.com

2. What we’ll
cover
What is edX?
What is Open edX?
How is Open edX architected?
How is Open edX deployed and scaled?
How can you get started with Open edX?
Where can you get more info about Open edX?
Q&A
www.appsembler.com

10. 1 7 M + S T U D E N T S F R O M E V E RY C O U N T RY

11. What is Open edX?
• Course Authoring and Delivery platform
• Data analytics, Credentials, Enterprise
• Open Source (AGPL licensed)
• Django/Python application

17. Learner-
facing
assessment

18. Studio:
Course
authoring
environment

20. Growth of
Open edX
community

21. Healthy community of participants
Source: https://groups.google.com/forum/#!aboutgroup/edx-code
# of posts on edx-code mailing list in 2018
~3,800 members of mailing list ~2,800 members on Slack
Source: https://openedx.slack.com

23. Healthy community of contributors
~400 individual contributors
Source: https://www.openhub.net/p/open-edx/contributors/summary
Source: https://www.openhub.net/p/open-edx/contributors/summary

24. Sites
1,500-
1300 -
1100 -
900 -
700 -
500 -
300 -
100 -
Courses
- 18000
- 15000
- 13000
- 11000
- 9000
- 7000
- 5000
- 3000
- 1000
24

25. 1500+ 18,000+Sites Courses
We learned about over 700 more sites and over 13,000
courses in the past year!

26. More than 35 million learners
on the Open edX platform
>17m edx.org learners
>18m Open edX learners

28. Top MOOC sites
9 of 39 on Class Central are built on Open edX!
Source: https://www.class-central.com/providers

29. National platforms
Israel (campus.gov.il)
Russia (openedu.ru)
Taiwan (www.openedu.tw)
Saudi Arabia (doroob.sa)
South Korea (kmooc.kr)
France (fun-mooc.fr)
Jordan (edraak.org)
China (xuetangx.com)
Mexico (mexicox.gob.mx)
Kazakhstan
(openu.kz)

30. Source: https://www.transifex.com/open-edx/edx-platform/
Translated into dozens of languages

31. Top contributions
XBlock Asides
Enhancements
Peer Instruction
v2
Grades API
Mobile apps
Figures reporting
i18n / RTL fixes
and enhancements
Office 365 / OneDrive
Azure Media
Bulk Grades API
Conditional
Authoring UI
Mobile app
Enhancements
Lots of
contributions!
Google OAuth2,
Google Drive,
Google Calendar

33. Source:
oep.readthedocs.io
github.com/edx/open-edx-proposals

34. Come for the code. Stay for the community.

35. What is Open edX made of?

38. Source: https://www.openhub.net/p/open-edx/analyses/latest/languages_summary

39. How is Open edX architected?

42. Learner analytics from edX Insights

47. So how the heck do you deploy this thing?

49. Two phases for deploying the Open edX platform
Infrastructure Provisioning Service Configuration

50. Ax finishes the job
Infrastructure Provisioning Service Configuration
Ax

51. ax.yml

52. We have three deployment tiers: Basic, Pro, and Enterprise. Each tier
comes with a default configuration, but most settings can be overridden.
• Basic (Single server) -- A single VM is created which will host the entire Open edX stack along with MongoDB and
MySQL.
• Pro (Multiple servers) -- Separate VMs are created to host the Open edX core stack, MongoDB, and MySQL. A
cloud-provided MySQL instance is used.
• Enterprise (Multiple servers with redundancy) -- Multiple VMs are created to host the Open
edX core stack, and are positioned behind a load balancer. Multiple VMs are also created to host a Mongo replica set. A cloud-provided
MySQL instance is created, optionally with master-slave replication.
Terraform: decide on deployment tier

53. ficus-pro.yml

54. server-vars.yml

55. inventory

56. $ ax terraform
--customer jfrog
--environment prod
--tier pro
--plan gcp
init --terraform-plugins-dir ~/terraform-plugins
How to init Terraform

57. $ ax terraform
--customer jfrog
--environment prod
--tier pro
--plan gcp
plan
How to Terraform a cluster for a customer

58. $ ax terraform
--customer jfrog
--environment prod
--tier pro
--plan gcp
apply
How to Terraform a cluster for a customer

59. $ ax ansible
--customer jfrog
--environment prod
--ansible-playbook server-vars.yml
—-ansible-vault-password-file secrets.yml
provision
How to provision a cluster

60. Open edX Devstack
powered by

61. Local dev environment with Docker Compose
github.com/edx/devstack
$ cd ~/workspace
$ mkvirtualenv devstack
$ git clone https://github.com/edx/devstack.git
$ cd devstack
$ make requirements
$ make dev.clone
$ make dev.provision
$ make dev.up

63. Open edX
Core
Platform
Open edX core platform
governance owned and managed by edX
Ecosystem projects community
governed and managed
Plugins &
Extensions
Independent
Commercial
Products & Services
Open edX Platform & Ecosystem

64. Where can I get more info?
Slack community: http://bit.ly/openedx-slack
Mailing list: groups.google.com/forum/edx-code
Main codebase: github.com/edx/edx-platform
Ansible playbooks: github.com/edx/configuration
Local development: github.com/edx/devstack
Documentation: docs.edx.org

65. • Feanil Patel for his talk “Hosting Architecture at edX”
http://goo.gl/5nlNqy
• Regis Behmo for his talk “Open edX 101: a source code review”
https://regisb.github.io/openedx-conference-2016/
• Nimisha Asthagiri for her talk “Evolution of LMS and Studio
Architecture” and “edX Architecture Onboarding”
https://speakerdeck.com/nasthagiri/evolution-of-edx-lms-and-studio-architecture
• John Mark for his talk “State of Open edX”
Credits and Special thanks to…

66. www.appsembler.com/tahoe/trial
Email me: nate@appsembler.com
Tweet at me: @natea