Valeri Karpov MongoDB - NodeJS Engineer Tuesday, Oct 20th 4:20 pm - Databases Find more by Valeri Karpov: http://www.slideshare.net/vkarpov15/

1. Developing and Testing a MongoDB
and Node.js REST API
Valeri Karpov
Node.js Engineer, MongoDB
www.thecodebarbarian.com
github.com/vkarpov15
@code_barbarian

2. *
What is this talk about?
•Briefly introduce MongoDB and Node.js
•MongoDB is great for storing web/mobile app data
•So let’s build a REST API using Node.js!
•+ learn a bit about test-driven dev with Node.js
•+ learn two MongoDB schema design principles
•Server-side only
•Upcoming EdX course

3. *
What is MongoDB?
•Document database - store objects, not columns
•Often better* performance
• Better data locality than RDBMS for 1 query vs 5 JOINs

4. *
What is MongoDB?
•Easier ops
• No need to set up databases, tables, schemas, etc.
• Run one executable, you now have a working db
• MongoDB Cloud Manager and Ops Manager
•Developer sanity
• In my experience, easiest database to get working
• Minimal data transformation between DB and server
• Company has dedicated “Developer Experience” team

5. *
Overview
•Part 1: Shopping Cart Application
• Search for products
• Add them to your cart
• Check out with Stripe
•Part 2: Using Node.js and the Mongoose ODM
•Part 3: Schema Design
•Part 4: Building an API with the Express framework
•Part 5: Testing with Mocha + Superagent

6. *
Part 1: What Does the App Do?

7. *
What Does the App Do?

8. *
What Does the App Do?

9. *
App Structure
•"Bad programmers worry about the code. Good
programmers worry about data structures and their
relationships." - Linus Torvalds
•3 schemas for 3 collections:
•Products
•Categories
•Users

10. *
Schema Relationships
•Product belongs to one or more categories
•Users can have multiple products in their cart
•Representing relationships in MongoDB is tricky
• MongoDB doesn’t have a JOIN equivalent
•But that’s what mongoose is for
• Mongoose provides pseudo-JOINs (separate queries)

11. *
Part 2: Using the Mongoose ODM
•“Object document mapper” (like ORM, but for
MongoDB)
•“MongoDB object modeling designed to work in an
asynchronous environment”
•Written for Node.js
•Provides schema validation, pseudo-JOINs, etc.

12. *
Brief Overview of Node.js
•require()
● Async I/O

13. *
What Does Async Mean?
Register event
handler
In node.js, you don’t execute I/O imperatively.
You register a callback to execute when the I/O is done
Single thread - handler can’t be interrupted
Prints before
“done reading”

14. *
Why Async?
•Most web app/mobile app backends are I/O bound
• Multi-threading doesn’t help if CPU is just waiting for I/O
• Mostly just waiting for database to respond
•Nowadays there’s a REST API for everything
• Analytics and tracking: Segment
• Transactional email: Vero
• Mailing lists: MailChimp
•Threads are cumbersome for I/O bound programs

15. *
Your First Mongoose Schema
matches X@Y.Z

16. *
Using Your Schema
Create User
Save user to MongoDB
Load user from MongoDB
Print user to stdout

17. *
Part 2 Takeaways
•Mongoose provides several neat features
• Model part of MVC
• Default values
• Schema validation and declarative schema design

18. *
Part 3: Schema Design
•3 schemas:
• Product
• Category
• User
•Going to use mongoose to define schemas
•Will use a couple key schema design principles

19. *
Product Schema

20. *
Product Schema in Action

21. *
Category Schema

22. *
Category Schema Queries
•What categories are descendants of “Electronics”?
•
•
•What categories are children of “Non-Fiction”?
•
•What categories are ancestors of “Phones”?

23. *
Product + Category Schemas

24. *
Category Schema Takeaways
•Queries in MongoDB should be simple
•Strive for minimal data transformation by server
•“Store what you query for”
•“If you need [the aggregation framework in a heavily
used API endpoint], you're screwed anyway, and should
fix your program.” - Linus Torvalds
•Good for performance and developer sanity

25. *
User Schema
User’s cart as an array
of ObjectIds...

26. *
Principle of Least Cardinality
•Product and user = many-to-many relationship
•Don’t necessarily need a mapping table
•User won’t have 1000s of products in cart
•Can represent relationship as array in user since one
side is small
•If one side of many-to-many is bounded and/or
small, it is a good candidate for embedding
•Arrays that grow without bound are an antipattern!
• 16mb document size limit
• network overhead

27. *
Part 4: The Express Framework
•Most popular Node.js web framework
•Simple, pluggable, and fast
•Great tool for building REST APIs

28. *
Your First Express App
Route Parameter

29. *
What is REST?
•Representational State Transfer
•HTTP request -> JSON HTTP response
•Business logic on top of MongoDB schemas
• Access control, emails, analytics, etc.

30. *
Structuring Your REST API

31. *
GET /category/id/:id
Find Category
Error handling
Output JSON

32. *
GET /category/parent/:id

33. *
GET /product/category/:id

34. *
Adding Products to User’s Cart
•Recall cart is an array of products

35. *
Adding Products to User’s Cart
Get cart from HTTP request
Overwrite user’s cart
Let mongoose handle
casting and validating data

36. *
PUT /me/cart Takeaways
•Mongoose lets you be lazy
•Access control using subdocs

37. *
Bonus: Stripe Checkout
Create a Stripe charge with the npm ‘stripe’ module

38. *
Bonus: Stripe Checkout
Error handling
Empty user cart
on success

39. *
Part 4 Takeaways
•Express REST API on top of mongoose
• Access control
• Business logic
• Define what operations user can take on database
•Mongoose casting and validation for APIs

40. *
Part 5: Test-Driven Development
•Building an API is tricky
•Lots of different error conditions
•Express has a lot of magic under the hood

41. *
NodeJS Concurrency and Testing
•Node.js runs in an event loop
•Single threaded
•Can run client and server on same thread!
• Client sends HTTP request
• Client registers a callback awaiting the result
• Server’s “on HTTP request” event handler is triggered
• Server sends response, continues waiting for events
• Client’s callback gets fired
•Test server end-to-end

42. *
Superagent
•NodeJS HTTP client
•Isomorphic: runs in both browser and NodeJS
•Same author as Express

43. *
Mocha
•Testing Framework for NodeJS
•Same author as Express
•BDD-style syntax
• describe() -> test suite
• it() -> individual test

44. *
Setting Up Category API Tests

45. *
Testing GET /category/id/:id

46. *
Part 5 Takeaways
•NodeJS concurrency makes testing easy
•Not just unit tests - full E2E for your REST API
•Can manipulate database and make arbitrary HTTP
requests

47. *
•Upcoming EdX Video Course
•Slides on http://www.slideshare.net/vkarpov15
•Looking for beta testers! Sign up for notifications
• http://goo.gl/forms/0ckaJ4YvJN
•Interested in learning about AngularJS?
• Professional AngularJS on Amazon
•More NodeJS+MongoDB content at:
• www.thecodebarbarian.com
• Twitter: @code_barbarian
Thanks for Listening!