This talk is the story of design and implementation of Marketing Communication Suite at Persado. Marketing Communication Suite is a platform serving tens of customers ranging from telecoms to finance and web properties with persuasion marketing language messaging. Our platform uses a range of technologies with the most important being MongoDB for the online transactional and analytical processing of messages. Topics this talk will be about: MongoDB Aggregation vs. Mapreduce Data Modeling Deployment Architecture Migration Scenarios Hybrid Solutions

1. Breaking the Oracle tie;
High Performance OLTP and analytics using MongoDB
AlexandrosGiamas
Senior Software Engineer
1

3. Can you afford to leave half the opportunity on the table?
You won't believe it You won't believe it You won't believe it
They dial, you answer on VoIP! Pick an Online Number! Pick an Online Number!
Why you'll love your Online Number: Why get an Online Number: Why you'll love your Online Number:
1. Family & friends without VoIP can call 1. Your friends without VoIP can call you 1. Your friends without VoIP can call you
you 2. You answer on VoIP 2. You answer on VoIP
2. You answer on VoIP 3. You also have voicemail included 3. You also have voicemail included
3. And you can use it from anywhere in the
world
I like that! I like that! I like that!
1.11% 1.42% 2.07%
…another 16 Million +
combinations
4

4. The Marketing Communication Suite
We Generate the marketing messages that work best.
For any customer, any product, at any time.

5. Persado History
Oracle shop

6. Persado History

7. Persado History
• Exponentially growing dataset
• Data value/KB?

8. Persado History
Not anymore...

9. Persado History
Transactional Data and Analytics

10. Transaction (Re)-defined
Social, Mobile, Email, Web, Display, Search
Which one stands out?

11. Conversational and Transactional Properties
Web based channels

12. Conversational and Transactional Properties
Mobile Text Messaging

13. Flexi-structured data
One User across campaigns and mediums{ "_id" : ObjectId("511e3cbea9f1fd01

14. Overall Architecture - Data flow

15. Sizing transactional data
☛ User Terminated data
☛ User Originated data
☛ Metadata (state for User per campaign and globally)
☛ Must hold data in memory, or at least indexes

16. ETL for OLAP
Offline / Online processing
•Going online is mostly simpler
•Offline must take into account data irregularities (data validation
policy driven by business needs)

17. ETL for OLAP
☛Custom Data transformation
☛Custom “continueOnError” implementation

18. Analytics
First cut
- Custom js server-side using $where

19. Analytics
GWL
Global Write Lock

20. Analytics In the real world

21. Your own mini transactions
Break down Spring Batch steps in idempotent and non idempotent
ones
•For idempotent steps, just replay them
•For non idempotent, replace current state with last known good
state before latest spring batch step invocation (undo log) and retry
the step

22. Your own mini transactions Issues
•16MB document size limit...
•Slow to replay
•Hard to test using Selenium

23. Analytics In the real world
Map Reduce Implementation

24. Analytics In the real world
Caching layers
✓ Caching in collections

25. Analytics In the real world
Caching layers
✓ Caching in ehcache

26. Analytics using the Aggregation Framework
{$project: { "rdd": {
$isoDate: {
year: {$year:"$_id.receivedDateHour"},
month: {$month:"$_id.receivedDateHour"},
dayOfMonth: {$dayOfMonth:"$_id.receivedDateHour"},
hour: {$hour:"$_id.receivedDateHour"}
}
},
"value.diffDaysSum.0":1,
"value.diffDaysSum.1":1,
"value.diffDaysSum.2":1
}
},
{$project:{rdd:1, diffDaysSum : {$add : ["$value.diffDaysSum.0",
"$value.diffDaysSum.1", "$value.diffDaysSum.2" ] } }
},
{$group: {
_id:"$rdd", totalSumPerDay: { $sum: "$diffDaysSum" }
}
}

27. Analytics using the Aggregation Framework
Double project phase, followed by grouping results

28. Analytics using the Aggregation Framework
Pros:
✓ More flexible than it sounds
✓ Rapid development
✓ Easy debugging
Cons:
✘ No custom js supported
✘ Memory limitation
✘ API still evolving

29. Fine grained write semantics and asynchronous magic
Fine grained write semantics
•WriteConcern.SAFE for most writes
•WriteConcern.REPLICAS_SAFE for writes that are costly
to recompute in case of failure
Reactive Mongo
•Asynchronous and non blocking scala driver for
MongoDB
•Async writes with WriteConcern.SAFE and callback retry
policy in case of error

30. Lessons Learned
Use
replica sets
Journaling
Aggregation Framework
MMS
Don't use
Development versions across the team
Unbound datasets that can't fit in memory
MapReduceif you don't need to

31. MongoDB on EC2
4 nodes with 6 mongod processes

32. MongoDB on EC2 Using LVM's
For high performance, use LVM's with RAID 0 or 10
Have your guerilla team ready:
http://goo.gl/8NbV7

33. MongoDB on EC2 Lesson Learned
Unix level tweaks:
•Raise ulimit
•Raise tcp timeout
•Noatimenodirtime
•Use XFS or ext4
•Use LVM for snapshotting
 Use journaling

34. Questions?
alexandergiamas@yahoo.com
alexandros.giamas@persado.com