- How do we currently think about Data Science? - Why is infrastructure important to our field? - Two tools we've built on Sailthru's Data Science team to deal with these problems are "Stolos" and "Relay.Mesos".

2. Balancing Infrastructure
with Optimization and
Problem Formulation
Sailthru Data Science

3. How do we
think about and practice
Data Science

4. Talk Outline
Part 1:
● What is Data Science
● Where should we spend our time as data
scientists?
Part 2:
● How we balance infrastructure,
optimization and problem formulation at
Sailthru.

5. What is Data Science?!

6. “Data Science is
the extraction of
knowledge from data”
… Wikipedia

7. http://drewconway.com/

8. wikibooks.org

9. Data Scientists are good at …
These Interpretations Suggest:

10. Data Scientists are good at structuring
problems, and solving for and optimizing them.
These Interpretations Suggest:

11. So what’s missing here?
zorger.com

12. ● problem formulation
● optimization
● infrastructure
The title of this talk mentions...

13. ● problem formulation
● optimization
● infrastructure
The title of this talk mentions...

14. Infrastructure
the basic physical and organizational
structures and facilities needed for the
operation of a society or enterprise
“
”
… Wikipedia

15. Infrastructure: Often under-appreciated
or undervalued by Data Scientists

16. A Data Scientist’s infrastructure?

17. Infrastructure
Something we become intimately familiar with

18. Infrastructure
A mission-critical component of our work!

19. Components of a Solid Infrastructure
● Lots of Machinery. VMs, Containers
● Machines require coordination, redundancy
and fault tolerance. CAP Theorem

20. Components of a Solid Infrastructure
● Resource Allocation Fair Scheduling, Bin Packing
● Control strategies Auto Scaling, Feedback, PID
● Communication algorithms Gossip, Paxos, ...
● Configuration Dynamic Persistence, Namespaces
● Monitoring Anomaly Detection, Visualization
● Data Storage Relational, Graph, Key-Value
● SO MANY TOOLS!

21. So What is Data Science?
Problem
Formulation
Infrastructure
Optimization

22. Central Question
As a data scientist,
how do I choose where to
spend my time?

23. As a Data Scientist, ...
...when do I:
○ build infrastructure that supports my ideas
○ optimize my existing models and
problems
○ find new problems to work on

24. Part 2 !

25. Here’s a glimpse of how we tackle these
choices at Sailthru.

27. ● Sailthru is a personalization platform.
● We help our clients communicate with their
customers.
● Our goal is to maximize the lifetime value of these
customers so that our clients do well, customers
are happy, and Sailthru is successful.

28. Sailthru Sightlines: User Predictions

29. Sightlines - Example Use Cases
Incentivize users with low
chance of purchasing
Personalize discounts
above expected order value
Suppress users likely to opt-
out of messages
Engage users unlikely to
open on other channels

30. Sightlines - How it Works

31. Computational Challenges
● Feature Engineering + ML
● Run many dependent jobs at scale
● Resource allocator
● Auto Scaler

32. Computational Challenges
● Feature Engineering + ML → Tidyjson & GBMs
● Run many dependent jobs at scale → Stolos
● Resource allocator → Mesos + AWS Spot Instances
● Auto Scaler → Relay.Mesos

33. github.com/sailthru/stolos
STOLOS

34. What problem does it solve?
A Directed Acyclic Multi-Graph task dependency
scheduler designed to simplify complex, distributed
pipelines.
It creates application queues that can be consumed
from in any order.

35. Sightlines - Stolos Pipeline
450 * 20
Each node is a job

36. Sightlines - Stolos Pipeline
Repeats over time
(currently, 1 day periods)

37. github.com/sailthru/relay
github.com/sailthru/relay.mesos
Relay.Mesos

38. What problem does it solve?
Relay actively minimizes the difference between a
measured signal and a target signal.
Relay.Mesos plugs Relay into a tool called Mesos.
→ Lets us auto-scale consumers of queued Stolos
jobs

39. FFT Visualization

40. Signal
FFT
f1
f2
f3
f4
FFT Visualization
k=0
k=1
k=2
ai-k
=1

41. Signal
FFT
f1
f2
f3
f4
FFT Visualization
k=0
k=1
k=2
ai-k
=1

42. The PID Algorithm
PV = Process Variable (Signal)
SP = Set Point (Target)
MV = Manipulated Variable (Output)
t = index on timesteps
**The “D” in PID is excluded here

43. The PID Algorithm
PV = Process Variable (Signal)
SP = Set Point (Target)
MV = Manipulated Variable (Output)
t = index on timesteps
**The “D” in PID is excluded here
+ Kd
Δ dt

44. Sightlines - Relay

45. Thank You! Our team:

46. Tidyjson github.com/sailthru/tidyjson
Stolos github.com/sailthru/stolos
Relay github.com/sailthru/relay
Relay.Mesos github.com/sailthru/relay.
mesos
Consulconf github.com/sailthru/consulconf
With more in progress!
Check out our open sourced tools!

48. Sightlines - On Mesos
←----------------> CPU Units <------------------>
←--------------------->RAM←--------------------->
←----------------> CPU Units <------------------>
←--------------------->RAM←--------------------->

49. Sightlines - Stages
Predict
API
Push
Sample &
Assemble
Grid Build

50. Sightlines - Stages
Predict
API
Push
Sample &
Assemble
Grid Build
Build train and
test sets from a
sample of data

51. Sightlines - Stages
Predict
API
Push
Sample &
Assemble
Grid Build
Run Grid Search
to identify
Hyperparameters
for the model

52. Sightlines - Stages
Predict
API
Push
Sample &
Assemble
Grid Build
Build the
model

53. Sightlines - Stages
Predict
API
Push
Sample &
Assemble
Grid Build
Generate
predictions for
all relevant
models

54. Sightlines - Pipeline
Sample
Database
Database
SampleSample &
Assemble
AssembleSampleSampleGrid
AssembleSampleSampleBuild
AssembleSampleSamplePredict SampleSampleAPI
Push

55. Sightlines - Pipeline
Sample
Database
Database
SampleSample &
Assemble
AssembleSampleSampleGrid
AssembleSampleSampleBuild
AssembleSampleSamplePredict SampleSampleAPI
Push
○ Upper branch: once per (client, day, model)
○ Lower branch: once per (client, day)