How to use the Google Cloud Platform - as used at Streak

1. Data @ Streak
How we use the Google Cloud Platform
Aleem Mawani, aleem@streak.com, @aloo

2. What is Streak?

3. What do we use Google Cloud Platform for?
1. Serve our app and run business logic
AppEngine + Prediction API + Search API
2. Store and Analyze user data
AppEngine Datastore + Memcache + Cloud Storage + BigQuery
3. Record and analyze every UI action
BigQuery (Streaming + Bigquery Extension)
4. Record and analyze all server logs
BigQuery (Streaming + Bigquery Extension)
5. Record and analyze all client logs
BigQuery (Streaming + Bigquery Extension)

6. Streak on Google Cloud
App Engine
...
app
data
Datastore +
Memcache

7. Streak on Google Cloud
Logstore
lo
App Engine
dat
a
g
da
ta
log
BigQuery
...
app
data
Datastore +
Memcache

8. Streak on Google Cloud
Logstore
lo
App Engine
dat
a
g
da
ta
log
BigQuery
...
app
data
Datastore +
Memcache
app
data
Cloud
Storage
Prediction
API

9. Streak on Google Cloud
Logstore
lo
App Engine
dat
a
g
da
ta
log
BigQuery
...
app
data
Datastore +
Memcache
app
data
Cloud
Storage
ata
pd
ap
App Engine
Search API
Prediction
API

10. Streak on Google Cloud
Logstore
lo
App Engine
dat
a
g
da
ta
log
BigQuery
Cloud
Storage
ata
pd
Translation API &
Human Translators
app
data
ap
n
Datastore +
Memcache
i18
...
app
data
App Engine
Search API
Prediction
API

11. Streak on Google Cloud
t
ta
lo
g
ta
da
da
App Engine
dat
a
ata
nt d
en
log
event d
ata
eve
ev
Logstore
BigQuery
Cloud
Storage
ata
pd
Translation API &
Human Translators
app
data
ap
n
Datastore +
Memcache
i18
...
app
data
App Engine
Search API
Prediction
API

12. BigQuery Demo

13. Thanks!
Like big data? Streak.com is hiring!
jobs@streak.com

14. Summary
Benefits
● Move fast and build lean!
● Google quality infrastructure for everyone
● Ask critical business questions
Tradeoffs
● Need to move data incrementally
● Need to make your pipelines fault tolerant at scale
Tips
● Mirror your data in multiple systems, storage is cheap!
● Build systems incrementally