SlideShare una empresa de Scribd logo
1 de 37
BERLIN • NEW YORK • SAN FRANCISCO • SÃO PAULO • PARIS • LONDON • MOSCOW • ISTANBUL
SEOUL • SHANGHAI • BEIJING • TOKYO • MUMBAI • SINGAPORE
Scaling a platform for
real-time fraud detection
2
Why is no one prepared for
success?
3
‣ Focus on idea
‣ Focus on execution is limited in time to market
‣ Underestimating importance of scalability and profitability
Startup Zeitgeist
4
‣ Director of Engineering
‣ Consulted with early design decisions
‣ Joined Adjust 2012 as Head of IT Operations
Who am I?
Who is Adjust?
1B+
Daily active users tracked
400+ Billion
Data points tracked monthly
22K+
Apps tracked
6
‣ Automatically reject fraudulent data before it gets paid
‣ Sends rejection reason callbacks to all parties
‣ Customers and their networks have full transparency
‣ Real-time statistical analysis of all ad engagements and app activity
Fraud Prevention Suite
7
How did we do that?
8
Bootstrapping a product
Current approach
‣ Using cloud services
‣ Base product on 3rd party IP
‣ Faster development
9
Bootstrapping a product
Current approach Our approach
‣ Using cloud services
‣ Base product on 3rd party IP
‣ Faster development
‣ Own infrastructure
‣ Develop own IP
‣ Slower ramp up
10
Bootstrapping a product
Current approach Our approach
‣ Using cloud services
‣ Base product on 3rd party IP
‣ Faster development
‣ Potential lock in
‣ More cost efficient in the beginning
‣ Shared environment
‣ Own infrastructure
‣ Develop own IP
‣ Slower ramp up
11
Bootstrapping a product
Current approach Our approach
‣ Using cloud services
‣ Base product on 3rd party IP
‣ Faster development
‣ Potential lock in
‣ More cost efficient in the beginning
‣ Shared environment
‣ Own infrastructure
‣ Develop own IP
‣ Slower ramp up
‣ Keep flexibility
‣ More cost efficient in the long run
‣ Dedicated environment
12
The machine room
13
General Overview
Realtime Callbacks
Raw Data Upload
Realtime Fraud
Prevention
Realtime
Aggregation
SDK Traffic Tracking Server Device Data
Store
14
So it begins…
15
Redis Small
haproxy
Redis Leader
Redis Replica
16
A little more please?
17
Redis Big
haproxy
Redis Leader
Redis Replica
18
We need to go further
19
It’s all connected :)
Leader 1 Replica 1 Leader n Replica n
Sentinel 1 Sentinel n
twemproxy twemproxy
20
And we felt like…
21
22
‣ Automation is tricky
Smoke in the machine room
23
‣ Automation is tricky
‣ Weird latency spikes
Smoke in the machine room
24
‣ Automation is tricky
‣ Weird latency spikes
‣ Increased average response time
Smoke in the machine room
25
‣ Automation is tricky
‣ Weird latency spikes
‣ Increased average response time
‣ Long failover times
‣ Disruptive failovers
Smoke in the machine room
26
And we felt like…
27
28
Sometimes you have to let go…
29
‣ Only index is in RAM
‣ Data is on SSD
‣ Cost reduced by 85%
‣ Server count reduced from 40 to 6
Migrating to Aerospike
30
‣ 1,1 million reads per second
‣ 500.000 writes per second
‣ 180TB of data
‣ 3 locations
Using Aerospike
31
What if?
32
‣ Easy to scale
‣ Redis interface
‣ Online resizing
‣ All dirty work is done by Amazon
Elasticache
33
Q: Can I use Amazon ElastiCache for use cases other
than caching?
A: Yes. ElastiCache for Redis can be used as a primary
in-memory key-value data store, providing fast, sub
millisecond data performance, high availability and
scalability up to 15 nodes plus up to 5 read replicas,
each of up to 9.5 TiB of in-memory data.
34
“Once we are successful, we will
take care of scalability and
profitability”
35
What will happen to my
infrastructure when I will be
successful?
36
Questions and answers
New York
Paris
São Paulo
San Francisco
London Berlin
Istanbul
Moscow
Mumbai
Beijing
Seoul
Tokyo
Shanghai
Singapore
Robert Abraham
DIRECTOR OF ENGINEERING

robert@adjust.com
ADJUST HQ

Saarbrücker Str. 37a

10405 Berlin

Germany

Más contenido relacionado

La actualidad más candente

La actualidad más candente (13)

Production Operations An Architect And Developers Perspective (Without Notes)
Production Operations   An Architect And Developers Perspective (Without Notes)Production Operations   An Architect And Developers Perspective (Without Notes)
Production Operations An Architect And Developers Perspective (Without Notes)
 
Cloud-Native Workshop New York- Dynatrace
Cloud-Native Workshop New York- DynatraceCloud-Native Workshop New York- Dynatrace
Cloud-Native Workshop New York- Dynatrace
 
Accelerate to Cloud
Accelerate to CloudAccelerate to Cloud
Accelerate to Cloud
 
SplunkLive! Utrecht 2017 - ASML Customer Presentation
SplunkLive! Utrecht 2017 - ASML Customer PresentationSplunkLive! Utrecht 2017 - ASML Customer Presentation
SplunkLive! Utrecht 2017 - ASML Customer Presentation
 
Turning Cloud Metrics into Results
Turning Cloud Metrics into ResultsTurning Cloud Metrics into Results
Turning Cloud Metrics into Results
 
SplunkLive! Utrecht 2016 - Exact
SplunkLive! Utrecht 2016 - ExactSplunkLive! Utrecht 2016 - Exact
SplunkLive! Utrecht 2016 - Exact
 
Provide Company Overview
Provide Company OverviewProvide Company Overview
Provide Company Overview
 
Microservices: A foundational approach for fully managed cloud data analytics
Microservices: A foundational approach for fully managed cloud data analyticsMicroservices: A foundational approach for fully managed cloud data analytics
Microservices: A foundational approach for fully managed cloud data analytics
 
ScanNet Linux
ScanNet LinuxScanNet Linux
ScanNet Linux
 
Monitoring with Elastic Machine Learning at Sky
Monitoring with Elastic Machine Learning at SkyMonitoring with Elastic Machine Learning at Sky
Monitoring with Elastic Machine Learning at Sky
 
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix WebinarFuture-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
 
LogicMonitor: An Overview
LogicMonitor: An Overview LogicMonitor: An Overview
LogicMonitor: An Overview
 
Implementing Powerful IT Search on the Cloud
Implementing Powerful IT Search on the CloudImplementing Powerful IT Search on the Cloud
Implementing Powerful IT Search on the Cloud
 

Similar a Big Data LDN 2018: SCALING A PLATFORM FOR REAL-TIME FRAUD DETECTION WITHOUT BREAKING THE BANK

Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...
Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...
Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...
Dynatrace
 

Similar a Big Data LDN 2018: SCALING A PLATFORM FOR REAL-TIME FRAUD DETECTION WITHOUT BREAKING THE BANK (20)

Lambda Architectures in Practice
Lambda Architectures in PracticeLambda Architectures in Practice
Lambda Architectures in Practice
 
SACON - Incident Response Automation & Orchestration (Amit Modi)
SACON - Incident Response Automation & Orchestration (Amit Modi)SACON - Incident Response Automation & Orchestration (Amit Modi)
SACON - Incident Response Automation & Orchestration (Amit Modi)
 
Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...
Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...
Neiman Marcus: Neiman Marcus's journey into the cloud with dynatrace with an ...
 
State of Streams | Gwen Shapira, Fall 2018
State of Streams | Gwen Shapira, Fall 2018State of Streams | Gwen Shapira, Fall 2018
State of Streams | Gwen Shapira, Fall 2018
 
Fifteen Years of DevOps -- LISA 2012 keynote
Fifteen Years of DevOps -- LISA 2012 keynoteFifteen Years of DevOps -- LISA 2012 keynote
Fifteen Years of DevOps -- LISA 2012 keynote
 
AppSphere 15 - Deep Dive into AppDynamics Application Analytics
AppSphere 15 - Deep Dive into AppDynamics Application AnalyticsAppSphere 15 - Deep Dive into AppDynamics Application Analytics
AppSphere 15 - Deep Dive into AppDynamics Application Analytics
 
Acting on Real-time Behavior: How Peak Games Won Transactions
Acting on Real-time Behavior: How Peak Games Won TransactionsActing on Real-time Behavior: How Peak Games Won Transactions
Acting on Real-time Behavior: How Peak Games Won Transactions
 
Pivoting to Cloud: How an MSP Brokers Cloud Services
Pivoting to Cloud: How an MSP Brokers Cloud Services Pivoting to Cloud: How an MSP Brokers Cloud Services
Pivoting to Cloud: How an MSP Brokers Cloud Services
 
Office 365 Monitoring Best Practices
Office 365 Monitoring Best PracticesOffice 365 Monitoring Best Practices
Office 365 Monitoring Best Practices
 
(R)evolutionize APM
(R)evolutionize APM(R)evolutionize APM
(R)evolutionize APM
 
VMworld 2013: How to make most out of your Hybrid Cloud
VMworld 2013: How to make most out of your Hybrid Cloud VMworld 2013: How to make most out of your Hybrid Cloud
VMworld 2013: How to make most out of your Hybrid Cloud
 
Taming Big Data With Modern Software Architecture
Taming Big Data  With Modern Software ArchitectureTaming Big Data  With Modern Software Architecture
Taming Big Data With Modern Software Architecture
 
Buisness drivers for real-time streaming analytics integrated to action frame...
Buisness drivers for real-time streaming analytics integrated to action frame...Buisness drivers for real-time streaming analytics integrated to action frame...
Buisness drivers for real-time streaming analytics integrated to action frame...
 
Disaster Recovery: Don't risk it--automate it
Disaster Recovery: Don't risk it--automate itDisaster Recovery: Don't risk it--automate it
Disaster Recovery: Don't risk it--automate it
 
Selecting SaaS providers
Selecting SaaS providersSelecting SaaS providers
Selecting SaaS providers
 
How to Better Manage Technical Debt While Innovating on DevOps
How to Better Manage Technical Debt While Innovating on DevOpsHow to Better Manage Technical Debt While Innovating on DevOps
How to Better Manage Technical Debt While Innovating on DevOps
 
RightScale Roadtrip - Accelerate To Cloud
RightScale Roadtrip - Accelerate To CloudRightScale Roadtrip - Accelerate To Cloud
RightScale Roadtrip - Accelerate To Cloud
 
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and DruidA Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
 
The Business Justification for APM
The Business Justification for APMThe Business Justification for APM
The Business Justification for APM
 
The new dominant companies are running on data
The new dominant companies are running on data The new dominant companies are running on data
The new dominant companies are running on data
 

Más de Matt Stubbs

Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...
Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...
Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...
Matt Stubbs
 
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...
Matt Stubbs
 

Más de Matt Stubbs (20)

Blueprint Series: Banking In The Cloud – Ultra-high Reliability Architectures
Blueprint Series: Banking In The Cloud – Ultra-high Reliability ArchitecturesBlueprint Series: Banking In The Cloud – Ultra-high Reliability Architectures
Blueprint Series: Banking In The Cloud – Ultra-high Reliability Architectures
 
Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...
Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...
Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...
 
Blueprint Series: Expedia Partner Solutions, Data Platform
Blueprint Series: Expedia Partner Solutions, Data PlatformBlueprint Series: Expedia Partner Solutions, Data Platform
Blueprint Series: Expedia Partner Solutions, Data Platform
 
Blueprint Series: Architecture Patterns for Implementing Serverless Microserv...
Blueprint Series: Architecture Patterns for Implementing Serverless Microserv...Blueprint Series: Architecture Patterns for Implementing Serverless Microserv...
Blueprint Series: Architecture Patterns for Implementing Serverless Microserv...
 
Big Data LDN 2018: DATA, WHAT PEOPLE THINK AND WHAT YOU CAN DO TO BUILD TRUST.
Big Data LDN 2018: DATA, WHAT PEOPLE THINK AND WHAT YOU CAN DO TO BUILD TRUST.Big Data LDN 2018: DATA, WHAT PEOPLE THINK AND WHAT YOU CAN DO TO BUILD TRUST.
Big Data LDN 2018: DATA, WHAT PEOPLE THINK AND WHAT YOU CAN DO TO BUILD TRUST.
 
Big Data LDN 2018: DATABASE FOR THE INSTANT EXPERIENCE
Big Data LDN 2018: DATABASE FOR THE INSTANT EXPERIENCEBig Data LDN 2018: DATABASE FOR THE INSTANT EXPERIENCE
Big Data LDN 2018: DATABASE FOR THE INSTANT EXPERIENCE
 
Big Data LDN 2018: BIG DATA TOO SLOW? SPRINKLE IN SOME NOSQL
Big Data LDN 2018: BIG DATA TOO SLOW? SPRINKLE IN SOME NOSQLBig Data LDN 2018: BIG DATA TOO SLOW? SPRINKLE IN SOME NOSQL
Big Data LDN 2018: BIG DATA TOO SLOW? SPRINKLE IN SOME NOSQL
 
Big Data LDN 2018: ENABLING DATA-DRIVEN DECISIONS WITH AUTOMATED INSIGHTS
Big Data LDN 2018: ENABLING DATA-DRIVEN DECISIONS WITH AUTOMATED INSIGHTSBig Data LDN 2018: ENABLING DATA-DRIVEN DECISIONS WITH AUTOMATED INSIGHTS
Big Data LDN 2018: ENABLING DATA-DRIVEN DECISIONS WITH AUTOMATED INSIGHTS
 
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...
 
Big Data LDN 2018: AI VS. GDPR
Big Data LDN 2018: AI VS. GDPRBig Data LDN 2018: AI VS. GDPR
Big Data LDN 2018: AI VS. GDPR
 
Big Data LDN 2018: REALISING THE PROMISE OF SELF-SERVICE ANALYTICS WITH DATA ...
Big Data LDN 2018: REALISING THE PROMISE OF SELF-SERVICE ANALYTICS WITH DATA ...Big Data LDN 2018: REALISING THE PROMISE OF SELF-SERVICE ANALYTICS WITH DATA ...
Big Data LDN 2018: REALISING THE PROMISE OF SELF-SERVICE ANALYTICS WITH DATA ...
 
Big Data LDN 2018: TURNING MULTIPLE DATA LAKES INTO A UNIFIED ANALYTIC DATA L...
Big Data LDN 2018: TURNING MULTIPLE DATA LAKES INTO A UNIFIED ANALYTIC DATA L...Big Data LDN 2018: TURNING MULTIPLE DATA LAKES INTO A UNIFIED ANALYTIC DATA L...
Big Data LDN 2018: TURNING MULTIPLE DATA LAKES INTO A UNIFIED ANALYTIC DATA L...
 
Big Data LDN 2018: MICROSOFT AZURE AND CLOUDERA – FLEXIBLE CLOUD, WHATEVER TH...
Big Data LDN 2018: MICROSOFT AZURE AND CLOUDERA – FLEXIBLE CLOUD, WHATEVER TH...Big Data LDN 2018: MICROSOFT AZURE AND CLOUDERA – FLEXIBLE CLOUD, WHATEVER TH...
Big Data LDN 2018: MICROSOFT AZURE AND CLOUDERA – FLEXIBLE CLOUD, WHATEVER TH...
 
Big Data LDN 2018: CONSISTENT SECURITY, GOVERNANCE AND FLEXIBILITY FOR ALL WO...
Big Data LDN 2018: CONSISTENT SECURITY, GOVERNANCE AND FLEXIBILITY FOR ALL WO...Big Data LDN 2018: CONSISTENT SECURITY, GOVERNANCE AND FLEXIBILITY FOR ALL WO...
Big Data LDN 2018: CONSISTENT SECURITY, GOVERNANCE AND FLEXIBILITY FOR ALL WO...
 
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICSBig Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
 
Big Data LDN 2018: EXPERIAN: MAXIMISE EVERY OPPORTUNITY IN THE BIG DATA UNIVERSE
Big Data LDN 2018: EXPERIAN: MAXIMISE EVERY OPPORTUNITY IN THE BIG DATA UNIVERSEBig Data LDN 2018: EXPERIAN: MAXIMISE EVERY OPPORTUNITY IN THE BIG DATA UNIVERSE
Big Data LDN 2018: EXPERIAN: MAXIMISE EVERY OPPORTUNITY IN THE BIG DATA UNIVERSE
 
Big Data LDN 2018: A LOOK INSIDE APPLIED MACHINE LEARNING
Big Data LDN 2018: A LOOK INSIDE APPLIED MACHINE LEARNINGBig Data LDN 2018: A LOOK INSIDE APPLIED MACHINE LEARNING
Big Data LDN 2018: A LOOK INSIDE APPLIED MACHINE LEARNING
 
Big Data LDN 2018: DEUTSCHE BANK: THE PATH TO AUTOMATION IN A HIGHLY REGULATE...
Big Data LDN 2018: DEUTSCHE BANK: THE PATH TO AUTOMATION IN A HIGHLY REGULATE...Big Data LDN 2018: DEUTSCHE BANK: THE PATH TO AUTOMATION IN A HIGHLY REGULATE...
Big Data LDN 2018: DEUTSCHE BANK: THE PATH TO AUTOMATION IN A HIGHLY REGULATE...
 
Big Data LDN 2018: FROM PROLIFERATION TO PRODUCTIVITY: MACHINE LEARNING DATA ...
Big Data LDN 2018: FROM PROLIFERATION TO PRODUCTIVITY: MACHINE LEARNING DATA ...Big Data LDN 2018: FROM PROLIFERATION TO PRODUCTIVITY: MACHINE LEARNING DATA ...
Big Data LDN 2018: FROM PROLIFERATION TO PRODUCTIVITY: MACHINE LEARNING DATA ...
 
Big Data LDN 2018: DATA APIS DON’T DISCRIMINATE
Big Data LDN 2018: DATA APIS DON’T DISCRIMINATEBig Data LDN 2018: DATA APIS DON’T DISCRIMINATE
Big Data LDN 2018: DATA APIS DON’T DISCRIMINATE
 

Último

Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Klinik kandungan
 
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
vexqp
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
gajnagarg
 
Abortion pills in Jeddah | +966572737505 | Get Cytotec
Abortion pills in Jeddah | +966572737505 | Get CytotecAbortion pills in Jeddah | +966572737505 | Get Cytotec
Abortion pills in Jeddah | +966572737505 | Get Cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样
wsppdmt
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
wsppdmt
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
nirzagarg
 
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
vexqp
 
怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制
怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制
怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制
vexqp
 
怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制
怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制
怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制
vexqp
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
nirzagarg
 
PLE-statistics document for primary schs
PLE-statistics document for primary schsPLE-statistics document for primary schs
PLE-statistics document for primary schs
cnajjemba
 

Último (20)

Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
 
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptxThe-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
 
Abortion pills in Jeddah | +966572737505 | Get Cytotec
Abortion pills in Jeddah | +966572737505 | Get CytotecAbortion pills in Jeddah | +966572737505 | Get Cytotec
Abortion pills in Jeddah | +966572737505 | Get Cytotec
 
一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书)加州大学戴维斯分校毕业证成绩单原件一模一样
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
 
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
 
怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制
怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制
怎样办理旧金山城市学院毕业证(CCSF毕业证书)成绩单学校原版复制
 
Sequential and reinforcement learning for demand side management by Margaux B...
Sequential and reinforcement learning for demand side management by Margaux B...Sequential and reinforcement learning for demand side management by Margaux B...
Sequential and reinforcement learning for demand side management by Margaux B...
 
怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制
怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制
怎样办理伦敦大学毕业证(UoL毕业证书)成绩单学校原版复制
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for Research
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
PLE-statistics document for primary schs
PLE-statistics document for primary schsPLE-statistics document for primary schs
PLE-statistics document for primary schs
 
Capstone in Interprofessional Informatic // IMPACT OF COVID 19 ON EDUCATION
Capstone in Interprofessional Informatic  // IMPACT OF COVID 19 ON EDUCATIONCapstone in Interprofessional Informatic  // IMPACT OF COVID 19 ON EDUCATION
Capstone in Interprofessional Informatic // IMPACT OF COVID 19 ON EDUCATION
 

Big Data LDN 2018: SCALING A PLATFORM FOR REAL-TIME FRAUD DETECTION WITHOUT BREAKING THE BANK

  • 1. BERLIN • NEW YORK • SAN FRANCISCO • SÃO PAULO • PARIS • LONDON • MOSCOW • ISTANBUL SEOUL • SHANGHAI • BEIJING • TOKYO • MUMBAI • SINGAPORE Scaling a platform for real-time fraud detection
  • 2. 2 Why is no one prepared for success?
  • 3. 3 ‣ Focus on idea ‣ Focus on execution is limited in time to market ‣ Underestimating importance of scalability and profitability Startup Zeitgeist
  • 4. 4 ‣ Director of Engineering ‣ Consulted with early design decisions ‣ Joined Adjust 2012 as Head of IT Operations Who am I?
  • 5. Who is Adjust? 1B+ Daily active users tracked 400+ Billion Data points tracked monthly 22K+ Apps tracked
  • 6. 6 ‣ Automatically reject fraudulent data before it gets paid ‣ Sends rejection reason callbacks to all parties ‣ Customers and their networks have full transparency ‣ Real-time statistical analysis of all ad engagements and app activity Fraud Prevention Suite
  • 7. 7 How did we do that?
  • 8. 8 Bootstrapping a product Current approach ‣ Using cloud services ‣ Base product on 3rd party IP ‣ Faster development
  • 9. 9 Bootstrapping a product Current approach Our approach ‣ Using cloud services ‣ Base product on 3rd party IP ‣ Faster development ‣ Own infrastructure ‣ Develop own IP ‣ Slower ramp up
  • 10. 10 Bootstrapping a product Current approach Our approach ‣ Using cloud services ‣ Base product on 3rd party IP ‣ Faster development ‣ Potential lock in ‣ More cost efficient in the beginning ‣ Shared environment ‣ Own infrastructure ‣ Develop own IP ‣ Slower ramp up
  • 11. 11 Bootstrapping a product Current approach Our approach ‣ Using cloud services ‣ Base product on 3rd party IP ‣ Faster development ‣ Potential lock in ‣ More cost efficient in the beginning ‣ Shared environment ‣ Own infrastructure ‣ Develop own IP ‣ Slower ramp up ‣ Keep flexibility ‣ More cost efficient in the long run ‣ Dedicated environment
  • 13. 13 General Overview Realtime Callbacks Raw Data Upload Realtime Fraud Prevention Realtime Aggregation SDK Traffic Tracking Server Device Data Store
  • 16. 16 A little more please?
  • 18. 18 We need to go further
  • 19. 19 It’s all connected :) Leader 1 Replica 1 Leader n Replica n Sentinel 1 Sentinel n twemproxy twemproxy
  • 20. 20 And we felt like…
  • 21. 21
  • 22. 22 ‣ Automation is tricky Smoke in the machine room
  • 23. 23 ‣ Automation is tricky ‣ Weird latency spikes Smoke in the machine room
  • 24. 24 ‣ Automation is tricky ‣ Weird latency spikes ‣ Increased average response time Smoke in the machine room
  • 25. 25 ‣ Automation is tricky ‣ Weird latency spikes ‣ Increased average response time ‣ Long failover times ‣ Disruptive failovers Smoke in the machine room
  • 26. 26 And we felt like…
  • 27. 27
  • 28. 28 Sometimes you have to let go…
  • 29. 29 ‣ Only index is in RAM ‣ Data is on SSD ‣ Cost reduced by 85% ‣ Server count reduced from 40 to 6 Migrating to Aerospike
  • 30. 30 ‣ 1,1 million reads per second ‣ 500.000 writes per second ‣ 180TB of data ‣ 3 locations Using Aerospike
  • 32. 32 ‣ Easy to scale ‣ Redis interface ‣ Online resizing ‣ All dirty work is done by Amazon Elasticache
  • 33. 33 Q: Can I use Amazon ElastiCache for use cases other than caching? A: Yes. ElastiCache for Redis can be used as a primary in-memory key-value data store, providing fast, sub millisecond data performance, high availability and scalability up to 15 nodes plus up to 5 read replicas, each of up to 9.5 TiB of in-memory data.
  • 34. 34 “Once we are successful, we will take care of scalability and profitability”
  • 35. 35 What will happen to my infrastructure when I will be successful?
  • 37. New York Paris São Paulo San Francisco London Berlin Istanbul Moscow Mumbai Beijing Seoul Tokyo Shanghai Singapore Robert Abraham DIRECTOR OF ENGINEERING
 robert@adjust.com ADJUST HQ
 Saarbrücker Str. 37a
 10405 Berlin
 Germany