Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Distributed Quota Management
in an Ad Serving Environment
How Reduce Data Manages its budgets without exceeding ad spends
...
Problem
Managing Quotas / Budgets in An Ad Serving
Environment is not a trivial task.
Ad Campaign Budgets Across a
Distrib...
Previous Solution
Sync Every 5 seconds
Cassandra Data

Ad Server
Ad Server
Ad Server

Kafka Pipe

Data processing
system

...
The Issues
Simple Syncing of data across distributed data structure is easy but sticking
within limits is tough.
5 second ...
Solution
●

●

●

Our Solution was based on this white paper
titled:
Quota enforcement for high-performance
distributed st...
Key Concepts
●
●

●

●

Epoc : A day is divided into epocs.
At each epoc DSPServers request for quota from Quota
Managemen...
How Quotas are Decided
●

●

Quota is distributed to end points (different data centre locations) and to
the individual se...
Quota Management Cluster
●

●

Quota management cluster has multiple servers and needs
a data store which supports transac...
Logical Architecture
Sync at every EPOC 2 ways
Ad Server
Ad Server
Ad Server

Ad servers can sync with the Quota managemen...
Sync Process
Ad Server

Quota Management Server

Zookeeper

Cassandra

Sync at every EPOC
Push budgets, get unspent amount...
Handling Ad Server Shut Downs
●

●

Zookeeper tracks server failures.
Quota management system reallocates remaining budget...
Handling Communications
The Quota Management Servers talk to Ad Servers using
REST Calls. This required lowest development...
Get in Touch
Engineering Team
Reduce Data
reducedata.com
twitter: @reducedata
facebook.com/reducedata
engineering@reduceda...
Próxima SlideShare
Cargando en…5
×

Distributed Quota Management in an Ad Ad Serving Environment OR How we manage ad spends across data centres without over spending

Distributed Quota Management in an Ad Ad Serving Environment OR How we manage ad spends across data centres without over spending

  • Inicia sesión para ver los comentarios

Distributed Quota Management in an Ad Ad Serving Environment OR How we manage ad spends across data centres without over spending

  1. 1. Distributed Quota Management in an Ad Serving Environment How Reduce Data Manages its budgets without exceeding ad spends http://reducedata.com
  2. 2. Problem Managing Quotas / Budgets in An Ad Serving Environment is not a trivial task. Ad Campaign Budgets Across a Distributed infrastructure when not managed properly leads to excess spends.
  3. 3. Previous Solution Sync Every 5 seconds Cassandra Data Ad Server Ad Server Ad Server Kafka Pipe Data processing system Hbase based campaign reporting data
  4. 4. The Issues Simple Syncing of data across distributed data structure is easy but sticking within limits is tough. 5 second refresh not guaranteed to be upto date. Reporting delays caused spends to exceed. Server specific budget was not proportionately allocated, based on historical availability of traffic on that server. No clear mechanism of allocation of budget spends or reallocation (when servers go down).
  5. 5. Solution ● ● ● Our Solution was based on this white paper titled: Quota enforcement for high-performance distributed storage systems http://www.ssrc.ucsc.edu/Papers/pollackmsst07.pdf
  6. 6. Key Concepts ● ● ● ● Epoc : A day is divided into epocs. At each epoc DSPServers request for quota from Quota Management Cluster. Ad Server doesn't decide how much quota it needs instead a new entity the Quota Management Cluster decides how much quota to assign to it. Traffic for a Campaign maybe available at multiple endpoints(US East, US West, Asia)
  7. 7. How Quotas are Decided ● ● Quota is distributed to end points (different data centre locations) and to the individual servers depending on the historical traffic / consumption of budgets. Two factors that decide how much is allocated in each epoc: ● ● Pacing variable as decided by the advertiser Historical consumption data (for example has the campaign been spending lesser than planned?, if so, increase EPOC allocation)
  8. 8. Quota Management Cluster ● ● Quota management cluster has multiple servers and needs a data store which supports transactions. The Quota Management Cluster is Synchronized using Zookeper.
  9. 9. Logical Architecture Sync at every EPOC 2 ways Ad Server Ad Server Ad Server Ad servers can sync with the Quota management server at EPOC which could be say every hour. Ad servers refund unspent balance to Quota Management Cluster. Quota management can be in one location initially but can be made redundant to each data centre. Quota Management Cluster uses a shared data store on cassandra to sync its other nodes for allocation. Quota Quota Management Management Server Server Cassandra based Cassandra Quota data based Quota data Zookeeper Zookeeper
  10. 10. Sync Process Ad Server Quota Management Server Zookeeper Cassandra Sync at every EPOC Push budgets, get unspent amounts Sync allocation data to cassandra cluster Check Server status Get list of available ad servers Push available budgets on a restart. Calculate available spends
  11. 11. Handling Ad Server Shut Downs ● ● Zookeeper tracks server failures. Quota management system reallocates remaining budget (last budget allocated - amount spent) to a new or the same server when it restarts.
  12. 12. Handling Communications The Quota Management Servers talk to Ad Servers using REST Calls. This required lowest development time and was built on top of the existing stack. Alternatives we considered but decided against. 2. Kafka 3. RPC over protobuf
  13. 13. Get in Touch Engineering Team Reduce Data reducedata.com twitter: @reducedata facebook.com/reducedata engineering@reducedata.com

×