Sifting, Sorting and Scanning for Gold – The Real-Time User Data Problem

•

0 recomendaciones•261 vistas

Brian Bulkowski, founder and CEO of Citrusleaf, will focus on the role of innovative NoSQL databases in advertising and the problem of real-time user data. The talk will feature case studies from online and mobile advertising companies who are facing data challenges of 100s of terabytes and requirements for millisecond response times. He will discuss the data problems in real-time bidding applications and present some ideas for moving beyond the last click attribution model. He will illustrate why scalability and speed are so critical and how new technology approaches are driving innovation in digital advertising.

Tecnología Empresariales

Citrusleaf
“Sifting, Sorting and Scanning for Gold –
g, g g
The Real‐Time User Data Problem”

Brian Bulkowski, CEO and Founder

© 2011 Citrusleaf. All rights reserved. 1

NoSQL in Digital Advertising

• Display advertising – today’s state of the art
Display advertising today s state of the art
• Real‐time data in digital advertising
• C
Case study
d
– User data store & cookie mapping
• Market differentiation
• Moving beyond the last click
g y

Citrusleaf Confidential © 2011 Citrusleaf. All rights reserved. 2

Display Advertising
• Lie:
– Content determines the value of an impression
• Lie:
– The value of an individual user is constant
– The decision funnel
The decision funnel
• Truth:
– The right ad to the right user at the right time
The right ad, to the right user, at the right time

© 2011 Citrusleaf. All rights reserved. 3

Today’s state of the art

• Impression logs => user segments (warehouse)
• Third part data => user segments
• Region segments
• Real‐time per‐user factor
• Frequency caps
• Session management for cookie less users
Session management for cookie‐less users
• Mapping external partner IDs
• Build optimization tables based on log analysis
Build optimization tables based on log analysis

Simple math determines highest value ad
Simple math determines highest value ad
© 2011 Citrusleaf. All rights reserved. 4

Real‐time data requirements
• Scalable and
flexible
fl ibl
• 100% availability
• Billion object
support
• High performance
with low hardware
cost
• Sophisticated data
eviction policies
© 2010 Citrusleaf. All rights reserved. 5

Case Study
Map store User store
Key
y cookie string or
g Key internal user id
partner id Value Segment data
Value internal user id (8 bytes) Frequency caps
Other optimization data

North America North America
1B ~ 1.5B objects 500M ~ 800M objects
Write load:
Write load: Write load:
Write load:
1k ~ 2k per second 10K ~ 20K per second
Read load: Read load:
10k ~ 50k per second
p 20k ~ 100k per second

Configuration Configuration
DRAM backed by disk
DRAM backed by disk 1T 4T user storage
1T ~ 4T user storage
Lowest latency (0.4 ms) Low latency (0.8 ms)
© 2011 Citrusleaf. All rights reserved. 6

Market differentiation

• Control over impressions
Control over impressions

• Quality and variety of inventory
y y y

• Better insight generation & tracking
• And:
– High quality user understanding

Trend: multiple user stores

© 2011 Citrusleaf. All rights reserved. 7

The next challenge
Weakness: “last ad takes all”
Performance is judged externally by the last
P f i j d d t ll b th l t
advertiser before a conversion

So… game the system to place strategically
New technology allows real‐time understanding of
user behavior

But …
Will advertisers stand for it?

© 2011 Citrusleaf. All rights reserved. 8

Indexed user behavior storage
• Store 100B+ user
behavior objects
b h i bj t
(1B per day, 90
days)
• Index by user_id,
giving sub‐second
access to entire
behavior chain
• O ti i d f
Optimized for
rotational media
• Time‐based eviction
Time‐based eviction
© 2010 Citrusleaf. All rights reserved. 9

Indexed user behavior storage
• Solves the “needle in a haystack” problem
• Allows immediate behavioral triggering
All i di b h i l i i
• Removes ETL and Map/Reduce scans
• Allows sophisticated attribution models,
(and advanced optimization and reporting)

The right ad,
g ,
to the right user,
at the right time
t th i ht ti
© 2011 Citrusleaf. All rights reserved. 10

Citrusleaf
Making Web Scale Easy and Affordable
M ki W b S l E d Aff d bl

Contact Info: brian@citrusleaf.com
Contact Info: brian@citrusleaf com

© 2011 Citrusleaf. All rights reserved. 11

Más contenido relacionado

Más de DATAVERSITY

Organizations today need a broad set of enterprise data cloud services with key data functionality to modernize applications and utilize machine learning. They need a comprehensive platform designed to address multi-faceted needs by offering multi-function data management and analytics to solve the enterprise’s most pressing data and analytic challenges in a streamlined fashion. In this research-based session, I’ll discuss what the components are in multiple modern enterprise analytics stacks (i.e., dedicated compute, storage, data integration, streaming, etc.) and focus on total cost of ownership. A complete machine learning infrastructure cost for the first modern use case at a midsize to large enterprise will be anywhere from $3 million to $22 million. Get this data point as you take the next steps on your journey into the highest spend and return item for most companies in the next several years.

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...

DATAVERSITY

Do you ever wonder how data-driven organizations fuel analytics, improve customer experience, and accelerate business productivity? They are successful by governing and mastering data effectively so they can get trusted data to those who need it faster. Efficient data discovery, mastering and democratization is critical for swiftly linking accurate data with business consumers. When business teams can quickly and easily locate, interpret, trust, and apply data assets to support sound business judgment, it takes less time to see value. Join data mastering and data governance experts from Informatica—plus a real-world organization empowering trusted data for analytics—for a lively panel discussion. You’ll hear more about how a single cloud-native approach can help global businesses in any economy create more value—faster, more reliably, and with more confidence—by making data management and governance easier to implement.

Data at the Speed of Business with Data Mastering and Governance

DATAVERSITY

What is data literacy? Which organizations, and which workers in those organizations, need to be data-literate? There are seemingly hundreds of definitions of data literacy, along with almost as many opinions about how to achieve it. In a broader perspective, companies must consider whether data literacy is an isolated goal or one component of a broader learning strategy to address skill deficits. How does data literacy compare to other types of skills or “literacy” such as business acumen? This session will position data literacy in the context of other worker skills as a framework for understanding how and where it fits and how to advocate for its importance.

Exploring Levels of Data Literacy

DATAVERSITY

Developing a Data Strategy for your organization can seem like a daunting task – but it’s worth the effort. Getting your Data Strategy right can provide significant value, as data drives many of the key initiatives in today’s marketplace – from digital transformation, to marketing, to customer centricity, to population health, and more. This webinar will help demystify Data Strategy and its relationship to Data Architecture and will provide concrete, practical ways to get started.

Building a Data Strategy – Practical Steps for Aligning with Business Goals

DATAVERSITY

Uncover how your business can save money and find new revenue streams. Driving profitability is a top priority for companies globally, especially in uncertain economic times. It's imperative that companies reimagine growth strategies and improve process efficiencies to help cut costs and drive revenue – but how? By leveraging data-driven strategies layered with artificial intelligence, companies can achieve untapped potential and help their businesses save money and drive profitability. In this webinar, you'll learn: - How your company can leverage data and AI to reduce spending and costs - Ways you can monetize data and AI and uncover new growth strategies - How different companies have implemented these strategies to achieve cost optimization benefits

Make Data Work for You

DATAVERSITY

Organizations with governed metadata made available through their data catalog can answer questions their people have about the organization’s data. These organizations get more value from their data, protect their data better, gain improved ROI from data-centric projects and programs, and have more confidence in their most strategic data. Join Bob Seiner for this lively webinar where he will talk about the value of a data catalog and how to build the use of the catalog into your stewards’ daily routines. Bob will share how the tool must be positioned for success and viewed as a must-have resource that is a steppingstone and catalyst to governed data across the organization.

Data Catalogs Are the Answer – What is the Question?

DATAVERSITY

Data Catalogs Are the Answer – What Is the Question?

DATAVERSITY

Because every organization produces and propagates data as part of their day-to-day operations, data trends are becoming more and more important in the mainstream business world’s consciousness. For many organizations in various industries, though, comprehension of this development begins and ends with buzzwords: “Big Data,” “NoSQL,” “Data Scientist,” and so on. Few realize that all solutions to their business problems, regardless of platform or relevant technology, rely to a critical extent on the data model supporting them. As such, data modeling is not an optional task for an organization’s data effort, but rather a vital activity that facilitates the solutions driving your business. Since quality engineering/architecture work products do not happen accidentally, the more your organization depends on automation, the more important the data models driving the engineering and architecture activities of your organization. This webinar illustrates data modeling as a key activity upon which so much technology and business investment depends. Specific learning objectives include: - Understanding what types of challenges require data modeling to be part of the solution - How automation requires standardization on derivable via data modeling techniques - Why only a working partnership between data and the business can produce useful outcomes

Data Modeling Fundamentals

DATAVERSITY

Analytics play a critical role in supporting strategic business initiatives. Despite the obvious value to analytic professionals of providing the analytics for these initiatives, many executives question the economic return of analytics as well as data lakes, machine learning, master data management, and the like. Technology professionals need to calculate and present business value in terms business executives can understand. Unfortunately, most IT professionals lack the knowledge required to develop comprehensive cost-benefit analyses and return on investment (ROI) measurements. This session provides a framework to help technology professionals research, measure, and present the economic value of a proposed or existing analytics initiative, no matter the form that the business benefit arises. The session will provide practical advice about how to calculate ROI and the formulas, and how to collect the necessary information.

Showing ROI for Your Analytic Project

DATAVERSITY

Data Mesh is a trending approach to building a decentralized data architecture by leveraging a domain-oriented, self-service design. However, the pure definition of Data Mesh lacks a center of excellence or central data team and doesn’t address the need for a common approach for sharing data products across teams. The semantic layer is emerging as a key component to supporting a Hub and Spoke style of organizing data teams by introducing data model sharing, collaboration, and distributed ownership controls. This session will explain how data teams can define common models and definitions with a semantic layer to decentralize analytics product creation using a Hub and Spoke architecture. Attend this session to learn about: - The role of a Data Mesh in the modern cloud architecture. - How a semantic layer can serve as the binding agent to support decentralization. - How to drive self service with consistency and control.

How a Semantic Layer Makes Data Mesh Work at Scale

DATAVERSITY

Enterprise data literacy. A worthy objective? Certainly! A realistic goal? That remains to be seen. As companies consider investing in data literacy education, questions arise about its value and purpose. While the destination – having a data-fluent workforce – is attractive, we wonder how (and if) we can get there. Kicking off this webinar series, we begin with a panel discussion to explore the landscape of literacy, including expert positions and results from focus groups: - why it matters, - what it means, - what gets in the way, - who needs it (and how much they need), - what companies believe it will accomplish. In this engaging discussion about literacy, we will set the stage for future webinars to answer specific questions and feature successful literacy efforts.

Is Enterprise Data Literacy Possible?

DATAVERSITY

Change is hard, especially in response to negative stimuli or what is perceived as negative stimuli. So organizations need to reframe how they think about data privacy, security and governance, treating them as value centers to 1) ensure enterprise data can flow where it needs to, 2) prevent – not just react – to internal and external threats, and 3) comply with data privacy and security regulations. Working together, these roles can accelerate faster access to approved, relevant and higher quality data – and that means more successful use cases, faster speed to insights, and better business outcomes. However, both new information and tools are required to make the shift from defense to offense, reducing data drama while increasing its value. Join us for this panel discussion with experts in these fields as they discuss: - Recent research about where data privacy, security and governance stand - The most valuable enterprise data use cases - The common obstacles to data value creation - New approaches to data privacy, security and governance - Their advice on how to shift from a reactive to resilient mindset/culture/organization You’ll be educated, entertained and inspired by this panel and their expertise in using the data trifecta to innovate more often, operate more efficiently, and differentiate more strategically.

The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...

DATAVERSITY

Emerging Trends in Data Architecture – What’s the Next Big Thing?

DATAVERSITY

As DATAVERSITY’s RWDG series hurdles into our 12th year, this webinar takes a quick look behind us, evaluates the present, and predicts the future of Data Governance. Based on webinar numbers, hot Data Governance topics have evolved over the years from policies and best practices, roles and tools, data catalogs and frameworks, to supporting data mesh and fabric, artificial intelligence, virtualization, literacy, and metadata governance. Join Bob Seiner as he reflects on the past and what has and has not worked, while sharing examples of enterprise successes and struggles. In this webinar, Bob will challenge the audience to stay a step ahead by learning from the past and blazing a new trail into the future of Data Governance. In this webinar, Bob will focus on: - Data Governance’s past, present, and future - How trials and tribulations evolve to success - Leveraging lessons learned to improve productivity - The great Data Governance tool explosion - The future of Data Governance

Data Governance Trends - A Look Backwards and Forwards

DATAVERSITY

Would you share your bank account information on social media? How about shouting your social security number on the New York City subway? We didn’t think so either – that’s why data governance is consistently top of mind. In this webinar, we’ll discuss the common Cloud data governance best practices – and how to apply them today. Join us to uncover Google Cloud’s investment in data governance and learn practical and doable methods around key management and confidential computing. Hear real customer experiences and leave with insights that you can share with your team. Let’s get solving. Topics that you will hear addressed in this webinar: - Understanding the basics of Cloud Incident Response (IR) and anticipated data governance trends - Best practices for key management and apply data governance to your day-to-day - The next wave of Confidential Computing and how to get started, including a demo

Data Governance Trends and Best Practices To Implement Today

DATAVERSITY

It is a fascinating, explosive time for enterprise analytics. It is from the position of analytics leadership that the enterprise mission will be executed and company leadership will emerge. The data professional is absolutely sitting on the performance of the company in this information economy and has an obligation to demonstrate the possibilities and originate the architecture, data, and projects that will deliver analytics. After all, no matter what business you’re in, you’re in the business of analytics. The coming years will be full of big changes in enterprise analytics and data architecture. William will kick off the fifth year of the Advanced Analytics series with a discussion of the trends winning organizations should build into their plans, expectations, vision, and awareness now.

2023 Trends in Enterprise Analytics

DATAVERSITY

Too often I hear the question “Can you help me with our data strategy?” Unfortunately, for most, this is the wrong request because it focuses on the least valuable component: the data strategy itself. A more useful request is: “Can you help me apply data strategically?” Yes, at early maturity phases the process of developing strategic thinking about data is more important than the actual product! Trying to write a good (must less perfect) data strategy on the first attempt is generally not productive –particularly given the widespread acceptance of Mike Tyson’s truism: “Everybody has a plan until they get punched in the face.” This program refocuses efforts on learning how to iteratively improve the way data is strategically applied. This will permit data-based strategy components to keep up with agile, evolving organizational strategies. It also contributes to three primary organizational data goals. Learn how to improve the following: - Your organization’s data - The way your people use data - The way your people use data to achieve your organizational strategy This will help in ways never imagined. Data are your sole non-depletable, non-degradable, durable strategic assets, and they are pervasively shared across every organizational area. Addressing existing challenges programmatically includes overcoming necessary but insufficient prerequisites and developing a disciplined, repeatable means of improving business objectives. This process (based on the theory of constraints) is where the strategic data work really occurs as organizations identify prioritized areas where better assets, literacy, and support (data strategy components) can help an organization better achieve specific strategic objectives. Then the process becomes lather, rinse, and repeat. Several complementary concepts are also covered, including: - A cohesive argument for why data strategy is necessary for effective data governance - An overview of prerequisites for effective strategic use of data strategy, as well as common pitfalls - A repeatable process for identifying and removing data constraints - The importance of balancing business operation and innovation

Data Strategy Best Practices

DATAVERSITY

The question is asked all the time: “What part of the organization should own your Data Governance program?” The typical answers are “the business” and “IT (information technology).” Another answer to that question is “Yes.” The program must be owned and reside somewhere in the organization. You may ask yourself if there is a correct answer to the question. Join this new RWDG webinar with Bob Seiner where Bob will answer the question that is the title of this webinar. Determining ownership of Data Governance is a vital first step. Figuring out the appropriate part of the organization to manage the program is an important second step. This webinar will help you address these questions and more. In this session Bob will share: - What is meant by “the business” when it comes to owning Data Governance - Why some people say that Data Governance in IT is destined to fail - Examples of IT positioned Data Governance success - Considerations for answering the question in your organization - The final answer to the question of who should own Data Governance

Who Should Own Data Governance – IT or Business?

DATAVERSITY

It is clear that Data Management best practices exist and so does a useful process for improving existing Data Management practices. The question arises: Since we understand the goal, how does one design a process for Data Management goal achievement? This program describes what must be done at the programmatic level to achieve better data use and a way to implement this as part of your data program. The approach combines DMBoK content and CMMI/DMM processes – permitting organizations with the opportunity to benefit from the best of both. It also permits organizations to understand: - Their current Data Management practices - Strengths that should be leveraged - Remediation opportunities

Data Management Best Practices

DATAVERSITY

MLOps is a practice for collaboration between Data Science and operations to manage the production machine learning (ML) lifecycles. As an amalgamation of “machine learning” and “operations,” MLOps applies DevOps principles to ML delivery, enabling the delivery of ML-based innovation at scale to result in: Faster time to market of ML-based solutions More rapid rate of experimentation, driving innovation Assurance of quality, trustworthiness, and ethical AI MLOps is essential for scaling ML. Without it, enterprises risk struggling with costly overhead and stalled progress. Several vendors have emerged with offerings to support MLOps: the major offerings are Microsoft Azure ML and Google Vertex AI. We looked at these offerings from the perspective of enterprise features and time-to-value.

MLOps – Applying DevOps to Competitive Advantage

DATAVERSITY

Más de DATAVERSITY (20)

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...

Data at the Speed of Business with Data Mastering and Governance

Exploring Levels of Data Literacy

Building a Data Strategy – Practical Steps for Aligning with Business Goals

Make Data Work for You

Data Catalogs Are the Answer – What is the Question?

Data Catalogs Are the Answer – What Is the Question?

Data Modeling Fundamentals

Showing ROI for Your Analytic Project

How a Semantic Layer Makes Data Mesh Work at Scale

Is Enterprise Data Literacy Possible?

The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...

Emerging Trends in Data Architecture – What’s the Next Big Thing?

Data Governance Trends - A Look Backwards and Forwards

Data Governance Trends and Best Practices To Implement Today

2023 Trends in Enterprise Analytics

Data Strategy Best Practices

Who Should Own Data Governance – IT or Business?

Data Management Best Practices

MLOps – Applying DevOps to Competitive Advantage

Último

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Scaling API-first – The story of a global engineering organization

Radu Cotescu

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

Real Time Object Detection Using Open CV

Khem

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Sifting, Sorting and Scanning for Gold – The Real-Time User Data Problem

2. NoSQL in Digital Advertising • Display advertising – today’s state of the art Display advertising today s state of the art • Real‐time data in digital advertising • C Case study d – User data store & cookie mapping • Market differentiation • Moving beyond the last click g y Citrusleaf Confidential © 2011 Citrusleaf. All rights reserved. 2

3. Display Advertising • Lie: – Content determines the value of an impression • Lie: – The value of an individual user is constant – The decision funnel The decision funnel • Truth: – The right ad to the right user at the right time The right ad, to the right user, at the right time © 2011 Citrusleaf. All rights reserved. 3

4. Today’s state of the art • Impression logs => user segments (warehouse) • Third part data => user segments • Region segments • Real‐time per‐user factor • Frequency caps • Session management for cookie less users Session management for cookie‐less users • Mapping external partner IDs • Build optimization tables based on log analysis Build optimization tables based on log analysis Simple math determines highest value ad Simple math determines highest value ad © 2011 Citrusleaf. All rights reserved. 4

5. Real‐time data requirements • Scalable and flexible fl ibl • 100% availability • Billion object support • High performance with low hardware cost • Sophisticated data eviction policies © 2010 Citrusleaf. All rights reserved. 5

6. Case Study Map store User store Key y cookie string or g Key internal user id partner id Value Segment data Value internal user id (8 bytes) Frequency caps Other optimization data North America North America 1B ~ 1.5B objects 500M ~ 800M objects Write load: Write load: Write load: Write load: 1k ~ 2k per second 10K ~ 20K per second Read load: Read load: 10k ~ 50k per second p 20k ~ 100k per second Configuration Configuration DRAM backed by disk DRAM backed by disk 1T 4T user storage 1T ~ 4T user storage Lowest latency (0.4 ms) Low latency (0.8 ms) © 2011 Citrusleaf. All rights reserved. 6

7. Market differentiation • Control over impressions Control over impressions • Quality and variety of inventory y y y • Better insight generation & tracking • And: – High quality user understanding Trend: multiple user stores © 2011 Citrusleaf. All rights reserved. 7

8. The next challenge Weakness: “last ad takes all” Performance is judged externally by the last P f i j d d t ll b th l t advertiser before a conversion So… game the system to place strategically New technology allows real‐time understanding of user behavior But … Will advertisers stand for it? © 2011 Citrusleaf. All rights reserved. 8

9. Indexed user behavior storage • Store 100B+ user behavior objects b h i bj t (1B per day, 90 days) • Index by user_id, giving sub‐second access to entire behavior chain • O ti i d f Optimized for rotational media • Time‐based eviction Time‐based eviction © 2010 Citrusleaf. All rights reserved. 9

10. Indexed user behavior storage • Solves the “needle in a haystack” problem • Allows immediate behavioral triggering All i di b h i l i i • Removes ETL and Map/Reduce scans • Allows sophisticated attribution models, (and advanced optimization and reporting) The right ad, g , to the right user, at the right time t th i ht ti © 2011 Citrusleaf. All rights reserved. 10

Sifting, Sorting and Scanning for Gold – The Real-Time User Data Problem

Recomendados

Recomendados

Más contenido relacionado

Más de DATAVERSITY

Más de DATAVERSITY (20)

Último

Último (20)

Sifting, Sorting and Scanning for Gold – The Real-Time User Data Problem