SlideShare una empresa de Scribd logo
1 de 12
Descargar para leer sin conexión
Copyright © 2015 Criteo
New machine learning challenges at Criteo
Olivier Koch
Engineering Program Manager, Criteo
Rythm Meetup
June 15, 2016
Copyright © 2015 Criteo
Banners… what else?
2
Advertiser Publisher
Copyright © 2015 Criteo
Machine learning applications at Criteo
• Bidding (2nd price auctions)
• Product recommendation
• Banner look and feel selection
Copyright © 2015 Criteo
Machine learning at Criteo
• Supervised learning using standard regression methods / optimization algorithms (SGD, L-BFGS)
• Distribution on Hadoop (MapReduce, Spark)
• 3B displays / day
• 40 PB of data -- 15,000 servers
• 7 data centers worldwide
Copyright © 2015 Criteo
Data sparsity
10 000 displays
lead to
50 clicks
lead to
1 sale
Copyright © 2015 Criteo
Now what?
Copyright © 2015 Criteo
Challenges in online advertising
• We have an impact on users
• A user is seen more than 20 times a day in average
• Every bid has an influence on our competitors
• We want to provide a better online advertising experience
• Personalized
• Cross-device
• Long tail (new users, new products)
Copyright © 2015 Criteo
Machine learning challenges
• Optimal bidding strategies under uncertainty -- reinforcement learning, policy learning
• Probabilistic match of devices
• Classification/prediction of time series
• Long tail (users, products) -- transfer learning, factorization
• Offline metrics – counterfactual analysis
Copyright © 2015 Criteo
The good news
• New generations of algorithms
• NLP (word embeddings), reinforcement learning, policy learning, deep networks
• Releases of ML infrastructures
• Caffe on Spark, TensorFlow, Torch, PhotonML, GPUs inside clusters
→ strong traction in the academic/industrial community
Copyright © 2015 Criteo
The good news (c’ed)
• A lot of data is available
• Interactions with banners : clicks
• Interactions with products/advertisers : sales, baskets, home views, listings, visit history
→ faster decision-making in AB test, feature engineering of ML models
• New data is coming : mobile, cross-device, (offline)
→ we need to make sense of it
Copyright © 2015 Criteo
Conclusions
• Machine learning applies well to online advertising at scale
• Yet we still need to improve the users’ experience significantly
• The community is pushing new algorithms and new infrastructures forward
• Lots of new data is coming : we need to make sense of it
Copyright © 2015 Criteo
Thanks! Questions?
o.koch@criteo.com

Más contenido relacionado

La actualidad más candente

Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16MLconf
 
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...Digiday
 
Sis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteoSis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteoMediaPost
 
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...Digiday
 
WebSpectator Presentation deck
WebSpectator Presentation deckWebSpectator Presentation deck
WebSpectator Presentation deckrkemmer
 
WS `Publisher Deck
WS `Publisher DeckWS `Publisher Deck
WS `Publisher DeckDavid Facter
 
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of ClickersCriteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of ClickersCriteo
 
CPEx Audience – Adform PAB
CPEx Audience – Adform PABCPEx Audience – Adform PAB
CPEx Audience – Adform PABMatěj Novák
 
Criteo - NOAH13 London
Criteo - NOAH13 LondonCriteo - NOAH13 London
Criteo - NOAH13 LondonNOAH Advisors
 
3 Minute Introduction
3 Minute Introduction3 Minute Introduction
3 Minute IntroductionJulian Tol
 
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation SlidesthinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation SlidesthinkLA
 
Essent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experienceEssent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experienceBBP
 
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation SlidesthinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation SlidesthinkLA
 
IAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 PresentationIAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 PresentationIAB Europe
 
Using Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that MatterUsing Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that MatterIncubeta NMPi
 
The History of Advertising Technology
The History of Advertising TechnologyThe History of Advertising Technology
The History of Advertising TechnologyClearcode
 
Snowplow at Sigfig
Snowplow at SigfigSnowplow at Sigfig
Snowplow at Sigfigyalisassoon
 
Programmatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the FunProgrammatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the FunHanapin Marketing
 
What You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid ManagerWhat You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid ManagerHanapin Marketing
 

La actualidad más candente (20)

Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
 
Criteo Couchbase live 2015
Criteo Couchbase live 2015Criteo Couchbase live 2015
Criteo Couchbase live 2015
 
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
 
Sis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteoSis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteo
 
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
 
WebSpectator Presentation deck
WebSpectator Presentation deckWebSpectator Presentation deck
WebSpectator Presentation deck
 
WS `Publisher Deck
WS `Publisher DeckWS `Publisher Deck
WS `Publisher Deck
 
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of ClickersCriteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
 
CPEx Audience – Adform PAB
CPEx Audience – Adform PABCPEx Audience – Adform PAB
CPEx Audience – Adform PAB
 
Criteo - NOAH13 London
Criteo - NOAH13 LondonCriteo - NOAH13 London
Criteo - NOAH13 London
 
3 Minute Introduction
3 Minute Introduction3 Minute Introduction
3 Minute Introduction
 
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation SlidesthinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
 
Essent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experienceEssent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experience
 
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation SlidesthinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
 
IAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 PresentationIAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 Presentation
 
Using Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that MatterUsing Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that Matter
 
The History of Advertising Technology
The History of Advertising TechnologyThe History of Advertising Technology
The History of Advertising Technology
 
Snowplow at Sigfig
Snowplow at SigfigSnowplow at Sigfig
Snowplow at Sigfig
 
Programmatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the FunProgrammatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the Fun
 
What You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid ManagerWhat You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid Manager
 

Destacado

C# development workflow @ criteo
C# development workflow @ criteoC# development workflow @ criteo
C# development workflow @ criteoIbrahim Abubakari
 
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2Paris Monitoring
 
Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!Changepoint
 
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012Romain Fonnier
 
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...MongoDB
 
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛Google Cloud Platform - Japan
 
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...Amazon Web Services
 
RecsysFR: Criteo presentation
RecsysFR: Criteo presentationRecsysFR: Criteo presentation
RecsysFR: Criteo presentationrecsysfr
 
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔Google Cloud Platform - Japan
 
RecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at CriteoRecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at CriteoRomain Lerallut
 
Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014Olivier Chapelle
 
Taga arthur d little
Taga arthur d littleTaga arthur d little
Taga arthur d littlesouthmos
 
A.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping PointA.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping PointbengillTU
 
Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419Alberto Garcia Romera
 
2016 투자동향보고서
2016 투자동향보고서2016 투자동향보고서
2016 투자동향보고서Platum
 

Destacado (18)

C# development workflow @ criteo
C# development workflow @ criteoC# development workflow @ criteo
C# development workflow @ criteo
 
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
 
Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!
 
Saintjo Two AV4
Saintjo Two AV4Saintjo Two AV4
Saintjo Two AV4
 
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012
 
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
 
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
 
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
 
RecsysFR: Criteo presentation
RecsysFR: Criteo presentationRecsysFR: Criteo presentation
RecsysFR: Criteo presentation
 
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
 
Hadoop summit-ams-2014-04-03
Hadoop summit-ams-2014-04-03Hadoop summit-ams-2014-04-03
Hadoop summit-ams-2014-04-03
 
RecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at CriteoRecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at Criteo
 
Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014
 
Taga arthur d little
Taga arthur d littleTaga arthur d little
Taga arthur d little
 
A.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping PointA.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping Point
 
Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419
 
2016 투자동향보고서
2016 투자동향보고서2016 투자동향보고서
2016 투자동향보고서
 
McKinsey: Understanding shifts in consumer behavior
McKinsey: Understanding shifts in consumer behaviorMcKinsey: Understanding shifts in consumer behavior
McKinsey: Understanding shifts in consumer behavior
 

Similar a New machine learning challenges at Criteo

Recommendation at scale
Recommendation at scaleRecommendation at scale
Recommendation at scalesimondolle
 
Kritter introduction - advertiser
Kritter   introduction - advertiserKritter   introduction - advertiser
Kritter introduction - advertiserKrittercorporate
 
Kritter introduction - technology player
Kritter   introduction - technology playerKritter   introduction - technology player
Kritter introduction - technology playerKrittercorporate
 
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...Dion Hinchcliffe
 
Mumbrella - Programmatic For Marketers
Mumbrella - Programmatic For MarketersMumbrella - Programmatic For Marketers
Mumbrella - Programmatic For MarketersLouder
 
Big Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionBig Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionMurtaza Doctor
 
100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspectiveaccenture
 
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion HinchcliffeVital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe7Summits
 
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...Dion Hinchcliffe
 
Webinar: Retargeting to the Max
Webinar: Retargeting to the MaxWebinar: Retargeting to the Max
Webinar: Retargeting to the MaxKatana Media
 
Selling display display to your smb clients final
Selling display display to your smb clients   finalSelling display display to your smb clients   final
Selling display display to your smb clients finalAcquisio
 
Selling Display to Your SMB Clients
Selling Display to Your SMB ClientsSelling Display to Your SMB Clients
Selling Display to Your SMB ClientsAcquisio
 
UK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed ITUK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed ITAndyHumphreys
 
Migrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloudMigrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloudCloudsoft
 
Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo Dataconomy Media
 
Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016Stanford University
 
Point roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_externalPoint roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_externalMichael Zarcone
 
Data Science Salon: Enabling self-service predictive analytics at Bidtellect
Data Science Salon: Enabling self-service predictive analytics at BidtellectData Science Salon: Enabling self-service predictive analytics at Bidtellect
Data Science Salon: Enabling self-service predictive analytics at BidtellectFormulatedby
 

Similar a New machine learning challenges at Criteo (20)

Recommendation at scale
Recommendation at scaleRecommendation at scale
Recommendation at scale
 
Kritter introduction - advertiser
Kritter   introduction - advertiserKritter   introduction - advertiser
Kritter introduction - advertiser
 
Kritter introduction - technology player
Kritter   introduction - technology playerKritter   introduction - technology player
Kritter introduction - technology player
 
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
 
Mumbrella - Programmatic For Marketers
Mumbrella - Programmatic For MarketersMumbrella - Programmatic For Marketers
Mumbrella - Programmatic For Marketers
 
Big Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionBig Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to Action
 
100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective
 
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion HinchcliffeVital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
 
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
 
Webinar: Retargeting to the Max
Webinar: Retargeting to the MaxWebinar: Retargeting to the Max
Webinar: Retargeting to the Max
 
Selling display display to your smb clients final
Selling display display to your smb clients   finalSelling display display to your smb clients   final
Selling display display to your smb clients final
 
Selling Display to Your SMB Clients
Selling Display to Your SMB ClientsSelling Display to Your SMB Clients
Selling Display to Your SMB Clients
 
UK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed ITUK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed IT
 
Machine Learning for Computational Advertising
Machine Learning for Computational AdvertisingMachine Learning for Computational Advertising
Machine Learning for Computational Advertising
 
Migrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloudMigrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloud
 
Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo
 
Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016
 
Capgemini Insights and Data
Capgemini Insights and Data Capgemini Insights and Data
Capgemini Insights and Data
 
Point roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_externalPoint roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_external
 
Data Science Salon: Enabling self-service predictive analytics at Bidtellect
Data Science Salon: Enabling self-service predictive analytics at BidtellectData Science Salon: Enabling self-service predictive analytics at Bidtellect
Data Science Salon: Enabling self-service predictive analytics at Bidtellect
 

Último

KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 

Último (20)

KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 

New machine learning challenges at Criteo

  • 1. Copyright © 2015 Criteo New machine learning challenges at Criteo Olivier Koch Engineering Program Manager, Criteo Rythm Meetup June 15, 2016
  • 2. Copyright © 2015 Criteo Banners… what else? 2 Advertiser Publisher
  • 3. Copyright © 2015 Criteo Machine learning applications at Criteo • Bidding (2nd price auctions) • Product recommendation • Banner look and feel selection
  • 4. Copyright © 2015 Criteo Machine learning at Criteo • Supervised learning using standard regression methods / optimization algorithms (SGD, L-BFGS) • Distribution on Hadoop (MapReduce, Spark) • 3B displays / day • 40 PB of data -- 15,000 servers • 7 data centers worldwide
  • 5. Copyright © 2015 Criteo Data sparsity 10 000 displays lead to 50 clicks lead to 1 sale
  • 6. Copyright © 2015 Criteo Now what?
  • 7. Copyright © 2015 Criteo Challenges in online advertising • We have an impact on users • A user is seen more than 20 times a day in average • Every bid has an influence on our competitors • We want to provide a better online advertising experience • Personalized • Cross-device • Long tail (new users, new products)
  • 8. Copyright © 2015 Criteo Machine learning challenges • Optimal bidding strategies under uncertainty -- reinforcement learning, policy learning • Probabilistic match of devices • Classification/prediction of time series • Long tail (users, products) -- transfer learning, factorization • Offline metrics – counterfactual analysis
  • 9. Copyright © 2015 Criteo The good news • New generations of algorithms • NLP (word embeddings), reinforcement learning, policy learning, deep networks • Releases of ML infrastructures • Caffe on Spark, TensorFlow, Torch, PhotonML, GPUs inside clusters → strong traction in the academic/industrial community
  • 10. Copyright © 2015 Criteo The good news (c’ed) • A lot of data is available • Interactions with banners : clicks • Interactions with products/advertisers : sales, baskets, home views, listings, visit history → faster decision-making in AB test, feature engineering of ML models • New data is coming : mobile, cross-device, (offline) → we need to make sense of it
  • 11. Copyright © 2015 Criteo Conclusions • Machine learning applies well to online advertising at scale • Yet we still need to improve the users’ experience significantly • The community is pushing new algorithms and new infrastructures forward • Lots of new data is coming : we need to make sense of it
  • 12. Copyright © 2015 Criteo Thanks! Questions? o.koch@criteo.com