Accelerating data driven culture at the largest media group of Latin America with Jupyter
•
3 recomendaciones•1,585 vistas
Presentation about Jupyter at globo.com held at #JupyterCon 2017 in New York!. Authors: Diogo Munaro and Felipe Ferreira
Recomendados
Recomendados
Más contenido relacionado
Similar a Accelerating data driven culture at the largest media group of Latin America with Jupyter
Similar a Accelerating data driven culture at the largest media group of Latin America with Jupyter (20)
Último
Último (20)
Accelerating data driven culture at the largest media group of Latin America with Jupyter
- 2. Diogo Munaro Felipe Ferreira
- 4. W E W E R E BORN DIGITAL TO TRANSFORM BRAZIL’S INTERNET by crafting digital products, services and operations for the largest media group in Latin America
- 5. O U R M A I N O F F I C E I S I N Rio de JaneiroOther offices: São Paulo and Porto Alegre
- 6. W E H A V E M O R E T H A N 87 MILLION USERS C O M S C O R E M M X - M AY / 1 7
- 7. W E R E A C H 78% OF BRAZIL’S INTERNET C O M S C O R E M M X - M AY / 1 7
- 8. W E A R E LEADERS IN VIDEOS SPORTS NEWS ENTERTAINMENT
- 9. 17 YEARSO F R E M A R K A B L E A C H I E V E M E N T S
- 11. LARGER THAN THE RUNNER UP 2× C O M S C O R E M M X - M AY / 1 7 & I N T E R N A L D ATA | G L O B O . C O M 1.000 54Mi UNIQUE VISITORS NEW ARTICLES PER DAY LAUNCHED LEADER Designed and developed from scratch G1 becomes leader in Brazil 2006 2008 2017
- 12. 27 STATES 57 NEWSROOMS 5490 CITIES
- 13. THE LEADING SPORTS PORTAL IN BRAZIL S I N C E 2 0 0 5 sportsnews livecoverage scouts statistics
- 14. THE LEADING SPORTS PORTAL IN BRAZIL S I N C E 2 0 0 5 sportsnews livecoverage scouts statistics 61% OF SPORTS CATEGORY C O M S C O R E M M X - M AY / 1 7
- 15. freemiummodel hugeuserbase highengagement SINCE 2005 THE BIGGEST FANTASY GAME IN BRAZIL 320.000SUBSCRIBERS +11MiTEAMS I N T E R N A L D ATA | G L O B O . C O M
- 16. ALL ABOUT ENTERTAINMENT tvshows,behindthescenes, fashion,interiordesign,recipes, music,BigBrotherBrasil… S I N C E 2 0 1 4
- 17. B I G B R O T H E R B R A Z I L I S A N INTERNET PHENOMENON
- 18. B I G B R O T H E R B R A Z I L I S A N INTERNET PHENOMENON NUMBER OF VOTES IN A SINGLE POLL 149Mi PEAK VOTES PER MINUTE 1Mi I N T E R N A L D ATA | G L O B O . C O M
- 19. page visited watching time video sharing comment time reading news scroll percentage Horizon Gateway HORIZON - OUR EVENT TRACKER
- 20. O U R P R O D U C T A P P R O A C H I S DATA DRIVEN DEVELOPMENTWe make our decisions based on what really works for real people
- 21. HYPOTESIS EXPERIMENT ANALYSIS LEARN Using Jupyter with our big data resources and looking for KPIs, we can evaluate our design experience — like a scientist A SCIENTIFIC BIAS FAST QUALITATIVE CYCLES
- 22. JUPYTER is playing an important role in the data-driven decisions Jupyter notebook is the heart of our data analysis process In the jupyter notebooks are performed AB test analysis, machine learning experiments, peformance evaluation of the recommendation systems and etc..
- 23. WHY DID WE CHOOSE JUPYTER ?
- 24. • Scalability for Hundreds of Data Scientists BECAUSE OF GLOBO.COM NEEDS: • Analysis of large, complex data sets • Works with R and Python • Knowledge share and reproducible studies • Security: data access from different sources
- 25. HOW TO MET THE NEEDS?
- 27. LARGE AMOUNT OF DATA +3billiondailyevents +2millionconcurrentconnections +50 million uniqueuserspermonth +100thousandnewcontents permonth
- 29. - Python2.7 - Python3.6 - R Different kernels
- 30. With this lib we accelerate data science process on jupyter WE CREATED OUR INTERNAL UTILITY LIB CALLED GLOBO DS
- 31. SPARK MANAGEMENT + DATA SCIENCE UTILITIES G L O B O D S F U N C T I O N S
- 32. - Start(localorcluster) - Stop - Monitoring TO SPARK U T I L I T Y F U N C T I O N S
- 33. - Removeoutliers - Plotgraphs - Dateformat - Devicetype - Saveparquetand csvinhdfs TO DATA SCIENCE PROCESS U T I L I T Y F U N C T I O N S
- 38. WE HAVE SECURITY HOOKS Load Run Cell Delete
- 39. ALL WITH OAUTH2 1. Access JupyterHub 2. Can Access ? 3. Authorized User Backstage Accounts 4. Request User Data 5. User Data
- 41. Diogo Munaro Felipe Ferreira www.linkedin.com/in/feliferr felipe.felipeam@gmail.com www.linkedin.com/in/dmvieira diogo.mvieira@gmail.com