As one of Big Data’s Founding Fathers, Google explored the technological changes we faced over the past 10 years and present their solutions to the new data challenges within the Google Cloud ecosystem

1. Nir Chinsky
Head of Google Cloud Platform, MEA

2. Google Cloud Platform 2
1 Billion Users

4. Data &
Analytics
Infrastructure
& Operations
Application
Development

5. Confidential & ProprietaryGoogle Cloud Platform 5
Everything You Need To Build And Scale
Compute
From virtual machines with
proven price/performance
advantages to a fully managed
app development platform.
Compute Engine
App Engine
Container Engine
Container Registry
Cloud Functions
Storage and Databases
Scalable, resilient, high
performance object storage and
databases for your applications.
Cloud Storage
Cloud Bigtable
Cloud Datastore
Cloud SQL
Networking
State-of-the-art software-defined
networking products on Google’s
private fiber network.
Cloud Virtual Network
Cloud Load Balancing
Cloud CDN
Cloud Interconnect
Cloud DNS
Management Tools
Monitoring, logging, and diagnostics
and more, all a easy to use web
management console or mobile app.
Stackdriver Overview
Monitoring
Logging
Error Reporting
Debugger
Deployment Manager & More
Big Data
Fully managed data warehousing,
batch and stream processing, data
exploration, Hadoop/Spark, and
reliable messaging.
BigQuery
Cloud Dataflow
Cloud Dataproc
Cloud Datalab
Cloud Pub/Sub
Genomics
Machine Learning
Fast, scalable, easy to use ML
services. Use our pre-trained models
or train custom models on your data.
Cloud Machine Learning Platform
Vision API
Speech API
Translate API
Developer Tools
Develop and deploy your applications
using our command-line interface and
other developer tools.
Cloud SDK
Deployment Manager
Cloud Source Repositories
Cloud Endpoints
Cloud Tools for Android Studio
Cloud Tools for IntelliJ
Google Plugin for Eclipse
Cloud Test Lab
Identity & Security
Control access and visibility to
resources running on a platform
protected by Google’s security model.
Cloud IAM
Cloud Resource Manager
Cloud Security Scanner
Cloud Platform Security Overview

7. Hitting the limits, early on...
The Anatomy of a Large-Scale Hypertextual Web Search Engine
1996, Sergey Brin and Lawrence Page
Computer Science Department, Stanford University, Stanford,
CA 94305

8. Building on Google’s infrastructure
2.8 million
devices activated
Every Day (1.1 billion devices)
10 billion
hours watched
Every Month (100 hours of
new content uploaded
every minute)
43 billion
pages crawled
Every Day

10. Challenges
11010101
0100101
1011101
0111100
10001101
Legacy
Applications
Regulatory
Environment
Changing View
on Value of Data
No Cloud Skills,
Hard to Recruit
Data Silos
Everywhere

12. GOOGLE CONFIDENTIAL
We want to change what you can do.
Not just where you can do it.

13. Select
IaaS
OS
Setup
Containerization
Storage
Container
Orchestration
Networking
Monitoring
Logging
Testing
Dev
User

14. Prototype Production Planet-scale

16. Original Launch Target Estimated Worst Case Actual Traffic
Cloud Datastore Transactions Per Second
50X
Actual Traffic
5X
Worst Case
Estimate
1X
Target Traffic

17. Google Cloud Platform 17
The Journey to a Web-Scale Cloud
2nd Wave1st Wave 3rd Wave

18. Open Matters

19. Walking the Walk

20. Leading Open Source Communities
#1Highest
Engagement
on Github
#2
Highest
Engagement
on Github
Kubernetes Tensorflow
Source: Analyzing GitHub issues and comments with BigQuery

21. Kubernetes
Apps

24. 600+
Security
Engineers
Security and Compliance
700+
CVEs Discovered
and/or Fixed
160
Academic Research
Papers on Security

25. Google’s Data Research
2002 2004 2006 2008 2010 2012 2014 2016
GFS
MapReduce TensorFlow
BigTable
Dremel
Colossus
Flume
Megastore
Spanner
Millwheel
PubSub
F1

26. Google’s Data Products
2002 2004 2006 2008 2010 2012 2014 2016
ML
PubSub
DataFlow
DataStore
DataFlow
Cloud Storage
BigQuery
BigTable
DataProc
Cloud Storage

27. Data &
Analytics
Insight Data

28. Programming
Resource
provisioning
Performance
tuning
Monitoring
Reliability
Deployment &
configuration
Handling
growing scale
Utilization
improvements
Typical Big
Data Jobs

29. Big Data
with Google
Focus on insights.
Not infrastructure.
Programming
Understanding

30. Dataproc: Fully managed Hadoop and
Spark w/ industry-leading performance
BigQuery: Fully managed data
warehouse for large-scale analytics
Dataflow: Real-time data pipelines, with
open source SDK via Apache Beam
Big Data with Google
30

31. 31
No DBA
NBD(no big deal)

32. “Right at the start of the partnership we were able to
reduce time to insight from 96 hours to 30 minutes by using BigQuery”
Gary Sanders
Head of Digital Analytics

33. Music for Everyone
75M+ Users
2B+ Playlists
30M+ Songs
Data is the center
of the Spotify
music experience
With GCP, data
teams get big data
insights in minutes
versus hours

34. “From traditional batch processing to rock-solid event delivery to the nearly
magical abilities of BigQuery, building on Google’s data infrastructure
provides us with a significant advantage where it matters the most.”
Nicholas Harteau
VP of Engineering and Infrastructure

36. Analytics to Machine Learning

37. Cloud
Vision API
Cloud
Translate API
Cloud
Natural Language
API
Cloud
Speech API
Ready to Use
Machine Learning Models
Cloud Machine
Learning
DEVELOP - MODEL - TEST
Use Your Own Data
To Train Models

38. Hypertune
Cloud Machine Learning
NoOps Machine
Learning
Training
at Scale
Cloud
Datalab UI

39. Rapidly Accelerating Use of Deep Learning at Google
Number of directories containing model description files
2012 2013 2014 2015
1500
1000
500
0
Used across products:

40. 40
Google Photos

41. Auto-organizing photo library
We just released this feature
worldwide couple weeks ago.

42. Navigate your photos by place

43. PhotoNet by
Google DeepMind

44. Confidential & ProprietaryGoogle Cloud Platform 44
Cloud Vision API
Insight from images with our powerful
Cloud Vision API

45. Faces
Faces, facial landmarks, emotions
OCR
Read and extract text, with
support for > 80 languages
Cloud Vision API
Call API from anywhere, with support for embeddable images, and Google Cloud storage
Label
Detect entities from furniture to
transportation
Logos
Identify product logos
Landmarks & Image Properties
Detect landmarks & dominant
color of image
Safe Search
Detect explicit content - adult,
violent, medical and spoof

46. API Usage: Detect Objects in an Image
Image Detected
Items
Vision API
Upload Images
Cloud/On premise
Process
the response
Call the
REST API1 2 3

48. 48
Cloud Vision API

49. Digital Transformation = APIs