Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Data Science on Google Cloud Platform
1. Ta Virot Chiraphadhanakul, PhD (@tvirot)
GDE in Machine Learning | Managing Director @ Skooldio
Data Science on Google Cloud Platform
Google Developers
Launchpad Build for Cloud Meetup, Bangkok
2. 90% of the data in the world
today has been created in
the last two years alone.
— IBM
8. How about a serverless big
data stack that scales
automatically?
9. Serverless Data Processing
• Focus on insights, not administration
• Practically infinite scale, exactly when you need it
• Pay only for what you use
• Freedom to experiment, fail quickly, and iterate. Successful experiments are
ready to go live right away
10. Storage & Databases Big Data Machine Learning
Data Science on Google Cloud Platform
12. Storage & Databases
Cloud Storage
A scalable object storage service
suitable for all kinds of
unstructured data
Cloud SQL
A fully-managed database service
that makes it easy to set up,
maintain, manage, and administer
your relational MySQL and
PostgreSQL databases in the cloud
Cloud Datastore
A highly-scalable NoSQL database
for your applications. Cloud
Datastore automatically handles
sharding and replication.
Cloud BigTable
A massively scalable NoSQL
database suitable for low-latency
and high-throughput workloads. It
supports the open-source, industry-
standard HBase API
13. Fully-managed real-time messaging service
that allows you to send and receive
messages between independent applications
Connect Anything to Everything
Use Cloud Pub/Sub to publish and
subscribe to data from multiple sources,
reducing dependencies between
components of distributed applications
Highly Scalable
Any customer can send up to 10,000
messages per second, by default
Guaranteed Delivery
Designed to provide “at least once” delivery
Cloud Pub/Sub
15. Fully-managed data processing service,
supporting both stream and batch execution
of pipelines
Fully Managed
Dynamically provision resources to
minimize latency while maintaining high
utilization efficiency
Unified Programming Model
Express computational requirements
regardless of data source
Cloud Dataflow
17. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
Create a Pipeline
https://cloud.google.com/dataflow/examples/wordcount-example
18. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
Read lines
Create a Pipeline
https://cloud.google.com/dataflow/examples/wordcount-example
19. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
Read lines
Create a Pipeline
Tokenize lines into words
https://cloud.google.com/dataflow/examples/wordcount-example
20. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
.apply(MapElements.via(new SimpleFunction<KV<String, Long>, String>() {
@Override
public String apply(KV<String, Long> element) {
return element.getKey() + ": " + element.getValue();
}
}))
Read lines
Create a Pipeline
Tokenize lines into words
Count words
https://cloud.google.com/dataflow/examples/wordcount-example
21. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
.apply(MapElements.via(new SimpleFunction<KV<String, Long>, String>() {
@Override
public String apply(KV<String, Long> element) {
return element.getKey() + ": " + element.getValue();
}
}))
.apply(TextIO.Write.to("gs://my-bucket/counts.txt"));
Format strings
Read lines
Create a Pipeline
Tokenize lines into words
Count words
https://cloud.google.com/dataflow/examples/wordcount-example
22. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
.apply(MapElements.via(new SimpleFunction<KV<String, Long>, String>() {
@Override
public String apply(KV<String, Long> element) {
return element.getKey() + ": " + element.getValue();
}
}))
.apply(TextIO.Write.to("gs://my-bucket/counts.txt"));
Format strings
Read lines
Create a Pipeline
Tokenize lines into words
Write to file
Count words
https://cloud.google.com/dataflow/examples/wordcount-example
25. Managed Spark and Hadoop service which is
fast, easy to use, and low cost
Fast & Scalable Data Processing
Create a cluster in minutes and resize them
at any time
Affordable Pricing
Based on actual use, measured by the
minute
Open Source Ecosystem
Move existing projects or ETL pipelines
without redevelopment
Cloud Dataproc
28. An intelligent data service for visually
exploring, cleaning, and preparing data
Visually explore data
Intelligent data manipulation
Serverless and works at any scale
Cloud Dataprep
30. Google's fully managed, petabyte scale, low
cost enterprise data warehouse for analytics
Fully Managed
No infrastructure to manage, and you don't
need a database administrator
Speed & Scale
Scans TB in seconds and PB in minutes
Convenience of SQL
Makes it more accessible
Security & Reliability
Automatically encrypts and replicates data
BigQuery
31. Google's fully managed, petabyte scale, low
cost enterprise data warehouse for analytics
Flexible Data Ingestion
Load your data from Google Cloud Storage
or Google Cloud Datastore, or stream it
Fully Integrated
With other Google Cloud products and
third-party applications
BigQuery
35. An easy to use interactive tool for data
exploration, analysis, visualization and
machine learning
Integrated & Open Source
Built on Jupyter (formerly IPython).
Enables analysis of your data on BigQuery,
ML Engine, Compute Engine, and Cloud
Storage
Cloud Datalab
37. Turns your data into informative dashboards
and reports that are easy to read, easy to
share, and fully customizable
Put all your data to work
Easily access all the data sources you
need to understand your business and
make better decisions
Build engaging visualizations
Create beautiful charts and graphs that
bring your data to life
Leverage teamwork that works
Share and collaborate in real time. Work
together quickly, from anywhere.
Cloud Data Studio
43. Waymo
The Google self-driving car
project became Waymo with a
mission to make it easy and
safe for people and things to
move around
Photo: Waymo
44. Machine Learning engine and APIs
Custom ML modelsPre-trained ML models
Machine Learning
Engine
TensorFlowVision API
Translation
API
Natural Language
API
Speech API Jobs API
45. Google Cloud
Vision API
Understand the content of
images
• Label Detection
• Optical Character Recognition
• Explicit Content Detection
• etc.
+
https://m.me/youpin.city | https://youpin.city/app
@tvirot
47. A managed service that enables you to easily
build machine learning models, that work on
any type of data, of any size
Scalable Service
Managed distributed training infrastructure
that supports CPUs and GPUs
HyperTune
Automatically tuning your hyper
parameters with HyperTune
Deep Learning Capabilities
Supports any TensorFlow models
Cloud ML Engine
54. Cucumber Sorter
"Farmers want to focus and
spend their time on growing
delicious vegetables.”
— Makoto Koike
Photos: Google Cloud Platform / Kaz Sato
57. Serverless
Less ops and administration
No waiting
Queries that used to take hours or days
now take minutes or seconds
Machine Intelligence
Gives everyone access to the deep learning
systems
58.
59. Thank you!
Ta Virot Chiraphadhanakul, PhD (@tvirot)
GDE in Machine Learning | Managing Director @ Skooldio