Cloud Spin - building a photo booth with the Google Cloud Platform

Cloud Spin
Bret McGowen
Google
@bretmcg
Learning about the cloud by dodging bullets, Matrix-style

Bret McGowen
Developer Advocate, Google
Google Cloud Platform
New York, NY
@bretmcg
Howdy DevFest MN!

3 weeks to minimum
viable product

@bretmcg @googlecloud @googlecloudspin

The user steps into the
center of the Cloud Spin
area

The user jumps, and we
take a photo from all the
phones

● Taking a photo at the same EXACT SAME INSTANT
● Coordinating the phone cameras
● File storage
● Scaling
● Image processing
● Controlling cost
● Create multiple versions
● Debugging & Logging
Problems to solve

Taking the photos:
how would you do it?

It isn’t that straightforward
● Need to coordinate 19 phones to take photos
of the same moment!
● Android - random delay when taking a photo!
Hence impossible to take photos of the same moment

What we ended up with?
● Record a video
● 30 fps is like burst mode

More problems
● How do we identify the frames?
● Mark the the frame some how...
● Many Experiments: flash, timestamp

Mic-In Into the Phone
MIC
OMTP
MIC
GND
AHJ / CTIA

Audio can be tricky

Finally - Off the shelf components

Record the audio cue in the video

How do we coordinate 19 phone
cameras?

No, seriously, how do we
coordinate 19 phone cameras?

What do we need to coordinate?
● Camera app status & error message
● Exposure ISO & shutter speed
● Internal versus external microphone
● Timestamp to start the recording
● And more...

Let's use Firebase!
● Can power our app's backend
● Stores our data and syncs it in real-time
● JSON NoSQL database with real-time sync
● Updates across devices within milliseconds

// Set this phone's status on Firebase at /cameras/{cameranumber}
Firebase firebaseRef = new Firebase(FIREBASE_BASE_URL);
Firebase cameraStatusRef = firebaseRef.child("cameras").child
(cameraNumber);
CameraStatus cameraStatus = new CameraStatus();
cameraStatus.appVersion = BuildConfig.VERSION_NAME;
cameraStatus.externalMic = mExternalMic;
...
cameraStatus.status = status;
cameraStatusRef.setValue(mCameraStatus);
Android: updating camera status with Firebase

Recording video with camera2 API
● camera2 API
● I learned a lot of this API from the android-
Camera2Video GitHub project

Scaling with Google Cloud Platform

Stitching Q
Extraction Q
Input
Bucket
Stitching Q
Devices
Extraction Q
Notification
Processor
Extractor
Stitcher
Video
Uploads
Object Change
Notifications
Output
Bucket
Coordinator
Input
Bucket
Notification
Processor
Extractor
Stitcher
Output
Bucket

Video Uploads and notification processing
Input
Bucket
Stitching Q
Devices
Extraction Q
Notification
Processor
Extractor
Stitcher
Video
Uploads
Notification
Output
Bucket
Coordinator
Input
Bucket
Notification
Processor

Cloud Datastore Cloud SQL Cloud Storage

4.5 trillion! Managed MySQL Cat Videos?Cat Videos?
Jumping
People

// Credentials for Google Cloud Platform.
Credential credential = ...
Storage storage = new Storage.Builder(httpTransport, jacksonFactory, credential)
.setApplicationName("...@developer.gserviceaccount.com")
.build();
StorageObject storageObject = new StorageObject();
storageObject.setBucket(bucket);
InputStream fileUploadStream = new FileInputStream(videoFile);
try {
InputStreamContent content = new InputStreamContent("video/mp4", fileUploadStream);
Storage.Objects.Insert insert = storage.objects().insert(bucket, null, content);
insert.setName(cloudStoragePath);
insert.execute();
} finally {
fileUploadStream.close();
}

Object Change Notifications
Cloud
Storage Bucket
[myapp-input-bucket]
App Engine
Object Change Notification
Client
Add/update/remove object

Edge Caching

Google App Engine
Choice of Runtimes
- Java, Python, Go
- PHP
- Custom, NodeJS!
Easy to develop
- Build and test locally
- Focus on App Code
- Versioning
- Traffic Splitting
Trivial to manage
- Fully managed
- No patches/updates
- 24x7 operation by
Google SREs
- Autoscale

Building Pipelines
Input
Bucket
Stitching Q
Devices
Extraction Q
Notification
Processor
Extractor
Stitcher
Video
Uploads
Object Change
Notifications
Output
Bucket
Coordinator

Subscriber Subscriber
Pub/Sub Pub/Sub
Push Subscription Pull Subscription
msg ack msg ack
RPC Send
RPC Return

Cloud Pub/Sub (Beta)
Reliable and
real-time
messaging
Designed for
Fast Data
Provides reliable, real-time,
many-to-many, asynchronous
messaging between
applications
Send data into processing
pipelines and out to other
apps, devices and Google Cloud
Services
Global by design and highly
available. ‘fire-and-forget’
with minimal latency.
Redundant forwarding paths
Designed for
Google scale

Pub/Sub
Stitcher
Frames
Topic
Extractor
Frame Ready Message
Publishers and Subscribers
Original Stitcher
Subscription

Pub/Sub
Original
Stitcher
Frames
Topic
Extractor
Frame Ready Message
Publishers and Subscribers
Original Stitcher
Subscription
Variation A
Stitcher
Variation A
Subscription
Variation B
Stitcher
Variation B
Subscription

Variations - How Pub/Sub Can Help

Meanwhile, in our secret startup lab

Our first prototype was cheap and
scrappy

We used anything we
could find.

Like, REAL
scrappy(selfie sticks!)

Pipeline Processing
Input
Bucket
Stitching Q
Devices
Extraction Q
Notification
Processor
Extractor
Stitcher
Video
Uploads
Object Change
Notifications
Output
Bucket
Coordinator
Extractor
Stitcher

from moviepy.video.io.VideoFileClip import VideoFileClip
import numpy
STD_DEV_SCALAR = 4
def calc_peak_time(clip):
audio_arr = clip.audio.to_soundarray(buffersize=1000)
frames = clip.fps * len(audio_arr) / clip.audio.fps
buckets = numpy.array_split(audio_arr, frames)
mean_arr = [numpy.sqrt(numpy.mean(bucket**2)) for bucket in buckets]
peaks = numpy.where(mean_arr > MEAN_SCALAR * numpy.mean(mean_arr))
if len(peaks) > 0 and len(peaks[0]) > 0:
return peaks[0][0] / float(clip.fps)
else:
return 0.0

Frames extracted

Google Compute Engine
Compute Resources
- US, Europe and
Asia zones
- Fast SDN-Based
virtual networking
Consistently Fast
- Fast VM Provisioning
- Consistent Performance
Cost Effective
- Sub-Hour Billing
- No IOPS charges for
Block Storage
- VMs not required for
Load balancing

Compute Engine Autoscaler
Managed
Instance
Group
create/
destroy VMs
Instance Group
Manager
actuator
monitor
Autoscaler
Cloud Monitoring
utilisation
VMVMVM
Pub/Sub Topic

Next step: Containerization
Extractor
Workers
create/
destroy VMs
Instance Group
Manager
actuator
monitor
Autoscaler
Cloud Monitoring
utilisation
VMVM
Container
VM
Extract Topic

Stitching the frames together
Extract Q
Temp
Store
Extractor
Stitch QExtractorExtractors
READY
Stitcher
Firebase
All READY?
Retrieve Images
Visualizer

● Taking a photo at the same EXACT SAME INSTANT - video with audio beep
● Coordinating the phone cameras - Firebase
● File storage - Google Cloud Storage
● Scaling - Cloud Pub/Sub, Compute Engine Instance Groups
● Image processing - Compute Engine & Containers
● Controlling cost - Scaling instance group sizes
● Create multiple versions - Cloud Pub/Sub topics
● Debugging & Logging - Google Cloud Logging
Problems to solve

Productivity!
We built a lot in just 3 weeks
Focus 100% of time coding

• Talk with us!
• Firebase
• Containers
• Kubernetes
• Big data
• BigQuery
Want to know more?

Bret McGowen
@bretmcg
@googlecloudspinTHANK YOU

Cloud Spin - building a photo booth with the Google Cloud Platform

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Cloud Spin - building a photo booth with the Google Cloud Platform

Similar a Cloud Spin - building a photo booth with the Google Cloud Platform (20)

Último

Último (20)

Cloud Spin - building a photo booth with the Google Cloud Platform