Slides from the TensorFlow meetup hosted on October 9th at the ML6 offices in Ghent. Join our Meetup group for updates and future sessions: https://www.meetup.com/TensorFlow-Belgium/
6. Deep Learning for Humans
@fchollet, Keras author
“High-level neural networks API, capable of running on multiple
backends … focus on fast experimentation and being able to go
from idea to result with the least possible delay”
14. Training the model
Out-of-memory data: with generators
Best practice is to subclass the Sequence object, which allows for safe multiprocessing
__getitem__ method yields data in batches
__len__ required to calculate no of batches/epoch
15. Training the model
Out-of-memory data: with generators
See stackoverflow and github for code
Common use case: out-of-memory image data
Read images from disk on the fly
16. Callbacks
Enhancing the training loop
Callback = an object that implements specific logic and is called by the model at various points during training
Keras provides callbacks for early stopping, learning rate scheduling, tensorboard, model saving...
19. Functional API
For complex architectures
Sequential API: linear stacks of layers
Functional API: DAG wrapped in a Keras model
Layers are called on previous layers
27. Full flexibility
Ultimate flexibility: subclass source code
See https://stackoverflow.com/questions/51874695/early-stop-when-validation-loss-satisfies-certain-criteria/51875492#51875492
28. Full flexibility
Ultimate flexibility: subclass source code
See https://stackoverflow.com/questions/51874695/early-stop-when-validation-loss-satisfies-certain-criteria/51875492#51875492
Solution: subclass
Keras’ EarlyStopping
Callback and implement
custom logic
29. Full flexibility
Ultimate flexibility: subclass source code
See https://stackoverflow.com/questions/51874695/early-stop-when-validation-loss-satisfies-certain-criteria/51875492#51875492
Solution: subclass
Keras’ EarlyStopping
Callback and implement
custom logic
callbacks = [MyCallBack(threshold=0.001, min_epochs=10, verbose=1)]
model.fit(features, labels, validation_data=(validation_feat, validation_labels),
callbacks=callbacks, epochs=100)
31. Pretrained models
Transfer learning & model finetuning
model = ResNet50(weights= 'imagenet')
preds = model.predict(x)
model = VGG16(weights= 'imagenet', include_top=False)
features = model.predict(x)
Directly use a model
Extract convolutional
features
base_model = VGG19(weights= 'imagenet')
model = Model(inputs=base_model.input,
outputs=base_model.get_layer('block4_pool').output)
block4_pool_features = model.predict(x)
Extract features at any
level of the model
Functional API: DAG wrapped in a Keras model
32. # create the base pre-trained model
base_model = InceptionV3(weights= 'imagenet', include_top= False)
# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# let's add a fully-connected layer
x = Dense(1024, activation='relu')(x)
# and a logistic layer -- let's say we have 200 classes
predictions = Dense( 200, activation='softmax')(x)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
Pretrained models
Transfer learning & model finetuning
Reuse model base, put
custom classifier on top
33. Tensorflow integration: tf.keras
TensorFlow 2.0 will rely heavily on its Keras integration as thé high level API to use
● comprises full Keras API
● Better integration with TF features:
estimators, dataset API…
● Fully compatible with TF serving: build
& train with Keras, productionize with
TF serving
● current best practice
See https://www.youtube.com/watch?v=dfQ8lZ9dTjs
43. Getting started
● Guides and examples at https://keras.io
● Stackoverflow
● End-to-end example blogpost with tf.Keras:
https://medium.com/tensorflow/training-and-servin
g-ml-models-with-tf-keras-fd975cc0fa27
44.
45.
46. 46 ● Came out of the “Torch” framework, which was used at DeepMind before
TensorFlow came along
● → support for Python instead of Lua led to “PyTorch”
● Shared support from:
Pytorch History
47. 47
The Computation Graph
Deep Learning libraries usually have 2 interpreters:
1. Host language (eg Python)
2. The computation graph (eg C++ backbone)
Can then be optimized and run in parallel on a GPU
55. 55
Dynamic Computation Graph
“In TensorFlow you define graph statically before a model can run.
Communication with the outer world is performed via tf.Session
object and tf.Placeholder which will be substituted by external data at
runtime. When you write in TensorFlow sometimes you feel that your
model is behind a brick wall with several tiny holes to communicate
over.
In PyTorch things are more imperative and dynamic: you can define,
change and execute nodes as you go, no special session interfaces
or placeholders. Overall, the framework is more tightly integrated
with Python language and feels more native most of the times.“
69. 69
Saving and restoring models
Saving and Loading models is waaaay less of a hassle compared to TF
● No meta_graph, index, graph_def, session, ...
● Load a model in a single line:
70. 70
Saving and restoring models
Saving and Loading models is waaaay less of a hassle compared to TF
● No meta_graph, index, graph_def, session, ...
● Load a model in a single line:
75. 75
Varia
● PyTorch easily integrates with TensorBoard!
TensorBoard is just great…
● Since computation graphs in PyTorch are defined at
runtime you can use your favorite Python debugging tools
such as pdb, ipdb, PyCharm debugger or old trusty print
statements :)
78. 78
One of PyTorch’s biggest strengths is its first-class Python integration, imperative style, simplicity of
the API and options. These are aspects that make PyTorch good for research and hackability.
One of its biggest downsides has been production-support. What we mean by production-support is
the countless things one has to do to models to run them efficiently at massive scale:
● exporting to C++-only runtimes for use in larger projects
● optimizing mobile systems on iPhone, Android, Qualcomm and other systems
● using more efficient data layouts and performing kernel fusion to do faster inference
(saving 10% of speed or memory at scale is a big win)
● quantized inference (such as 8-bit inference)
1.0
80. 80
“OMG I just read this cool new paper and they
have an implementation in …”
“Great! I’ll be able to run this code
easily in a distributed training setup
and serve it to thousands of
customers through an API with
tf.serving”
“Great! I’ll be able to dive into the
code, figure out how it works and
easily tweak the entire codebase into
something I can use!”
83. Any application that can be written in JavaScript
will eventually be written in JavaScript.
Jeff Atwood (co-founder StackOverflow.com)
11 years ago
89. Why TensorFlow.js?
No drivers or installs required
It just works! No ‘install’ difference with server-side ML.
GPU acceleration is available through WebGL
Highly interactive playground.tensorflow.com
90. Why TensorFlow.js?
Perfect for transfer learning
Hard to train entire models, but tuning is very feasible.
MobileNetPoseNet
92. Why TensorFlow.js?
Easily generalizable to edge devices
No major adjustments required to provide functionality.
Direct access to sensor data
New applications involving camera, microphone and accelerometer.
Accelerometer
Camera
Microphone
Location
93. Why TensorFlow.js?
Eliminating server-side processing
Model has to be downloaded once, but is cached afterwards.
Feedback is still possible, e.g. by exchanging
model weights (“memory”).
94. Why TensorFlow.js?
Eliminating server-side processing
Eliminate data flow. No input data needs to be sent back and forth
Low latency, near instant results.
Huge savings in server costs.
95. Why TensorFlow.js?
Enables the usage of private data
Enables tuning the model with private data.
The resulting weights can still be stored.
No concerns as the data is not exchanged.
Models can always be used in offline mode.
No need for central data storage.
Performance Data Medical Data Conversation Data Financial Data Contact Data Media
96. Why TensorFlow.js?
Future applications
Possibilities for Browser Extensions & Plug-ins
Encryption
Website accessibility
Text autocompletion
Applications for edge devices
ML frameworks for Web Developers (with low latency)
Recommender systems
Website-specific context generation
Voice controlled search engines or assistants
97. Why TensorFlow.js?
Easy model conversion:
Easy model importing:
source: Nikhil Thorat and Daniel Smilkov @ TensorFlow Dev Summit 2018
98. Why TensorFlow.js?
Good
Great
Summary
A tiny bit of JavaScript versus
Low latency predictions
Privacy guarantees
Avoiding data flow and
elimination of server costs
New applications