1.
Technology Solutions Professional, Data & AI @ Microsoft
source:xkcd.com

2.
The Problem

3.
Oil Traps

4.
Salt Dome Traps

5.
github.com/neaorin/kaggle-tgs-challenge

6.
The Data
TRAIN: 4000 images and masks
101x101, grayscale
TEST: 18000 images
101x101, grayscale

7.
The Data
VALIDATION: X images and masks
101x101, grayscale
TRAIN: 4000 - X images and masks
101x101, grayscale

8.
The Metric: IoU (Intersection over Union)

9.
The Tools

10.
Hardware
GeForce GTX 1080 Ti
332 GFLOPS
$769
Tesla K80
1,864 GFLOPS
$1,798
Tesla P100
4,036 GFLOPS
$6,598

11.
Going Cloud
NC6
Tesla K80
1,864 GFLOPS
$0.9 / hour
NC6 v2
Tesla P100
4,036 GFLOPS
$2.07 / hour
N-Series VMs

12.
https://azure.microsoft.com/en-us/free

13.
https://azure.microsoft.com/en-us/free/students

14.
Software
• Python
• Numpy
• Pandas
• Scikit-learn
• Scikit-image
• …

15.
Azure Data Science Virtual Machines
https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines/

16.
Azure DSVM for Linux (Ubuntu)
https://azuremarketplace.microsoft.com/en-gb/marketplace/apps/microsoft-ads.linux-data-science-vm-
ubuntu

17.
The Approach

18.
What Can Machine Vision Do Today?

19.
Input Output

20.
OutputInput
w11
w21
w31
Weights
Activation
Outputs

21.
3
Error back propagation
Feedforward of information
Loss Function (L)
≠2
Optimizer
Gradient of the loss with respect
to each weight:
𝜕L
𝜕w
= ?
w11
w21
w31
Gradient Descent
𝜕L
𝜕w

22.
https://medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural-networks-260c2de0a050

23.
https://medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural-networks-260c2de0a050

24.
https://medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural-networks-260c2de0a050

25.
U-Net
U-Net: Convolutional Networks for Biomedical Image Segmentation https://arxiv.org/abs/1505.04597

26.
Deconvolutions

27.
inputs = Input((im_height, im_width, im_chan))
s = Lambda(lambda x: x / 255) (inputs)
c1 = Conv2D(8, (3, 3), activation='relu', padding='same') (s)
c1 = Conv2D(8, (3, 3), activation='relu', padding='same') (c1)
p1 = MaxPooling2D((2, 2)) (c1)
c2 = Conv2D(16, (3, 3), activation='relu', padding='same') (p1)
c2 = Conv2D(16, (3, 3), activation='relu', padding='same') (c2)
p2 = MaxPooling2D((2, 2)) (c2)
...
c5 = Conv2D(128, (3, 3), activation='relu', padding='same') (p4)
c5 = Conv2D(128, (3, 3), activation='relu', padding='same') (c5)
...
u8 = Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same') (c7)
u8 = concatenate([u8, c2])
c8 = Conv2D(16, (3, 3), activation='relu', padding='same') (u8)
c8 = Conv2D(16, (3, 3), activation='relu', padding='same') (c8)
u9 = Conv2DTranspose(8, (2, 2), strides=(2, 2), padding='same') (c8)
u9 = concatenate([u9, c1], axis=3)
c9 = Conv2D(8, (3, 3), activation='relu', padding='same') (u9)
c9 = Conv2D(8, (3, 3), activation='relu', padding='same') (c9)
outputs = Conv2D(1, (1, 1), activation='sigmoid') (c9)
model = Model(inputs=[inputs], outputs=[outputs])
model.compile(optimizer='sgd', loss='binary_crossentropy', metrics=[iou])

28.
U-Net
U-Net: Convolutional Networks for Biomedical Image Segmentation https://arxiv.org/abs/1505.04597

29.
Overfitting

30.
Overfitting

31.
Overfitting

32.
Get More Data!
DATA

33.
Image Augmentation
Original
Flip
Shear
Rotate

34.
Image Augmentation with Keras
image_datagen = ImageDataGenerator(
horizontal_flip=True,
rotation_range=20,
shear_range=0.3,
zoom_range=0.25,
fill_mode='reflect')

35.
Image Augmentation with imgaug
image_datagen = iaa.Sequential([
iaa.Fliplr(0.5),
iaa.GaussianBlur((0, 3.0)),
iaa.Dropout(0.02),
iaa.AdditiveGaussianNoise(scale=0.01*255),
iaa.AdditiveGaussianNoise(loc=32,
scale=0.0001*255),
iaa.Affine(translate_px={"x": (-40, 40)})])

36.
Test-Time Augmentation

37.
Overfitting - Dropout Layers

38.
Cross-Validation

39.
www.image-net.org

40.
Pre-Trained Architectures on ImageNet Data

41.
Transfer Learning

42.
Loss Function
Binary Cross-Entropy (BCE)

43.
Loss Function
Lovász Loss
The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in
neural networks https://arxiv.org/abs/1705.08790
https://github.com/bermanmaxim/LovaszSoftmax

44.
http://wiki.fast.ai/index.php/Lesson_1_Notes
Gradient Descent

45.
Optimizers
http://cs231n.github.io/neural-networks-3/

46.
http://ruder.io/optimizing-gradient-descent/index.html

47.
Learning Rate

48.
Learning Rate Annealing

49.
Learning Rate – Reduce on Plateau
reduce_lr = ReduceLROnPlateau(
monitor='val_iou’,
patience=20,
factor=0.5,
min_lr=0.0001)
history = model1.fit(x_train, y_train,
callbacks=[reduce_lr],
...)

50.
Learning Rate – Cosine Annealing

51.
Visualization

52.
Visualization
tb = TensorBoard
(log_dir=f'tb_logs/{model_name}’,
batch_size=batch_size)

53.
Postprocessing: A Jigsaw Puzzle
https://www.kaggle.com/vicensgaitan/salt-jigsaw-puzzle/notebook

54.
Other Things to Try Out
Deep Supervision https://arxiv.org/abs/1505.02496
Snapshot Ensembles https://arxiv.org/abs/1704.00109
Hypercolumns https://arxiv.org/abs/1411.5752
'Squeeze & Excitation' Blocks https://arxiv.org/abs/1808.08127
Conditional Random Fields https://arxiv.org/abs/1502.03240
Convolutional Block Attention Module https://arxiv.org/abs/1807.06521
Stochastic Weight Averaging https://arxiv.org/abs/1803.05407

55.
From Testing
to Production

56.
run = experiment.submit(config=TensorFlow(
entry_script=entry_script,
script_params=script_params,
compute_target=compute_target,
use_gpu=True,
use_docker=True,
pip_requirements_file_path='requirements.txt'))

57.
https://azure.microsoft.com/en-us/services/machine-learning-service/

58.
Thank You