IA à portée de la main !

1. Meetup "IA à portée de la main »
20 mai 2019 – IBM France Lab@Paris-Saclay

2. Deep Learning ?
© 2019 IBM Corporation
Classification
Counting
Detection
{
"webAPIId":"8ace1c11-0f7e-4a17-a54d-63c741c7b31d",
"imageUrl":"http://powerai-vision-portal:9080/powerai-vision-
api/uploads/temp/8ace1c11-0f7e-4a17-a54d-63c741c7b31d/3f3db18c-eb02-4dca-8015-
8a68a830eb5b.jpg",
"imageMd5":"c13e24226dbd67404a4b464f18a231ac",
"classified":[
{
"confidence":0.9997112154960632,
"label":"up",
"ymax":608,"xmax":600,"xmin":503,"ymin":430
},
{
"confidence":0.9993865489959717,
"label":"up",
"ymax":543, "xmax":949,"xmin":837,"ymin":372
}
],
"result":"success"
}

3. Le véritable objectif
© 2019 IBM Corporation

4. Cycle de vie d’une application cognitive
© 2019 IBM Corporation
Idée / ROI
Application
Services
Développement
Déploiement Utilisation

5. © 2019 IBM Corporation
Suivi du parcours client
SERVER
PERFORMANCE
RETAIL
E-COMMERCE
Suivi des clients d’un supermarché

6. © 2019 IBM Corporation
Surveillance des tunnels d’autoroute
SERVER
PERFORMANCE
Détection des incidents

7. Application
© 2019 IBM Corporation
IBM Cloud Private
DevOps
Microservices

8. Services
© 2019 IBM Corporation
Deux approches
Prêt-à-porter
IBM
One AI
Sur-mesure

9. Watson
© 2019 IBM Corporation
Discovery
Knowledge
Studio
Knowledge
Catalog
Discovery
Conversation
Assistant
Empathy
Personality
Insights
Tone
Analyser
Language
Language
Understanding
Language
Classifier
Language
Translator
Voice
Speech
to Text
Text to
Speech
Perception
Visual
Recognition

10. IBM One AI
© 2019 IBM Corporation
Data Scientist App Developer AI Ops
Build AI Run AI Operate AI
Watson OpenScale
Fairness & Explainability
Inputs for Continuous Evolution
Business KPIs and production
metrics
Watson Studio
Watson Machine
Learning
Build
Deploy
and run
Operate
trusted AI
Business user
Consume AI
Data Exploration
Data Preparation
Model Development
Model Deployment
Model Management
Retraining
Watson Knowledge
Catalog
Data Profiling
Quality and Lineage
Data Governance
Organize and
Govern data
Data Engineer
Organize
Data for AI
Watson Machine Learning AcceleratorWatson Machine Learning Accelerator

11. Power AI
© 2019 IBM Corporation
Les frameworks opensource habituels optimisés
Plus de capacités, plus de possibilités, plus de
scalabilité
Des serveurs conçus pour le calcul accéléré
5.6x plus de bande passante qu’avec PCIe gen3
Solutions de partage de ressources
Des solutions pour aider à la conception et la mise à
disposition de vos services d’IA
AC922
Watson Machine Learning Accelerator

12. Surveiller l’usage
© 2019 IBM Corporation
Watson OpenScale

13. Partenariats
© 2019 IBM Corporation
Our clients
H2O.ai IBM IVAPlanetANEO
OKTAL SE Cyclope.ai

14. Aide à la réflexion sur
votre stratégie d’IA
Preuves de concepts
Collaboration avec des
partenaires
Accès à des experts
HPC / IA
Planification
d’infrastructure IA
© 2019 IBM Corporation
Ne restez pas tout seuls !

15. Partenaire
© 2019 IBM Corporation

16. © 2019 IBM Corporation
Power AI Vision

17. Démo
© 2019 IBM Corporation

18. Une stack logicielle basée sur l’opensource
© 2019 IBM Corporation
Watson Machine Learning (WML)
PowerAI
Watson ML
Watson ML AcceleratorWatson Studio
AI OpenScale
Development
Environment
Train Models
Runtime
Environment
Train, Deploy, & Manage Models
Operation Fabric
Monitor & Improve
Deployed Models
SnapML
Previous Names:
WML Accelerator = PowerAI Enterprise
GPU-Accelerated
Power Servers
Storage
PowerAI

19. Des besoins de plus en plus « gros »
© 2019 IBM Corporation
Volume of data generated
by a single car = 1 TB+ / h

20. Exemples de datasets de haute résolution
© 2019 IBM Corporation
•CityScapes : 1024x2048 5k images.
Semantic Segmentation, Benchmark
results.
•Mapillary Vistas : various sizes - split into
18,000 for training, 2,000 for validation,
5,000 for testing (average image
resolution of ~9 megapixels)
•Synthia : Large volume of data & ground
truth: +200,000 HD images from video
streams and +20,000 HD images from
independent snapshots

21. Impact de la résolution sur la précision des modèles
© 2019 IBM Corporation

22. La solution d’IBM à la limitation de la mémoire des GPUs
© 2019 IBM Corporation
Large Model Support
POWER
CPU
DDR4
GPU
NVLink
Graphics
Memory
Traditional Model Support
CPUDDR4
GPU
PCIe
Graphics
Memory
Limited memory on GPU forces trade-off in
model size / data resolution
Use system memory and GPU to support more
complex and higher resolution data
32GB max
2TB max
2TB max
32GB max
• Leveraging NVLink and CPU-GPU memory coherence enables larger and more complex models
• Improves model accuracy with more images and higher resolution images

23. Large Memory Support (LMS)
© 2019 IBM Corporation
LMS Concept
1. Treat GPU memory as an application-level cache
w.r.t the host main memory.
2. All data reside on host memory and are copied to
GPU memory only when needed.
3. After the GPU memory is used, depending on
whether it has been modified or has future usage, it
can be either copied back to the system memory or
simply discarded.
4. To efficiently utilize GPU memory, LMS
implementation keeps a large memory pool so that
different memory pieces from CPU memory can
share the same GPU memory chunk. Therefore, at
any moment, the GPU only holds data necessary to
process one operation, for example, the forward
propagation of one operation in a neural network
(Feature maps).
If the memory requirement from any operation is larger
than the GPU memory, then even LMS will fail as well. In
theory, LMS should be able to handle a deep neural
network of an arbitrary capacity, as long as all the data
from the largest operation can fit into the GPU memory.
à As LMS uses system memory to store features map
tensors, the training performance limiting factor is no
longer the PCIe bus but the CPU-RAM high-bandwidth.
Ex: Feature Maps tensor backward reuse for the gradient
update with AlexNet
Without LMS: forward temporary data are kept in
GPU memory for the backward computation

24. LMS: Caffe
© 2019 IBM Corporation
Caffe is one of the most popular open-source deep learning frameworks, developed by the Berke-
ley AI Research. It clearly separates the network description (made with a Protocol Buffer file,
similar to
JSON), and the training run, where we have a second file to specify the parameters, how we optimize
the
network, what will be logged.
This is called "define-AND-run", and most of the frame-works work that way. In addition, it provides
a Python API to write for example new layer types, which is useful for loading complex data and
feeding it to the network.
Caffe LMS Command line Flags:
-lms : to activate LMS for the current training.

25. LMS: Chainer
© 2019 IBM Corporation
Chainer Chainer is a lesser-known open-source deep learning framework in Python, popular in Japan but
not really beyond. It is developed by a Japanese com-
pany called Preferred Networks, in partnership with several companies including IBM.
Unlike Caffe, it has a "define-BY-run" approach, where network description and training are "mixed" in
the same code. This is more flexible and allows to dynamically update the
network if needed (although it is useful on very specific deep learning algorithms I haven’t used during
the internship).
Light python code change for LMS activation :
with chainer.out_of_core_mode(fine_granularity=True, async=False,
devices=gpu):
trainer.run()

26. LMS: Tensorflow
© 2019 IBM Corporation
Keras API
from tensorflow.contrib.lms import LMSKerasCallback lms_callback =
LMSKerasCallback()
…
model.fit_generator(generator=training_gen,
callbacks=[lms_callback])
Estimator API
with tf.name_scope(‘gradientscope'):
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
train_op = optimizer.minimize(loss=loss,
global_step=tf.train.get_global_step())
from tensorflow.contrib.lms import LMSHook
lms_hook = LMSHook({‘gradientscope '})
mnist_classifier.train(input_fn=train_input_fn, steps=20000,
hooks=[logging_hook, lms_hook])

27. IBM AC922
4 GPUs @150GB/s
CPU ßà GPU bandwidth
6 GPUs @100GB/s
CPU ßà GPU bandwidth
Coherent access to system memory
PCIe Gen 4 and CAPI 2.0 to InfiniBand
Air and Water cooled options
Coherent access to system memory
PCIe Gen 4 and CAPI 2.0 to InfiniBand
Water cooled only
NVLink
100GB/s
NVLink
100GB/s
NVDIA V100
Coherent
access to
system memory
(2TB)
NVLink
100GB/s
NVLink
100GB/s
NVLink
100GB/s
170GB/s
CPU
PCIe Gen 4
CAPI 2.0
NVDIA V100NVDIA V100
DDR4
I
B
Coherent
access to
system memory
(2TB)
NVLink
150GB/s
NVLink
150GB/s
170GB/s
CP
U
PCIe Gen 4
CAPI 2.0
NVLink
150GB/s NVDIA V100NVDIA V100
DDR4
IB
© 2019 IBM Corporation

28. AC922: Des capacités uniques pour l’IA
28
Large Model Support
Only server that can leverage System Memory
from GPU side for 2TB+ per node!
Distributed Deep Learning
Most efficient scaling, due to better I/O
between nodes (95% across 256 GPUS)
3.1 Hours
49 Mins
0
2000
4000
6000
8000
10000
12000
Xeon x86 2640v4 w/
4x V100 GPUs
Power AC922 w/ 4x
V100 GPUs
Time(secs)
Caffe with LMS (Large Model Support)
Runtime of 1000 Iterations
GoogleNet model on Enlarged
ImageNet Dataset (2240x2240)
Near Ideal Scaling to 256 GPUs
1
2
4
8
16
32
64
128
256
4 16 64 256
Speedup
Number of GPUs
Ideal Scaling
DDL Actual Scaling
Caffe with PowerAI DDL, Running on S822LC Power System
ResNet-50, ImageNet-1K
~4x faster insights
Train More Build More Know More
© 2019 IBM Corporation

29. LMS: impact du NVLink entre CPU et GPU
© 2019 IBM Corporation
NVLink 2.0 connected GPU training one high res 3D MRI with large model support

30. LMS: Impact sur le temps d’une epoch
© 2019 IBM Corporation
0
250
500
750
1000
1250
1500
1750
2000
2250
AC922
with NVLink 2.0
GPU server with PCI GPU server with PCI
contention
Seconds
Epoch times at max resolution with swapping

31. LMS: Impact sur le taux d’utilisation des GPUs
© 2019 IBM Corporation
GB/s
10 GB/s
20 GB/s
30 GB/s
40 GB/s
50 GB/s
60 GB/s
70 GB/s
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AC922
with NVLink 2.0
GPU server with PCI GPU server with PCI
contention
MemoryCopyThroughput
GPUUtilization
GPU utilization
Memory copy throughput

32. LMS: Overhead par rapport à la mémoire du GPU
© 2019 IBM Corporation
3%
13%
25%
0%
20%
40%
60%
80%
100%
0
100
200
300
400
500
600
700
1.4x 1.8x 2.4x
OverheadPercentage
Epochtimeinseconds Swapping overhead using resolutions above the 16GB GPU
memory limit
32 GB V100 GPU, no swapping 16GB V100 GPU, swapping Overhead

33. Une architecture évolutive
© 2019 IBM Corporation
Experimentation
Single Tenant
Stabilization & Production
Secure Multitenant
Expansion
Enterprise Scale / Multiple Lines of Business
Data
Scientist’s
workstations
Internal
SAS
drives &
NVM’s
IBM
Power
Systems
AC922
High-Speed
Network
Subsystem
Existing
Organization
Infrastructure
IBM
Elastic
Storage
Server
(ESS)
Training & Inference Cluster
IBM Power Systems AC922, LC921 & LC922
Master & Failover Master
Nodes IBM Power Systems
LC921 & LC922
Login Nodes
IBM Power Systems
LC921 & LC922
Training Cluster
IBM Power Systems AC922
IBM
Elastic
Storage
Server
(ESS)
High-Speed
Network
Subsystem
Existing
Organization
Infrastructure
One software stack from experimentation to expansion
IBM PowerAI Enterprise
Red Hat Enterprise Linux (RHEL)
IBM Power System & x86 Servers
Services&
Support
IBM Spectrum Scale / IBM Elastic Storage Server (ESS)

34. Aide à la réflexion sur
votre stratégie d’IA
Preuves de concepts
Collaboration avec des
partenaires
Accès à des experts
HPC / IA
Planification
d’infrastructure IA
© 2019 IBM Corporation
Ne restez pas tout seuls !

35. Merci !
IBM AI Portfolio
Dédiaboliser les projets d’IA…