Más contenido relacionado La actualidad más candente (20) Similar a Construindo Aplicações Deep Learning com TensorFlow e Amazon SageMaker - MCL302 - Sao Paulo Summit (20) Más de Amazon Web Services (20) Construindo Aplicações Deep Learning com TensorFlow e Amazon SageMaker - MCL302 - Sao Paulo Summit1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Samir Araújo
AI Solutions Architect
Session BDA304
Construindo aplicações de Deep
Learning com TensorFlow e Amazon
SageMaker
2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
IA em 2018 (não 2020) — Algumas aplicações de ML
3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Fundação de IA
• Apps Inteligentes & Analytics
• IoT Inteligente
• Cloud na borda
• Gêmeos Digitais
• Experiencias Imersivas
• Plataforma coversacionais
• Blockchain
• Eventos
• Confiança contínua de risco adaptável
10 Tendências estratégicas para 2018 — Gartner
4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Fundação de IA
• Apps Inteligentes & Analytics
• IoT Inteligente
• Cloud na borda
• Gêmeos Digitais
• Experiencias Imersivas
• Plataforma coversacionais
• Blockchain
• Eventos
• Confiança contínua de risco adaptável
10 Tendências estratégicas para 2018 — Gartner
Envolve IA
5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML é o novo campo de batalha
6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Amazon Glacier AWS Glue Data Lake
Amazon Redshift +
Spectrum
EMR AthenaKinesis Analytics
Plataforma
&
Ferramentas
Serviços
Amazon
Rekognition
Lex
Polly Translate
Transcribe Comprehend
Snowball
AWS é o centro de gravidade para ML
AWS IoT
Snowmobile
DBS Migration
AWS Greengrass
ML
Borda
MTurk
Kinesis
Amazon SageMakerGPU
9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning é parte de um Pipeline
11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data lake Algoritimo Modelo
12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data lake Algoritimo Modelo
Predição
Novos Dados
13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Imagens Rede Neural
Convolucional
Modelo
Objetos, localização, faces
Nova Imagem
14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Inventário Cluster
K-means
Modelo
Segmentação de clientes
Novo item
15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Compras Regressão
Logistica
Modelo
Risco de fraude
Nova compra
16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data lake Algoritimo Modelo
Predição
Novo dadoAmazon
SageMaker
17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Algoritimo
como serviço
+ código aberto
Data lake Modelo
Predição
Novo dadoAmazon
SageMaker
Modelo
treinamento
& otimização
Modelo
altamente
disponivel
18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Inferencia
com baixa
latência
Algoritimo
como serviço
+ código aberto
Data lake Modelo
Predição
Novo dadoAmazon
SageMaker
Modelo
treinamento
& otimização
Modelo
altamente
disponivel
19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Algoritimo
como serviço
+ código aberto
Data lake
Amazon
SageMaker
DeepAR
BlazingText
Matriz de factorização
K-Means
PCA
LDA
Classificação de imagens
Seq2Seq
XGBoost
NTM
Aprendiz linear
BYOA
20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Adicionando inferência além da nuvem
21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data lake Algoritimo Modelo
Amazon
SageMaker Greengrass ML
Borda
22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Plataforma
&
Ferramentas
Plataforma e Ferramentas de IA
Amazon SageMaker: Platforma de ML End-to-end
Facilita a construção, treinamento, e publicação de modelos de ML em produção
Machine learning
Amazon SageMakerGPU
23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Plataforma e Ferramentas de IA
Plataforma
&
Ferramentas
Amazon SageMaker
Machine learning
+
Amazon SageMaker
Amazon SageMaker: Platforma de ML End-to-end
Facilita a construção, treinamento, e publicação de modelos de ML em produção
GPU
24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Plataforma e Ferramentas de IA
Plataforma
&
Ferramentas
Amazon SageMaker
Machine learning
+
Amazon SageMaker
Agregação de dados, anotação e
analytics
Amazon SageMaker: Platforma de ML End-to-end
Facilita a construção, treinamento, e publicação de modelos de ML em produção
GPU
25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Plataforma e Ferramentas de IA
Plataforma
&
Ferramentas
Amazon SageMakerGPU
26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Clientes do Amazon SageMaker
27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Construa teu modelo com
no Amazon SageMaker
28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Passo 1: Workspace do Data Science
Amazon SageMaker. Instância
preparada com Jupiter Notebook
29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Passo 2: Introdução à Tensorflow
Exemplos de notebooks no Amazon SageMaker
31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Source: https://en.wikipedia.org/wiki/Neuron
32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Conceitos Básicos
import numpy as np
import tensorflow as tf
33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tensores
import numpy as np
import tensorflow as tf
# Tensors and nodes
with tf.name_scope("sample_nodes"):
node_1 = tf.placeholder(tf.float32, shape=(None,28), name="tensor_1")
tensor_2 = tf.fill((2,5), 10, name="tensor_2")
tensor_3 = tf.fill((5,4), 10, name="tensor_3")
Conceitos Básicos
34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
import numpy as np
import tensorflow as tf
# Tensors and nodes
with tf.name_scope("sample_nodes"):
node_1 = tf.placeholder(tf.float32, shape=(None,28), name="tensor_1")
tensor_2 = tf.fill((2,5), 10, name="tensor_2")
tensor_3 = tf.fill((5,4), 10, name="tensor_3")
# Sample operation
with tf.name_scope("sample_op"):
res1 = tf.matmul(tensor_2, tensor_3, name="matrix_multiplication")
Nós
Conceitos Básicos
35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
import numpy as np
import tensorflow as tf
# Tensors and nodes
with tf.name_scope("sample_nodes"):
node_1 = tf.placeholder(tf.float32, shape=(None,28), name="tensor_1")
tensor_2 = tf.fill((2,5), 10, name="tensor_2")
tensor_3 = tf.fill((5,4), 10, name="tensor_3")
# Sample operation
with tf.name_scope("sample_op"):
res1 = tf.matmul(tensor_2, tensor_3, name="matrix_multiplication")
# Variables and Operations overload
with tf.name_scope("sample_graph"):
a = tf.Variable(initial_value=3.5, dtype= tf.float32, name="a")
b = tf.Variable(initial_value=5.2, dtype=tf.float32, name="b")
c = tf.Variable(initial_value=0.25, dtype=tf.float32, name="c")
# Operation overload
u = tf.add((2 * a), (6 * b), name="u")
J = tf.add((3 * u), c, name="J")
Gráfo Computacional
Conceitos Básicos
36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
import numpy as np
import tensorflow as tf
# Tensors and nodes
with tf.name_scope("sample_nodes"):
node_1 = tf.placeholder(tf.float32, shape=(None,28), name="tensor_1")
tensor_2 = tf.fill((2,5), 10, name="tensor_2")
tensor_3 = tf.fill((5,4), 10, name="tensor_3")
# Sample operation
with tf.name_scope("sample_op"):
res1 = tf.matmul(tensor_2, tensor_3, name="matrix_multiplication")
# Variables and Operations overload
with tf.name_scope("sample_graph"):
a = tf.Variable(initial_value=3.5, dtype= tf.float32, name="a")
b = tf.Variable(initial_value=5.2, dtype=tf.float32, name="b")
c = tf.Variable(initial_value=0.25, dtype=tf.float32, name="c")
# Operation overload
u = tf.add((2 * a), (6 * b), name="u")
J = tf.add((3 * u), c, name="J")
# Variables initialization and session execution
init = tf.global_variables_initializer()
with tf.Session() as s:
init.run()
result1 = s.run(res1)
result2 = s.run(J)
Inicialização da sessão
e execução
Conceitos Básicos
37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Passo 3: Tensorflow no
Amazon SageMaker
38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
com Amazon SageMaker
e SDK Python
import os
import sagemaker
from sagemaker import get_execution_role
sagemaker_session = sagemaker.Session()
role = get_execution_role()
1) Importar pacotes
2) Iniciar uma seção no Amazon SageMaker
39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
# Upload data to S3 bucket
inputs = sagemaker_session.upload_data(
path='/tmp/cifar10_data',
key_prefix='data/cifar10')
# Create a training job using sagemaker.TensorFlow estimator
from sagemaker.tensorflow import TensorFlow
source_dir = os.path.join(os.getcwd(), 'source_dir')
estimator = TensorFlow(entry_point='resnet_cifar_10.py',
source_dir=source_dir,
role=role,
training_steps=1000, evaluation_steps=100,
train_instance_count=5, train_instance_type='ml.c4.xlarge',
base_job_name='tensorboard-example')
estimator.fit(inputs, run_tensorboard_locally=True)
import os
import sagemaker
from sagemaker import get_execution_role
sagemaker_session = sagemaker.Session()
role = get_execution_role()
1) Importar pacotes
2) Iniciar uma seção no Amazon SageMaker
3) Criar um job de Treinamento
i. Usar o objeto Tensorflow Estimator
ii. Coloque seu script no source_dir
4. Fit the Estimator, e inicie TensorBoard
com Amazon SageMaker
e SDK Python
40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
# Upload data to S3 bucket
inputs = sagemaker_session.upload_data(
path='/tmp/cifar10_data',
key_prefix='data/cifar10')
# Create a training job using sagemaker.TensorFlow estimator
from sagemaker.tensorflow import TensorFlow
source_dir = os.path.join(os.getcwd(), 'source_dir')
estimator = TensorFlow(entry_point='resnet_cifar_10.py',
source_dir=source_dir,
role=role,
training_steps=1000, evaluation_steps=100,
train_instance_count=5, train_instance_type='ml.c4.xlarge',
base_job_name='tensorboard-example')
estimator.fit(inputs, run_tensorboard_locally=True)
# Deploy the trained model for predictions
predictor = estimator.deploy(
initial_instance_count=1,
instance_type='ml.m4.xlarge')
import os
import sagemaker
from sagemaker import get_execution_role
sagemaker_session = sagemaker.Session()
role = get_execution_role()
5) Faça o deploy do modelo treinado para predições
1) Importar pacotes
2) Iniciar uma seção no Amazon SageMaker
3) Criar um job de Treinamento
i. Usar o objeto Tensorflow Estimator
ii. Coloque seu script no source_dir
4. Fit the Estimator, e inicie TensorBoard
com Amazon SageMaker
e SDK Python
41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
6) Log em tempo real
com Amazon SageMaker
e SDK Python
42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
7) Suporte ao TensorBoard
com Amazon SageMaker
e SDK Python
43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
7) CloudWatch Logs
com Amazon SageMaker
e SDK Python
44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Iniciando com
Amazon SageMaker
45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Algoritimos e Notebooks
46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshops e treinamento AWS DeepLens
Amazon SageMaker
Algoritimos e Notebooks
47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ML Solutions Lab
Workshops e treinamento AWS DeepLens
Amazon SageMaker
Algoritimos e Notebooks
48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
aws.amazon.com/sagemaker
49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kunumi - Rodrigo Borges
ML/Software Engineer
Diagnóstico de Alzheimer
50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Infraestrutura de Deep Learning da
AWS para diagnóstico de Alzheimer”
“A infraestrutura de Deep
Learning da AWS
simplifica bastante o
treinamento de redes
neurais”
- Eduardo Nigri,
Data Scientist
A Kunumi é uma start-up de
Inteligência Artificial aplicada para
solucionar vários tipos de problema
em diferentes mercados
Foco em modelos preditivos
utilizando Machine Learning e
desenvolvimento de pesquisa na
área
51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
O Desafio
Classificação de imagens de
ressonância magnética para diagnóstico
de Alzheimer
Imagens 3D de alta complexidade
Necessário treinamento em GPU
(treino ~13x mais rápido que em CPU)
Volume de dados: ~2500 imagens
Crédito: https://www.alz.org
52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solução
Saudável
ou
Alzheimer
Rede Neural Convolucional
AWS Deep Learning AMIs
(Amazon Machine Images)
53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solução
● Arquitetura de rede convolucional: VGG (16 camadas)
● Taxa de acerto de 85%
● Regiões com evidência de Alzheimer:
54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Por favor complete o survey da seção no app
do AWS Summit SP
55. Submit Session Feedback
1. Tap the Schedule icon. 2. Select the session you
attended.
3. Tap Session Evaluation
to submit your feedback.
56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Obrigado.