Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps (MOB310) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bridging the Gap Between Real
Time/Offline and AI/ML Capabilities in
Modern Serverless Apps
Brice Pellé
Enterprise Support Lead
Amazon Web Services
M O B 3 1 0
Ed Lima
Solutions Architect
Amazon Web Services

Agenda
• The mobile/web app challenges in 2018
• Addressing the difficult basics
• Solving the challenges
• Boosting your application with intelligence

https://www.flickr.com/photos/118006733@N05/12619573163/
The mobile and web app
challenges in 2018

https://www.flickr.com/photos/atbaker/302838820/
Anybody
(AuthN/Z)

Anywhere
(mobility)
https://www.flickr.com/photos/mrtiking/8615241850/

On anything
(platform)
https://www.flickr.com/photos/mikecogh/7348035690/

In an instant
(Real-time)
https://www.flickr.com/photos/claustom/15288837503/

At any time
(offline)
https://www.flickr.com/photos/labdog2010/32955584723/

The mobile/web app challenge in 2018
• In 2018, these problems are still difficult to solve
• But they should be the basic requirements
• Making an app stand out is becoming even harder
• Millions of apps available across various application stores

To solve the challenge, developers need to
• Iterate quickly
• Focus on development, not servers
• Provide the basics at a low cost of development
• Offer innovative ways for users to interact with their apps
• Make applications smarter, without a PhD in data sciences

The modern mobile/web app must
Solve the security challenge
• AuthN and AuthZ by default
Solve the data challenge
• Network friendly (less calls / more data / just right)
• Instantaneous, with real-time feedback
Solve the screen and mobility challenge
• Responsive & progressive
• Always available, even offline
Solve the differentiation challenge
• Innovative & new ways to interact
• How can your app stand out?
Oh and … no servers!

Messaging application architecture overview
Machine learning services
AppSync data
sources
Raw data

https://github.com/aws-samples/aws-appsync-chat-starter-react
Machine learning services
AppSync data
sources
Raw data

Iterating quickly with AWS Amplify
• CLI & toolchain for the client
• Client library & Javascript framework support
• Supported services
• Database
• API
• Lambda functions
• Authentication
• Analytics
• Hosting
• Storage
> npm install aws-amplify
> npm install –g @aws-amplify/cli

Identity is mission critical for applications
Three key areas to address
• Authentication
• Authorization
• User management
Implementing auth infrastructure is hard!

Secure, always!
• Leverage Amazon Cognito
• Built with best security practices
• Iterate quickly with AWS Amplify
• With toolchain
• Provision serverless backend resources
• Use Auth module to easily interact
• Wrap app in authenticator to require sign-in
• Uses customizable pre-built UI components
amplify
import awsmobile from './aws-exports'
import Amplify, { Auth } from 'aws-amplify'
import { withAuthenticator } from 'aws-
amplify-react’
...
Amplify.configure(awsmobile)
...
class App extends Component {
async componentDidMount() {
session = await Auth.currentSession()
this.setState({ session })
}
}
export default withAuthenticator(App)

Solve the data challenge: GraphQL
/posts /comments /authors
REST API
posts comments authors
GraphQL API

AWS AppSync
Managed serverless
GraphQL service
Connect to data
sources in your account
Add data sync, real-time, and
offline capabilities for any data
source or API
GraphQL facade for any
AWS service
Conflict detection and
resolution in the cloud
Enterprise security features:
IAM, Amazon Cognito, OIDC,
API keys

Building a serverless messaging app with NoSQL
1..1
1..n
1..1
1..n
1..n
Relations between
NoSQL Tables
User
MessageConversation
User
Conversation

Schemas with AWS
Amplify
> amplify add api
type Conversation @model {
id: ID!
name: String!
createdAt: String
messages: [Message] @connection(name:
"ConvoMsgs")
}
type Message @searchable @model {
id: ID!
content: String
file: S3Object
messageConversationId: ID
conversation: Conversation
@connection(name: "ConvoMsgs")
}
• Define and deploy DynamoDB
tables based on schema
• Creates AWS AppSync API &
resolvers
• Supports custom transformers
> amplify api gql-compile; amplify api push
type Mutation @custom {
createMessage(input:
CustomCreateMessageInput!): Message
@custom(subscription: true,
watchedFields:
["messageConversationId"])
}

GraphQL transformer – mix and match data sources
searchMessages
mutations
queries
listMessages
createMessage
type Message
@searchable @model {
id: ID!
content: String
file: S3Object
messageConversationId: ID
conversation: Conversation
@connection(
name: "ConvoMsgs")
}

Application overview
• Data presentation split
between components
• Design independent of data
source
• Components wrapped in
graphql HOC
InputBar
ConversationBar
SearchBar
const SearchResultListWithData = graphql(search, {
skip: props => !props.term,
options: props => ({
variables: {
userTerm: `.*${props.term}.*`,
msgFilter: buildMsgFilter(props.term,
props.conversations)
},
fetchPolicy: 'cache-and-network'
})
})(SearchResultList)

AWS AppSync with
AWS Amplify
Four easy steps to configure AWS
AppSync
1. Import modules
2. Configure client
3. Advanced settings
a. Complex data handling
b. Union and interface handling
4. Wrap app in HOCs
a. Make client available to provider
b. Load app data from storage
c. Launch app when data ready
d. Require authentication
import AWSAppSyncClient from 'aws-appsync'
import { Rehydrated } from 'aws-appsync-react’
import awsmobile from './aws-exports'
const client = new AWSAppSyncClient({
url: awsmobile.aws_appsync_graphqlEndpoint,
region: awsmobile.aws_appsync_region,
auth: {
type: awsmobile.aws_appsync_authenticationType,
jwtToken: async () =>(await Auth.currentSession())
.getIdToken().getJwtToken()
},
complexObjectsCredentials: () =>
Auth.currentCredentials(),
cacheOptions: { fragmentMatcher }
})
const WithProvider = () => (
<ApolloProvider client={client}>
<Rehydrated>
<App />
</Rehydrated>
</ApolloProvider>
)
export default withAuthenticator(WithProvider)
a
b
c
d
a
b

Network friendly—
Batch requests
• Wrap component with
graphql
• Data fetch when component
is mounted but skipped if
necessary
• Cache lookup for quick user
feedback
• Use fetchMore to fetch
additional data
Messenger
const MessengerWithData =
graphql(getConvoMessages, {
skip: props => !props.conversation,
variables: {convoId:
props.conversation},
fetchPolicy: 'cache-and-network'
})
})(Messenger)
export default Messenger
export { MessengerWithData }

Instantaneous—Using
subscriptions
Subscriptions are
• Granular
• Real-time
• In line notifications
Initiated on component mount
for each opened conversation
Canceled on unmount or
conversation change
componentDidMount() {
const {
subscribeToMore,
conversation: { id: convoId }, userId
} = this.props
this.unsubscribe = subscribeToMore({
document: onCreateMessage,
variables: { messageConvoId: convoId },
updateQuery: (previous, data) => {...}
})
}
componentWillUnmount() {
this.unsubscribe()
}
MessagePane

subscription onCreateMessage {
createMessage {…}
}
Websocket URL and connection payload
Secure Websocket connection (wss://)
Subscriptions—Handshake process
AppSync Data
Sources

Solve the screen challenge: Responsive
Using a responsive
design allows for “write
once, run anywhere”
Several well-known
frameworks available

Secure ShareableNetwork agnostic Always up-to-date
Solve the mobility challenge: Progressive
Write once,
run anywhere

Solve the mobility challenge: Always available
With AWS AppSync SDK
• Configured by default
• Use Rehydrated component
to rehydrate cache from
storage
• Use: fetchPolicy to
control cache and network
usage
• AWS AppSync SDK maintains
an outbox of pending actions
• If no connection, then store action
• On reconnection, send action
• Development: Use redux tools

Offline data rendering
• Provide optimistic response
• Useful when offline or slow
network
• Required when using offline
capabilities!
• Provide all fields present in
the returned selection set
mutation({
variables,
optimisticResponse: {
createMessage: {...}
},
update: (proxy, data) => {
const QUERY = { query:
getConvoMessages, variables: { convoId } }
const prev = proxy.readQuery(QUERY)
const data = {...}
proxy.writeQuery({ ...QUERY, data })
}
})
MessagePane

Platform services
Application services
Frameworks & interfaces
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Torch Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker
Amazon
Rekognition
Amazon
Transcribe
Amazon
Translate
Amazon Polly
Amazon
Comprehend
Amazon Lex
Amazon Mechanical Turk
The AWS Machine Learning stack

PLATFORM SERVICES
Application services
FRAMEWORKS & INTERFACES
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Torch Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker
Amazon
Rekognition
Amazon
Transcribe
Amazon
Translate
Amazon Polly
Amazon
Comprehend
Amazon Lex
Amazon Mechanical Turk
The AWS Machine Learning stack

The AWS Machine Learning stack—App services
Vision Speech Language Chatbots

Vision Speech Language Chatbots
The AWS Machine Learning stack—ChatQL app

Common language use cases
Information bots
Education
Accessibility
Knowledge management
Voice of customer
applications
Customer service/
call centers
Enterprise
digital assistant
Semantic search
Captioning workflows
LocalizationPersonalization

Amazon Translate
Natural and fluent language translation
Real-time
translation
Batch analysis Automatic language
recognition
Low cost

Amazon Polly
Turn text into lifelike speech using deep learning
Wide selection of
voices and languages
Synchronize
speech
Fine-grained
control
Unlimited replay

Amazon Comprehend
Storm
World Series
Australia
Stock market
Washington
Health
Crisis Machine learning
Library of news
articles
Amazon
Comprehend
Discover insights and relationships in text using deep learning

Amazon Comprehend
Discover insights and relationships in text using deep learning
Entities
Key phrases
Language
Sentiment
Amazon
Comprehend

A m a z o n . c o m , I n c . i s l o c a t e d i n
S e a t t l e , W A a n d w a s f o u n d e d
J u l y 5 t h , 1 9 9 4 b y J e f f B e z o s .
O u r c u s t o m e r s l o v e b u y i n g
e v e r y t h i n g f r o m b o o k s t o
b l e n d e r s a t g r e a t p r i c e s
N a m e d e n t i t i e s
A m a z o n . c o m : O r g a n i z a t i o n
S e a t t l e , W A : L o c a t i o n
J u l y 5 t h , 1 9 9 4 : D a t e
J e f f B e z o s : P e r s o n
K e y p h r a s e s
O u r c u s t o m e r s
b o o k s
b l e n d e r s
g r e a t p r i c e s
S e n t i m e n t
P o s i t i v e
L a n g u a g e
E n g l i s h
Text analysis

Amazon Lex
Building conversational interfaces into any application
using voice and text
Integrated
development in the
AWS console
Trigger
Lambda
functions
Multi-step
conversations
One-click
deployment
Enterprise
connectors
Fully
managed

Amazon Rekognition Image
Detect objects, scenes, and faces, extract text and identify unsafe content in images
Object and scene
detection
Facial
analysis
Face
recognition
Text in imageUnsafe image
detection
Celebrity
recognition
Face comparison

AppSync Data
Sources
Raw Data
Machine Learning Services

Request template Response template
Data sources
1 2 3 4 5
GraphQL data flow in AWS AppSync

An AI component
import React from "react";
import { graphql } from "react-apollo";
import rekognition from "../../images/rekognition.png";
import detectLabels from "../../graphql/queries/detectLabels";
class Labels extends React.Component {
render() {
const {
data: { loading, error, detectLabels }
} = this.props;
if (loading) {
return (<JSX…>);
} else if (error) {
const err = JSON.stringify(error.message);
return (
return (<JSX…>);
} else {
const response = JSON.parse(detectLabels.response);
return (<JSX…>);
}
}
export default graphql(detectLabels, {
skip: props => !props.path,
variables: {
bucket: props.bucket,
key: props.path
},
fetchPolicy: "cache-and-network"
})
})(Labels);

AWS AppSync as GraphQL facade for AI services
import gql from "graphql-tag";
export default gql`
query invokeBot($bot: String, $text: String) {
invokeBot(bot: $bot, text: $text) {
response
}
}
`;
{
"version" : "2017-02-28",
"operation": "Invoke",
"payload": {
"field": "lex",
"arguments": {
"bot": "$context.arguments.bot",
"sender": "$context.identity.sub",
"text": "$context.arguments.text"
}
}
}
exports.handler = (event, context, callback) => {
switch (event.field) {
case "lex":
const lexparams = {
botAlias: "$LATEST",
botName: event.arguments.bot,
inputText: event.arguments.text,
userId: event.arguments.sender
};
lex.postText(lexparams, function(err, data) {
if (err) {
console.error("Error JSON: ",
JSON.stringify(err, null, 2));
callback(err);
} else {
let result = {
bot: event.arguments.bot,
text: event.arguments.text,
response: data.message
};
callback(null, result);
}
});
break;

Conversation
Real-time intelligence

Conversation
Real-Time Intelligence

Real-time intelligence

Automatic language translation with speech

Sentiment and entity analysis

Offline intelligence

Image and audio intelligence

Image and audio intelligence
bucket
object

The modern mobile/web app must
• AWS Amplify and Amazon Cognito
Solve the data challenge
• AWS AppSync and GraphQL
Solve the screen and mobility challenge
• AWS AppSync + PWA built-in offline capabilities
Solve the differentiation challenge
• Great user (+ developer) experience (real-time, pagination, toolchain)
• Add Intelligence to your app with AWS AI/ML
Oh and … no servers!

https://github.com/aws-samples/aws-appsync-chat-starter-react

Thank you!
Brice Pellé
Enterprise Support Lead
Amazon Web Services
Ed Lima
Solutions Architect
Amazon Web Services
@ednergizer@BicePelle

Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps (MOB310) - AWS re:Invent 2018

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Similar a Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps (MOB310) - AWS re:Invent 2018

Similar a Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps (MOB310) - AWS re:Invent 2018 (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps (MOB310) - AWS re:Invent 2018