Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Market Prediction Using ML
Experiment with Amazo...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our mission
Put machine learning
in the hands of...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
 Overview of Amazon SageMaker
 Overview...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The Amazon Machine Learning stack
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
1
I
Notebook instances
2
I
Algo...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
End-to-end machine
learning platform
Zero setup ...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stock market data maintained by Deutsche Börse
•...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deutsche Börse Public Data Set
Source:
s3://deut...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless data lake & analytics with AWS
Crawle...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visualization for a trading analyst
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Begin with a notebook
• First step: create lifec...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Explore stock behavior
• Generate database
• Cre...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Clustering of similar stocks
• Clustering algori...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Containerize custom MLP code
• Data channels
• P...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy custom MLP model
• Model deployment
• Cre...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Train DeepAR model
• DeepAR data set
• Format cl...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy DeepAR model
• Model deployment
• Create ...
Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Binoy Das
binoyd@amazon.com
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Market Prediction Using ML: Experiment with Amazon SageMaker and the Deutsche Börse Public Dataset (FSV309) - AWS re:Inven...
Próxima SlideShare
Cargando en…5
×

Market Prediction Using ML: Experiment with Amazon SageMaker and the Deutsche Börse Public Dataset (FSV309) - AWS re:Invent 2018

351 visualizaciones

Publicado el

In this workshop, learn how to use machine learning to analyze the Deutsche Börse Public Dataset, which consists of trade data aggregated to one-minute intervals from the Tradex and Eurex engines, comprising a variety of equities, funds, and derivative securities. The public dataset provides initial price, lowest price, highest price, final price, and volume for every minute of the trading day, and for every tradeable security. Learn how to apply a variety of ML models to the data to find patterns and methods to predict price movements or identify trends in the market. Also learn how to interact with data staged for analysis in Amazon S3, use AWS Glue to transform data into analysis-ready formats, and use Amazon SageMaker and Amazon EC2 to experiment with a variety of ML models to derive insights from the data.

  • Sé el primero en comentar

Market Prediction Using ML: Experiment with Amazon SageMaker and the Deutsche Börse Public Dataset (FSV309) - AWS re:Invent 2018

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Market Prediction Using ML Experiment with Amazon SageMaker and the Deutsche Börse Public Dataset Binoy Das Partner Solution Architect Amazon Web Services F S V 3 0 9 Dave Rocamora Partner Solution Architect Amazon Web Service
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our mission Put machine learning in the hands of every developer, data scientist, and architect
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda  Overview of Amazon SageMaker  Overview of Deutsche Börse Public Data Set  Overview of Amazon Athena and Amazon QuickSight  Hands-on—data preparation  Hands-on—data analysis  Hands-on—custom container on Amazon Elastic Container Registry (Amazon ECR) to implement RNN model  Hands-on—Use DeepAR on Amazon SageMaker
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The Amazon Machine Learning stack
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker 1 I Notebook instances 2 I Algorithms 3 I ML training service 4 I ML hosting service
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. End-to-end machine learning platform Zero setup Flexible model training Pay by the second Amazon SageMaker—Value proposition The quickest and easiest way to get ML models from idea to production $
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Stock market data maintained by Deutsche Börse • Opening, closing, highest and lowest price, and trading volume for every security traded Published on Registry of Open Data on AWS https://registry.opendata.aws/deutsche-boerse-pds/
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deutsche Börse Public Data Set Source: s3://deutsche-boerse-xetra-pds (for Xetra data) s3://deutsche-boerse-eurex-pds (for Eurex data) Data dictionary: Field Description Type Mnemonic Stock exchange ticker symbol Text Date Date of trading period Date Time Minute of trading to which this entry relates Time StartPrice Trading price at the start of period Decimal MaxPrice Maximum price over the period Decimal MinPrice Minimum price over the period Decimal EndPrice Trading price at the end of the period Decimal TradedVolume Total value traded Decimal NumberOfTrades Number of distinct trades during the period Integer
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serverless data lake & analytics with AWS Crawlers scan your data sets and populate the AWS Glue Data Catalog The AWS Glue Data Catalog serves as a central metadata repository Once catalogued in AWS Glue, your data is immediately available for analytics 1 2 3 Amazon Redshift Spectrum Amazon EMR Amazon Athena AWS Glue Data Catalog AWS Glue crawler Amazon S3 Amazon QuickSight 1 2 3
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Visualization for a trading analyst
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Begin with a notebook • First step: create lifecycle configurations • Clone from GitHub • Create folder structures • Obtain cleaned up data https://github.com/aws-samples/db-reinvent-workshop
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Explore stock behavior • Generate database • Create AWS Glue crawler • Create tables • Queries for introspection • Create Athena queries • Create custom views • Stock visualization • Create Amazon QuickSight dashboard • Explore similarities in behavior
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Clustering of similar stocks • Clustering algorithm • Run custom clustering algorithm • Find correlation • Group clustered stocks • Retrieve metrices for clustered stocks • Create separate data files for clusters
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containerize custom MLP code • Data channels • Peek into data • Load to Amazon SageMaker S3 bucket • Container image • Bundle code and libraries • Create Amazon ECR repository • Model training • Create Amazon SageMaker estimator • Fit estimator to the data
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deploy custom MLP model • Model deployment • Create Amazon SageMaker predictor • Deploy to Amazon SageMaker endpoint • Inference generation • Generate prediction with test data • Visualize prediction against observed • Performance evaluation • Measure accuracy • Scope for improvement
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Train DeepAR model • DeepAR data set • Format clustered data for DeepAR • Create training and test data • Model training • Create Amazon SageMaker training job • Observe training behavior
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deploy DeepAR model • Model deployment • Create endpoint configuration • Create Amazon SageMaker endpoint • Inference generation • Generate prediction with test data • Visualize prediction against observed • Performance evaluation • Measure accuracy • Repeat with different cluster
  25. 25. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Binoy Das binoyd@amazon.com
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×