AI in the Enterprise at Scale

1. AI in the Enterprise at Scale ClarisseTaaffe-Hedglin Executive IT Architect IBM Garage IBM Systems clarisse@us.ibm.com

2. Agenda Data AnalyticsToday The AI Ladder and Lifecycle AI at ScaleThemes Infrastructure Considerations

3. • No governance • No collaboration • Limited complexity How Customers Do Data Analytics Traditionally Spreadsheets • Broad rules and categories • Not dynamic Business Rules • Hard to maintain • Pre-set rules and approaches Homegrown Applications • Limited use of analytics • Hard coded models that do not apply to unique needs • Slow response Other Applications

4. 4 Enterprise Analytics Modernization: From Data to Actions 010101010101010111100010011001010111 0000000000010101010100000000000 111101011 11000 000000000000 111111 010101 101010 10101010100 Prescriptive What should we do ? Descriptive What Has Happened? Cognitive Learn Dynamically Predictive What Will Happen? ACTIONDATA HUMAN INPUTS < < > < > > >

5. Predict a Future Event Segment Data / Detect Anomalies Determine optimal quantity, price, resource allocation, or best action Understand Past Activity Discover Insights in Content (text, images, video) Interact in Natural Language Forecast and Budget based on past activity Supervised Unsupervised Predictive: What will happen? Prescriptive: What should we do? Descriptive: What happened? Planning: What is our Plan? NLPDeep Learning Supervised Common Patterns of Analytics Solving challenges with Data and AI will utilize a combination of these analytics patterns

6. Three broad categories of AI Use Cases “Structured” Data Use Cases Computer Vision Use Cases - Big Data (Rows and Columns) - GPU Servers - Available AI Software More Accuracy ! This is sort of “Magic” - a deep learning Model is trained to detect and classify objects Natural Language Processing Use Cases - A Model learns to read and hear and “understand” language

7. Organizations are adopting AI to solve business problems Fraud Safety, inspection and process improvement Defense and security

8. “AI is the fastest-growing workload”* 8*Forrester Research Inc. “AI Deep Learning Workloads Demand a New Approach to Infrastructure”, by Mike Gualtieri, Christopher Voce, Srividya Sridharan, Michele Goetz, Renee Taylor, May 4, 2018.

9. COLLECT - Make data simple and accessible ORGANIZE - Create a trusted analytics foundation ANALYZE - Scale AI everywhere with trust & transparency Data of every type, regardless of where it lives MODERNIZE your data estate for an AI and multicloud world INFUSE – Operationalize AI across business processes The AI Ladder A prescriptive approach to accelerating the journey to AI 9 AI AI-optimized systems infrastructure

10. Unstructured, Landing, Exploration and Archive Operational Data Real-time Data Processing & Analytics Transaction and application data Machine, sensor data Enterprise content Image, geospatial, video Social data Third-party data Information Integration & Governance Data is Prerequisite to AI Risk, Fraud Chat bots, personal assistants Supply Chain Optimization Dynamic Pricing, Recommenders Behavior Modeling Vision, Autonomous Systems

11. Available data Sources Public data Anything data system can pull from the outside world for free through web connections, databases, IoT and sensors Proprietary data What private data from the outside world could the system be given permission to use? Purchased data What pre-trained data could the system buy or subscribe to? IBM Skills Academy / © Copyright 2018 IBM Corporation Ground truth Data used to define what the system knows from day one Domain knowledge Data resources that can be used to teach the system to understand and be an expert in a particular field Private data Unique data the creator owns and only shares internally Personal public data What unique data does the creator share with the outside world? Transaction and application data Machine, sensor data Enterprise content Image, geospatial, video Social data Third-party data

12. Enterprise Data Pipeline for AI Insights Out Trained Models, simulations Inference Data In Transient Storage SDS/Cloud Global Ingest Throughput-oriented, globally accessible Cloud ETL High throughput, Random I/O, SSD/Hybrid Archive High scalability, large/sequential I/O HDD Cloud Tape Hadoop / Spark Data Lakes Throughput-oriented Hybrid/HDD ML / DL Prep ⇨ Training ⇨ Inference High throughput, low latency, Random I/O SSD/NVMe Classification & Metadata Tagging High volume, index & auto-tagging zone Fast Ingest / Real-time Analytics High throughput SSD Throughput-oriented, software defined temporary landing zone capacity tier performance tier performance & capacity Tier performance & capacity Tier performance tier capacity tier FitsTraditional and New UseCases EDGE INGEST ORGANIZE ANALYZE INSIGHTSML / DL IBM Spectrum Scale / Storage for AI / © 2020 IBM Corporation

13. Metadata-Fueled Data Analysis Large Scale Data Ingest • Scan records at high speed • Live event notifications • Capture system-level tags • Automatic indexing Business-Oriented Data Mapping • Custom data tagging • Content-inspection via APIs • Policy-driven workflows Data Activation • Data movement via APIs • Extensible architecture • Solution Blueprints Data Visualization • Query billions of records in seconds • Multi-faceted search • Drilldown dashboard • Customizable reports

14. AI Model Development Workflow •Data preparation, cleaning, labelling •Model development environment •Runtime environment •Train, deploy and manage models •Business KPI and production metrics •Explainability and fairness Data Engineering and Data Science Team IT Operations Team

15. Data Science Exploration to Production Use Case Exploration Data Science Model Build Use Case Deployment in Production Requires solution architecture Deploy Source: https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf Use Case Exploration Data Science Model Build Security, Privacy and Governance

16. • #1 Model Quality • Not enough knowledge about the problem to build a good model • #2 System Usage • Typical optimizations are limited to serial data collection • #3 Complexity • Typical optimizations do not work in high dimensions • #4 Trust • Typical optimizations do not explain their logic to the user Designing Models Driven By User Desires

18. AI Use Case: Automate diagnostics to increase productivity DIAGNOSTICS Faster results with higher accuracy can be achieved with an image processing system designed to • address workflow burdens, • data governance challenges, and • analysis challenges with the goal of reducing • false negative rates in imaging diagnostics and in clinical settings, • patient risk and medical legal risk.

19. Examples of Medical Imaging Applications s. 3D-UNet segmentation models with higher resolution images allows for learning and labeling finer details and structures of brain tumors. https://developer.ibm.com/linuxonpower/2018/07/27/tensorflow-large- model-support-case-study-3d-image-segmentation/ Automatic skin lesion image analysis for melanoma detection with Memorial Sloan Kettering (MSK-CC) Diagnosis of blood-based with characterization of patient blood samples to detect and classify blood cell subtypes DIAGNOSTICS

20. Optimizing Medical Imaging Enhance image identification with deep learning to assist physicians and benefit patients 1300 MRI images trained by IBM Power Systems and IBM Storage in just two hours, compared to forty hours on traditional architectures 20x faster DIAGNOSTICS

21. Actionable Decisions Image Sensor Database Text Data Fusion Sources Fact Tables AI & Analytics Language Analytics Classification Detection Time Series Descriptive Statistics AI Use Case: NoviLens AI Appliance Under the Hood – End to End Workflow 21© 2020 NoviSystems Corporation Questions ? ** TechData IBM Reseller in Research Triangle Park *Patents pending on the methods employed by NoviLens DATA FUSION

23. Accelerated workflow uses fewer calculations to achieve orders of magnitude resolution increase AI Use Case: Molecular Modeling Achieves human level performance in days instead of months. Force Field Tuning Intelligent Phase Diagram Exploration MOLECULAR SIMULATION

24. 24 • Advances in instrument design, sample preprocessing and mathematical methods have enabled high volume throughput imaging at atomic scale. • Cryogenic electron microscopes generate an average of 5 TB of image data per day BIOMOLECULAR STRUCTURE AI Use Case: Massive data sets require massive processing capability

25. Accelerating Cryo-EM Imaging Analysis Reduced time-to-completion for high resolution image analysis jobs while increasing resource utilization Using IBM AC922 cluster, more than 100 cryo-EM high resolution image workload analysis jobs running in parallel on Satori cluster 100+ BIOMOLECULAR STRUCTURE

26. Traditional infrastructure isn’t suited for AI workloads Systems don't easily scale to meet demand Processor not optimized for AI workloads The wrong infrastructure puts AI at risk. Data pipeline too slow, causing bottleneck effect

27. Common AI Data Considerations Data Compute Legacy Data Stores IoT, Mobile & Sensors Collaboration Partners New Data Ingest InferenceTrainingPreparation Iterative Model training to improve accuracy Champion Challenge r -”Data Center” - At Edge Trained Model  Ease to Massively Scale  High Performance  Tiered / Archive  Secure  High Performance  Metadata Tagging  Single Name Space Low Latency Dev & Inference Stack - Open Source - Stable and Supported - Auditable Productivity Performance Robustness Considerations

28. Infrastructure Demands for AI Equipped for volumes of data Flexible storage for a range of data demands Versatile, power-efficient data center accelerators Advanced I/O for minimal latency Scalability and distributed data center capability Inference Powerful data center accelerators with coherence Advanced I/O for high bandwidth and low latency Proven scalability Training Equipped for volumes of data

30. Inferencing Considerations Real-Time (vs Batch): Many AI applications have response times in milli-seconds and in many cases have 100K+ IOT events per second (Latency, Latency, Latency) Scalability: Ability to scale inference engine and manage infrastructure Data Pipeline: The data that is feed into models has to be cleaned and structured to produce accurate results Security: Applications running AI models in the field and back-offices Multi-Tenancy: Multiple business applications leveraging shared infrastructure, Multiple Models per Business Application Tools Proliferation: Analytics, Data/Object Tagging, Model Training and Inferencing Model Management: Continuous Training/Re-Training of Models, AI-DevOps, Ease of Deployment Transparency: Ability to explain decisions A C C U R A C Y Transaction integration Huge Scale As-a-Service offering Inference Data Center or In-Cloud Multi-Tenancy Low latency Data movement considerations Near Edge Inferencing On-prem or In-Cloud Inference at Edge On-prem/device Stand alone device Low latency Data movement considerations Typical AI Inferencing Scenarios

31. © 2020 IBM Corporation31 Quality Inspection - Very low latency Equipment Sensors - low latency Servers GPU (IC922) Storage ( ESS ) Optimization - batch Factory location 2 Use Case: Manufacturing Cloud / IOT Servers GPU Storage Quality Inspection - Very low latency - Device Inference? Equipment Sensors - low latency Servers GPU / FPGA Storage ( ESS ) Plant Optimization - batch Factory location 1 On-Prem AI Model Training Enterprise Systems AI inferencing In Transaction Systems Headquarters AI Applications and Data Hybrid Cloud - Containers - Cloud Paks Data and meta-data Archive

33. OpenPOWER is a technical community dedicated to expanding the the IBM Power architecture ecosystem https://github.com/open-ce Open-CE Minimize time to value for foundational ML/DL packages Provide a flexible source-to-image solution to provide a complete and customizable AI environment.

35. Data Data Data Microservices Containerized Workloads Multicloud Provisioning Public Cloud On-prem ises An architecture of loosely coupled data services, easily refactored to create containerized workloads Stand-alone workloads composed of microservices & data that are flexibly deployed, orchestrated and managed Agile provisioning of containerized workloads in multicloud environments and consumption of cloud services Cloud Native Platforms Agility Efficiency Cost Savings IBM Cloud Pak for Data

36. Hybrid Cloud can help business innovate and transform © 2020 IBM Corporation36 Migrate to Hybrid Cloud Transform the Business Innovate the Business Evolve the IT Landscape with Power Systems 20K+ clients running mission critical workloads on Power Systems. IBM Systems is the engine behind Enterprise. 62% of Power customers prefer cloud deployment by 2021 Innovation is only possible if the IT landscape can evolve leveraging hybrid cloud technologies IBM Systems, IBM Cloud, and IBM Services jointly create hybrid cloud solutions

37. IBM Power Systems: Enhancements to hybrid multicloud capabilities Existing apps Mission critical | Data Intensive Emerging apps Containerized | Cloud native AIX VMs Red Hat OpenShift Cloud Paks Application | Data Linux VMs IBM i VMs Infrastructure View IBM Cloud Power Virtual Server On-Prem IaaS: IBM PowerVC Other Clouds Low upfront cost Pay per use consumption Optimum resource utilization Cloud Pak for Multicloud Management ON-PREM AND PRIVATE CLOUD CAPABILITIES: • Expanding Power Private Cloud Solution to include Scale Out systems • New cloud optimized scale-out models (including more affordable entry point for IBM i) POWER PUBLIC CLOUD EXPANSION: • More capacity and more locations for Power in the IBM Cloud • Introducing SAP HANA on Power in the IBM cloud* HYBRID CLOUD MANAGEMENT AND AIX/IBM i APPLICATION MODERNIZATION • Launching Cloud Pak for Data, Cloud Pak for Applications and OpenShift 4 on Power • Consistent and automated hybrid cloud management with Ansible for IBM Power Systems Power Systems Infrastructure ARCHITECTED FOR EMERGING APPS AND MISSION CRITICAL CHOOSE WHERE YOU DEPLOY (ON-PREM, PUBLIC, PRIVATE) Private Cloud Solution Containerized apps HA/DR DevTest Red Hat Ansible content available for automation Ansible community content available for automation IBM Power Systems / © 2020 IBM Corporation

38. 38 Existing Analytics/AI on IBM Enterprise Power Systems

39. Use Case: On-Premise consolidation SAP infrastructure • 106 SAP Instance • 25 SAP HANA databases • 128TB Total memory Challenge Customer running SAP ECC and BW on IBM Power with AIX for many years. New IT strategy based on ”Cloud only”. Approach IBM focused on: 1. Total cost of ownership (TCO) within 3 and 5 years 2. Proven technology and customer’s Power experiences. 3. IBM Virtualization, Flexibility and Availability 4. ESG - low carbon footprint Profile: Global company committed to pioneering solutions to the world’s water and climate challenges and improving people’s quality of life. Company aspires to be Climate Positive and aims to halve its own water consumption by 2025. IBM Power Systems / © 2020 IBM Corporation $ < 3X

40. Use Case: Divestiture SAP infrastructure • 16 LPARs | 330TB Tier 1, 290TB Tier 3 • POWER9 infrastructure (85+ Cores) • GTS Managed but not Managed Apps Challenge • Has a HANA and S/4HANA roadmap and are hiring accordingly. • #1 priority is to move out from parent’s datacenters to the cloud in 2020 Approach Migration to IBM Power Virtual Server in 2020 S/4HANA and HANA migration progresses in parallel on a longer timescale Pre-requisites • SAP Netweaver on IBM Power Virtual Server • SAP HANA + S/4HANA on IBM Power Virtual Server Profile: International industrial service company and one of the world's largest oil field services companies. The company provides the oil and gas industry with products and services for oil drilling, formation evaluation, completion, production and reservoir consulting IBM Power Systems / © 2020 IBM Corporation

41. Provision Faster Scale Affordably Maximize Uptime • Provision SAP HANA instances faster with built-in virtualization • Easily make capacity changes • Simplify management consolidating HANA instances • Minimize infrastructure with scale up environment • Granular capacity allocation • Share and optimize CPU allocation • Capacity on Demand • Ranked most reliable server for over a decade1 • Zero impact planned maintenance with LPM • Virtual persistent memory for faster restart and shutdown 1. ITIC 2018 Global Server Hardware, Server OS Reliability Survey Mid-Year Update. The highest uptime of 99.9996% is calculated based on 2.0 minutes/server/annum unplanned downtime of any non-mainframe Linux platforms Your smart choice to run SAP HANA IBM Power Systems 41

42. What is SASViya? A cloud-enabled, in-memory analytics engine – Provides quick, accurate and reliable analytical insights. – Elastic, scalable and fault-tolerant processing addresses the complex analytical challenges of today – Effortlessly scaling for the future. SASViya provides: – Faster processing for huge amounts of data and the most complex analytics, – Including machine learning, deep learning and artificial intelligence – Standardized code base that supports programming in SAS and other languages, like Python, R, Java and Lua. – Support for cloud, on-site or hybrid environments. – It deploys seamlessly to any infrastructure or application ecosystem. Page 42

43. 43Page SAS 9.4 & SAS Viya Similarities/ Differences/ Relationships SAS® 9.4 – Discover insights, manage data analytics approachable. Legacy SAS Viya – Cloud-enabled, in-memory that provides quick, accurate and analytical insights. They compliment each other direct replacement SAS Visual Analytics SAS Report Viewer

44. 44 SAS Visual Analytics SAS high-performance technologies accelerate analytic computations derive value from massive amounts of data

45. Eliminate bottlenecks • Generate insights on time, every time by scaling on demand • Easily allocate precise capacity at the push of a button • Simplify management with co- located workloads in same system • Optimize resource utilization Drive agility Reduce risk • Reduce risk with #1 ranked systems in reliability • Zero impact planned downtime with Live Partition Mobility • Eliminate bottlenecks with the industry leading throughput • 2x I/O and 1.8x memory bandwidth vs compared x86 platforms 1. ITIC 2018 Global Server Hardware, Server OS Reliability Survey Mid-Year Update. The highest uptime of 99.9996% is calculated based on 2.0 minutes/server/annum unplanned downtime of any non-mainframe Linux platforms Accelerate insights from SAS solutions with IBM Power Systems 45Page

46. Best Practice Approach: Think Solutions ! Gaining insights with Machine Learning/Deep Learning requires a flexible end to end solution first approach Focus on solving problems and use cases Data is a pre-requisite ML/DL is just a piece of an overall workflow Infrastructure matters Establish trusted collaborations, partners In Summary

47. Thank You

AI in the Enterprise at Scale

AI in the Enterprise at Scale

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente(20)

Similar a AI in the Enterprise at Scale

Similar a AI in the Enterprise at Scale(20)

Más de Ganesan Narayanasamy

Más de Ganesan Narayanasamy(20)

Último

Último(20)

AI in the Enterprise at Scale