Analytic Ops is an approach that focuses on continuously improving business outcomes through artificial intelligence by getting AI solutions into production quickly while ensuring regulatory compliance. It addresses typical challenges where only 15% of advanced analytics projects reach production due to underestimating complexity and lack of agility. Analytic Ops prioritizes production, focuses on business value, and allows for iterative changes through agile processes and best practices from software development. This enables the creation of sustainable data products and models in a fraction of the usual time.

1. Artificial Intelligence and AnalyticOps
Continuously Improve Business Outcomes
Nick Switanek, PhD
Marketing Director for Artificial Intelligence
4/18/2018

2. 2
Why Analytic Ops
Analytic Ops Case Study
Key Features of Analytic Ops
Solutions
Agenda
© 2018 Teradata2

3. 3
The Race is On: AI Is Happening in the Enterprise
PayPal WalmartJohn Deere Lowe’s Wells FargoJP Morgan

4. 4
Typical Scenario: The Value Challenge
Value from Advanced Analytics is not realized until you reach production
© 2018 Teradata
Years 15% 6 Months
Time-to-market of a new
data product in typical large-
scale organization. By time
product is finalized, it’s
already obsolete.
Only 15% of Big Data
projects reach production
(Gartner 2015)
Underestimated complexity,
lack of agile iteration with
business requirements.
Without work a static data
product will lose value
completely within a typical
period of 6 months

5. 5
Analytic Ops: Overcoming The Value Challenge
Prioritize production and focus on business value from the start.
© 2018 Teradata
Focus Iterate Sustain
Build for well-chosen
business use case. Build for
production. Use best
practices. Follow agile to
bring typical project from 1
year to 3 months.
Anticipate and integrate
change. Inflexible projects
are doomed to failure.
Allowing for changes in
requirements and
understanding ensures
success.
Best practices from
application/software
development enable the
creation of sustainable data
products quickly.
Think Big. Start Smart. Scale Fast.

6. 6
Analytic Ops: Overcoming The Value Challenge
© 2018 Teradata

7. 7
Value into Production Quickly: Robust and Compliant
Benefits of Analytic Ops
• Reduce time to market
• Improve quality of product
• Reduce maintenance overhead
• Ensure auditability and regulatory compliance
Which departments benefit?
• Business end users: receive a better product faster
• Data systems owners: governance & clearer processes
• Data science teams: optimization & automation
• IT support: reduced operational overhead
Toolkits
Processes
Best
Practices
Accelerators
Contents of Analytic Ops

8. 8 © 2018 Teradata
Analytics Ops Case Study: Fraud Detection

9. 9
Fraud Types: Customer Initiated
© 2018 Teradata

10. 10
Fraud Types: Fraudster Initiated
© 2018 Teradata

11. 11
Challenges for Fraud Detection
© 2018 Teradata
Legacy systems
can’t keep up
on their own

12. 12
Banking Anti-Fraud Solution
© 2018 Teradata
• Real-time data integration
• Security and protocol: follow
existing bank procedures
• Organization and integration
of silos of data
Data Modeling,
Pipeline & Ingestion
• Operationalize insights
quickly and continually
• Enhance collaboration
among data scientists
• Improve intelligibility of
complex DL models
Machine Learning &
Artificial Intelligence
• Monitor and manage many
models running in production
at same time
• Integrate traditional ML and
deep learning into tried &
true rules engines
Model Management
Framework

13. 13
A Framework to Enable Analytic Ops Capabilities
© 2018 Teradata

14. 14
Analytic Ops Enhances Existing Detection Systems
© 2018 Teradata

15. 15
Deep Learning: Results on fraud verification dataset
© 2018 Teradata
• Ensemble model (AUC 0.89)
• ConvNets (AUC 0.95)
• LSTM (AUC 0.90)
• ResNet (AUC 0.94)
>30% increase in detection
>40% reduction in false positives
Massive operational cost reduction

16. 16
Key Requirement: Model Interpretability
© 2018 Teradata
• We have deployed LIME (Locally Interpretable Model-agnostic Explanation) for
customers
– Improves trust in the model results
– Complies with EU’s General Data Protection Regulation (GDPR)

17. 17
Lessons Learned from Analytics Ops at Danske Bank
© 2018 Teradata

18. 18 © 2018 Teradata
Key Features of Analytic Ops

19. 19
Analytic Ops Accelerator
© 2018 Teradata
Platform
Analytic Ops
DS Lab ProductionQA
DeploymentValidation
DS Workbench
Training
Feature Extraction
Model Creation
Feature Generation
Visualization / Explanation Orchestration
Execution
Model Management, Data Set Management, Versioning
Incremental training
Check-In
…
End-to-end framework to facilitate the training, deployment, and management of traditional
and deep learning models at scale

20. 20
Analytic Ops enhances Model Training
Simplifies and automates
continuous model training and
iterative innovation
• Configurable training options
(e.g. optimization algorithm,
learning rate)
• Visibility into model vitals
© 2018 Teradata

21. 21
Model Deployment
Automates production
deployment of models
• Supports hybrid infrastructure,
across cloud and on-premises
• Supports Champion/Challenger
model management:
– Winning model pushed to
production with 1 click
© 2018 Teradata

22. 22
Facilitates collaboration on
analytic assets
• Version control for both models
and data sets
• Change approval logging
• Version roll back
© 2018 Teradata

23. 23
Value into Production Quickly: Robust and Compliant
Benefits of Analytic Ops
• Reduce time to market
• Improve quality of product
• Reduce maintenance overhead
• Ensure auditability and regulatory compliance
Which departments benefit?
• Business end users: receive a better product faster
• Data systems owners: governance & clearer processes
• Data science teams: optimization & automation
• IT support: reduced operational overhead
Toolkits
Processes
Best
Practices
Accelerators
Contents of Analytic Ops

24. 2424 © 2018 Teradata