SlideShare una empresa de Scribd logo
1 de 21
Data Intelligence
How the Amalgamation of Data, Science, and Technology is
Changing the Way We Do Business
January 22, 2017
Presented by:
Joe Caserta
Caserta Timeline
LaunchedBig Data practice
Co-author, with Ralph Kimball, The Data
Warehouse ETL Toolkit (Wiley)
Data Analysis, Data Warehousing andBusiness
Intelligence since 1996
Began consultingdatabase programing anddata
modeling 25+ years hands-on experience building database
solutions
Founded CasertaConcepts, LLC in NYC
Web log analytics solution published in Intelligent
Enterprise magazine
Launched Data Science, Data Interaction andCloud
practices
Laser focus on extending Data Analytics with Big Data
solutions
1986
2004
1996
2009
2001
2013
2012
2016
Dedicated to Data GovernanceTechniques onBig
Data (Innovation)
Awarded Top 20 Big Data Companies
Top 20 Most Powerful
Big Data consulting firms
Launched Big DataWarehousing (BDW) Meetup
NYC:4,000 Members
2017
Top 20 Most Admired Tech Leaders in Business
Established best practicesfor big dataecosystem
implementations
Caserta InnovationLab invents Blockchain, AI,AR
Solutions
About Caserta
Data Intelligence and Strategic Consulting
Data Lakes, Data Laboratories, Data Warehouses
Award-winning company for Data Innovation
Data Science, Machine Learning, Artificial Intelligence
Internationally recognized work force
Best Practices, Authors, Educators, Mentors
Strategy, Governance, Architecture, Implementation
Our Clients
Retail/eCommerce
& Manufacturing
Finance, Healthcare,
Energy & Insurance
Digital Media/AdTech
Education & Services
Evolution of Analytics
What
happened?
Why did it
happen?
What will
happen?
How can we
make It happen?
Data Analytics Sophistication
BusinessValue
Source: Gartner
How to interact with
the customer?
Reports  Correlations  Predictions  Recommendation s  Artificial Intelligence
Why is Data so Important?
1500s
Prin ng Press
1840s
Penny Post
1850s
Telegraph
1850s
Rural Free Post
1890s
Telephone
1900s
Radio
1950s
TV
1970s
PCs
1980s
Internet
1990s
Web
2000s
Social Media, Mobile, Big Data, Cloud
98,000+ Tweets
695,000 Status Updates
11 Million instant messages
698,445 Google Searches
168 million+ emails sent
1,829 TB of data created
217 new mobile web
users
Every 60 Seconds
Data Analytics is your Differentiator
Acquiring, analyzing and acting on data with a focus on speed to action
Artificial Intelligence
 “AI is one of the most important things that humanity is
working on. It’s more profound than electricity or fire”
- Sundar Pichai, CEO, Google
The Customer Journey
PR
Radio
TV
Print
Outdoor
Word of Mouth
Direct Mail
Customer Service
Physical Touchpoints
Digital Touchpoints
Search
Paid Content
email
Website/
Landing Pages
Social Media
Community
Chat
Social Media
Call Center
Offers
Mailings
Survey
Loyalty Programs
email
Agents
Partners
Ads
Website
Mobile
3rd Party Sites
Offers
Web self-service
Learning the Path-to-Purchase
Attribution
Type
Comments
Single Touch Rules-Based Statistically Driven
Assign the credit to the
first or last exposure
Assign the credit to each
interaction based on
business rules
Assign the credit to
interactions based on
data-driven model
Ad-Click Mailing MailingE-mail E-mailAd-Click Ad-Click
100% 33% 33% 33% 27% 49% 24%
- Last touch only
- Ignores bulk of
customer journey
- Undervalues other
interactions and
influencers
- Subjective
- Assigns arbitrary values
to each interaction
- Lacks analytics rigor to
determine weights
 Looks at full behavior
patterns
 Consider all touch points
 Can apply different models
for best results
 Use data to find
correlations between touch
points (winning
combinations)
Data Science in Practice
Source: https://www.collaberatact.com/data-science-stay/
Data Science for the Enterprise
CRISP-DM: Cross Industry Standard Process for Data Mining
1. Business Understanding
• Solve a single business problem
2. Data Understanding
• Discovery
• Data Munging
• Cleansing Requirements
3. Data Preparation
• ETL
4. Modeling
• Evaluate various models
• Iterative experimentation
5. Evaluation
• Does the model achieve business objectives?
6. Deployment
• PMML; application integration; data platform; Excel
Business
Understanding
Data
Understanding
Data
Preparation
Modeling
Evaluation
Deployment
Data
Governing Data Innovation
S3
Ingest Storage ETL Presentation VisualizationData Sources
• OPRA
• Equifax
• CDS
• Moody’s
• BlackBox
Relational Datasets
• Barclay
• Eureka
• Hedge Fund
Intelligence
• Hedge Fund
Research
• Lipper
• Morningstar
• MF Holdings
• BD/ ADV
Flat File Datasets
S/ FTP
Push
Kinesis
• CAT
Landing
Data Lake
(Tier 1)
Data Lake
(Tier 2)
Data Science
(Ephemeral)
Redshift
Spark
(Streaming*
/ Batch)
Lambda
Data Science
• Python
• SQL
• Scala
• Predic ve
Analy cs
• Text Analy cs
• Business
Intelligence
Structured
Data
Redshift
Metadata
Repository
• Data
Marketplace
• Clean
• Match
• Derive
• Aggregate
• Mllib
• CoreNLP
• Prepare
• Deliver
Streaming Data Sets
Data Analytics Innovation Ecosystem
SAP
Oracle
Financials
Marketing
Relational DBs
Salesforce
Workday
RESTful APIs
Cloud DBs
Bloomberg
Capital IQ
FactSet
Quandl
Alternative Data
Web logs
IoT
Streaming Data
Data Quality & Monitoring
• Build a robust data quality subsystem:
• Metadata and error event facts
• Orchestration
• Based on Data Warehouse ETL Toolkit
• Each error instance of each data
quality check is captured
• Implemented as sub-system after
ingestion
• Each fact stores unique identifier of the
defective source row
Change Management
Global economics
Intensity of competition
Reduce costs
Move to cross-functional teams
New executive leadership
Social trends and changes
Speed of technical change
Period of time in present role
Status & perks of office/dept under threat
No apparent reasons for proposed changes
Lack of understanding of proposed changes
Fear of inability to cope with new technology
Concern over job security
Forces for Change Forces ResistingChange
Status Quo
http://www.change-management-coach.com/force-field-analysis.html
Agile Data Organization
Cloud Platform Components
Cloud Component AWS Google Microsoft
Scalable distributed storage S3 GCS Azure Storage
Pluggable fit-for-purpose processing EMR DataProc HDInsight
Compute Services EC2 GCE VMs
Consistent extensible framework Spark Spark Spark
Dimensional MPP Data Warehouse
Redshift/
Snowflake
BigQuery
Azure SQL Data
Warehouse
Data Streaming Kenesis PubSub Azure Stream
Common Interface Jupyter DataLab Azure Notebook
Machine Learning SageMaker TensorFlow Azure ML
Customer Journey Dashboard
What the Future Holds
• DevOps for Analytics
• Search-Based BI (NLP)
• Artificial Intelligence (AI)
• Virtual Reality BI (VR)
• Virtual Assistant BI (Voice)
• Reporting/Predictions Converge
• Citizen Data Scientists Emerge
Thank You
joe@caserta.com
@Joe_Caserta
Joe Caserta
President, Caserta Concepts

Más contenido relacionado

La actualidad más candente

Moving Past Infrastructure Limitations
Moving Past Infrastructure LimitationsMoving Past Infrastructure Limitations
Moving Past Infrastructure Limitations
Caserta
 
An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...
An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...
An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...
Neo4j
 
DataOps: Nine steps to transform your data science impact Strata London May 18
DataOps: Nine steps to transform your data science impact  Strata London May 18DataOps: Nine steps to transform your data science impact  Strata London May 18
DataOps: Nine steps to transform your data science impact Strata London May 18
Harvinder Atwal
 
Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...
Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...
Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...
DATAVERSITY
 

La actualidad más candente (20)

The Emerging Role of the Data Lake
The Emerging Role of the Data LakeThe Emerging Role of the Data Lake
The Emerging Role of the Data Lake
 
Intro to Data Science on Hadoop
Intro to Data Science on HadoopIntro to Data Science on Hadoop
Intro to Data Science on Hadoop
 
You're the New CDO, Now What?
You're the New CDO, Now What?You're the New CDO, Now What?
You're the New CDO, Now What?
 
Big Data Analytics on the Cloud
Big Data Analytics on the CloudBig Data Analytics on the Cloud
Big Data Analytics on the Cloud
 
Moving Past Infrastructure Limitations
Moving Past Infrastructure LimitationsMoving Past Infrastructure Limitations
Moving Past Infrastructure Limitations
 
An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...
An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...
An Overview of the Neo4j Cloud Strategy and the Future of Graph Databases in ...
 
DataOps: Nine steps to transform your data science impact Strata London May 18
DataOps: Nine steps to transform your data science impact  Strata London May 18DataOps: Nine steps to transform your data science impact  Strata London May 18
DataOps: Nine steps to transform your data science impact Strata London May 18
 
Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...
Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...
Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...
 
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
Agile Leadership: Guiding DataOps Teams Through Rapid Change and UncertaintyAgile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
 
Focus on Your Analysis, Not Your SQL Code
Focus on Your Analysis, Not Your SQL CodeFocus on Your Analysis, Not Your SQL Code
Focus on Your Analysis, Not Your SQL Code
 
Reveal the Intelligence in your Data with Talend Data Fabric
Reveal the Intelligence in your Data with Talend Data FabricReveal the Intelligence in your Data with Talend Data Fabric
Reveal the Intelligence in your Data with Talend Data Fabric
 
What Data Do You Have and Where is It?
What Data Do You Have and Where is It? What Data Do You Have and Where is It?
What Data Do You Have and Where is It?
 
Balancing Data Governance and Innovation
Balancing Data Governance and InnovationBalancing Data Governance and Innovation
Balancing Data Governance and Innovation
 
How to Consume Your Data for AI
How to Consume Your Data for AIHow to Consume Your Data for AI
How to Consume Your Data for AI
 
Using Machine Learning to Understand and Predict Marketing ROI
Using Machine Learning to Understand and Predict Marketing ROIUsing Machine Learning to Understand and Predict Marketing ROI
Using Machine Learning to Understand and Predict Marketing ROI
 
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
 
A Year in Review - Building a Comprehensive Data Management Program
A Year in Review - Building a Comprehensive Data Management ProgramA Year in Review - Building a Comprehensive Data Management Program
A Year in Review - Building a Comprehensive Data Management Program
 
Maximize the Value of Your Data: Neo4j Graph Data Platform
Maximize the Value of Your Data: Neo4j Graph Data PlatformMaximize the Value of Your Data: Neo4j Graph Data Platform
Maximize the Value of Your Data: Neo4j Graph Data Platform
 
Reinventing the Modern Information Pipeline: Paxata and MapR
Reinventing the Modern Information Pipeline: Paxata and MapRReinventing the Modern Information Pipeline: Paxata and MapR
Reinventing the Modern Information Pipeline: Paxata and MapR
 
Big Data Analytics Architecture PowerPoint Presentation Slides
Big Data Analytics Architecture PowerPoint Presentation SlidesBig Data Analytics Architecture PowerPoint Presentation Slides
Big Data Analytics Architecture PowerPoint Presentation Slides
 

Similar a Data Intelligence: How the Amalgamation of Data, Science, and Technology is Changing the Way We Do Business

3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию
antishmanti
 
Day 2 aziz apj aziz_big_datakeynote_press
Day 2 aziz apj aziz_big_datakeynote_pressDay 2 aziz apj aziz_big_datakeynote_press
Day 2 aziz apj aziz_big_datakeynote_press
IntelAPAC
 

Similar a Data Intelligence: How the Amalgamation of Data, Science, and Technology is Changing the Way We Do Business (20)

Delivering Value Through Business Analytics
Delivering Value Through Business AnalyticsDelivering Value Through Business Analytics
Delivering Value Through Business Analytics
 
Self-Service Analytics Framework - Connected Brains 2018
Self-Service Analytics Framework - Connected Brains 2018Self-Service Analytics Framework - Connected Brains 2018
Self-Service Analytics Framework - Connected Brains 2018
 
Why Everything You Know About bigdata Is A Lie
Why Everything You Know About bigdata Is A LieWhy Everything You Know About bigdata Is A Lie
Why Everything You Know About bigdata Is A Lie
 
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder AtwalDataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
 
Webinar: Transforming Customer Experience Through an Always-On Data Platform
Webinar: Transforming Customer Experience Through an Always-On Data PlatformWebinar: Transforming Customer Experience Through an Always-On Data Platform
Webinar: Transforming Customer Experience Through an Always-On Data Platform
 
Self-service analytics @ Leaseplan Digital: from business intelligence to int...
Self-service analytics @ Leaseplan Digital: from business intelligence to int...Self-service analytics @ Leaseplan Digital: from business intelligence to int...
Self-service analytics @ Leaseplan Digital: from business intelligence to int...
 
Analytics Service Framework
Analytics Service Framework Analytics Service Framework
Analytics Service Framework
 
A Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyA Winning Strategy for the Digital Economy
A Winning Strategy for the Digital Economy
 
3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию
 
uae views on big data
  uae views on  big data  uae views on  big data
uae views on big data
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-shared
 
Accelerating Data-Driven Enterprise Transformation in Banking, Financial Serv...
Accelerating Data-Driven Enterprise Transformation in Banking, Financial Serv...Accelerating Data-Driven Enterprise Transformation in Banking, Financial Serv...
Accelerating Data-Driven Enterprise Transformation in Banking, Financial Serv...
 
IBM Solutions Connect 2013 - Getting started with Big Data
IBM Solutions Connect 2013 - Getting started with Big DataIBM Solutions Connect 2013 - Getting started with Big Data
IBM Solutions Connect 2013 - Getting started with Big Data
 
Knowledge Graphs Webinar- 11/7/2017
Knowledge Graphs Webinar- 11/7/2017Knowledge Graphs Webinar- 11/7/2017
Knowledge Graphs Webinar- 11/7/2017
 
Day 2 aziz apj aziz_big_datakeynote_press
Day 2 aziz apj aziz_big_datakeynote_pressDay 2 aziz apj aziz_big_datakeynote_press
Day 2 aziz apj aziz_big_datakeynote_press
 
Brainstorm:KC 2016
Brainstorm:KC 2016Brainstorm:KC 2016
Brainstorm:KC 2016
 
Turning Big Data to Business Advantage
Turning Big Data to Business AdvantageTurning Big Data to Business Advantage
Turning Big Data to Business Advantage
 
BDW Chicago 2016 - Ramu Kalvakuntla, Sr. Principal - Technical - Big Data Pra...
BDW Chicago 2016 - Ramu Kalvakuntla, Sr. Principal - Technical - Big Data Pra...BDW Chicago 2016 - Ramu Kalvakuntla, Sr. Principal - Technical - Big Data Pra...
BDW Chicago 2016 - Ramu Kalvakuntla, Sr. Principal - Technical - Big Data Pra...
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
BAR360 open data platform presentation at DAMA, Sydney
BAR360 open data platform presentation at DAMA, SydneyBAR360 open data platform presentation at DAMA, Sydney
BAR360 open data platform presentation at DAMA, Sydney
 

Más de Caserta

Not Your Father's Database by Databricks
Not Your Father's Database by DatabricksNot Your Father's Database by Databricks
Not Your Father's Database by Databricks
Caserta
 
Mastering Customer Data on Apache Spark
Mastering Customer Data on Apache SparkMastering Customer Data on Apache Spark
Mastering Customer Data on Apache Spark
Caserta
 

Más de Caserta (15)

Introduction to Data Science (Data Summit, 2017)
Introduction to Data Science (Data Summit, 2017)Introduction to Data Science (Data Summit, 2017)
Introduction to Data Science (Data Summit, 2017)
 
Looker Data Modeling in the Age of Cloud - BDW Meetup May 2, 2017
Looker Data Modeling in the Age of Cloud - BDW Meetup May 2, 2017Looker Data Modeling in the Age of Cloud - BDW Meetup May 2, 2017
Looker Data Modeling in the Age of Cloud - BDW Meetup May 2, 2017
 
The Data Lake - Balancing Data Governance and Innovation
The Data Lake - Balancing Data Governance and Innovation The Data Lake - Balancing Data Governance and Innovation
The Data Lake - Balancing Data Governance and Innovation
 
Making Big Data Easy for Everyone
Making Big Data Easy for EveryoneMaking Big Data Easy for Everyone
Making Big Data Easy for Everyone
 
Benefits of the Azure Cloud
Benefits of the Azure CloudBenefits of the Azure Cloud
Benefits of the Azure Cloud
 
Not Your Father's Database by Databricks
Not Your Father's Database by DatabricksNot Your Father's Database by Databricks
Not Your Father's Database by Databricks
 
Mastering Customer Data on Apache Spark
Mastering Customer Data on Apache SparkMastering Customer Data on Apache Spark
Mastering Customer Data on Apache Spark
 
Balancing Data Governance and Innovation
Balancing Data Governance and InnovationBalancing Data Governance and Innovation
Balancing Data Governance and Innovation
 
Introducing Kudu, Big Data Warehousing Meetup
Introducing Kudu, Big Data Warehousing MeetupIntroducing Kudu, Big Data Warehousing Meetup
Introducing Kudu, Big Data Warehousing Meetup
 
Setting Up the Data Lake
Setting Up the Data LakeSetting Up the Data Lake
Setting Up the Data Lake
 
Incorporating the Data Lake into Your Analytic Architecture
Incorporating the Data Lake into Your Analytic ArchitectureIncorporating the Data Lake into Your Analytic Architecture
Incorporating the Data Lake into Your Analytic Architecture
 
Real Time Big Data Processing on AWS
Real Time Big Data Processing on AWSReal Time Big Data Processing on AWS
Real Time Big Data Processing on AWS
 
Big Data: Setting Up the Big Data Lake
Big Data: Setting Up the Big Data LakeBig Data: Setting Up the Big Data Lake
Big Data: Setting Up the Big Data Lake
 
Big Data Warehousing Meetup: Dimensional Modeling Still Matters!!!
Big Data Warehousing Meetup: Dimensional Modeling Still Matters!!!Big Data Warehousing Meetup: Dimensional Modeling Still Matters!!!
Big Data Warehousing Meetup: Dimensional Modeling Still Matters!!!
 
Big Data's Impact on the Enterprise
Big Data's Impact on the EnterpriseBig Data's Impact on the Enterprise
Big Data's Impact on the Enterprise
 

Último

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 

Data Intelligence: How the Amalgamation of Data, Science, and Technology is Changing the Way We Do Business

  • 1. Data Intelligence How the Amalgamation of Data, Science, and Technology is Changing the Way We Do Business January 22, 2017 Presented by: Joe Caserta
  • 2. Caserta Timeline LaunchedBig Data practice Co-author, with Ralph Kimball, The Data Warehouse ETL Toolkit (Wiley) Data Analysis, Data Warehousing andBusiness Intelligence since 1996 Began consultingdatabase programing anddata modeling 25+ years hands-on experience building database solutions Founded CasertaConcepts, LLC in NYC Web log analytics solution published in Intelligent Enterprise magazine Launched Data Science, Data Interaction andCloud practices Laser focus on extending Data Analytics with Big Data solutions 1986 2004 1996 2009 2001 2013 2012 2016 Dedicated to Data GovernanceTechniques onBig Data (Innovation) Awarded Top 20 Big Data Companies Top 20 Most Powerful Big Data consulting firms Launched Big DataWarehousing (BDW) Meetup NYC:4,000 Members 2017 Top 20 Most Admired Tech Leaders in Business Established best practicesfor big dataecosystem implementations Caserta InnovationLab invents Blockchain, AI,AR Solutions
  • 3. About Caserta Data Intelligence and Strategic Consulting Data Lakes, Data Laboratories, Data Warehouses Award-winning company for Data Innovation Data Science, Machine Learning, Artificial Intelligence Internationally recognized work force Best Practices, Authors, Educators, Mentors Strategy, Governance, Architecture, Implementation
  • 4. Our Clients Retail/eCommerce & Manufacturing Finance, Healthcare, Energy & Insurance Digital Media/AdTech Education & Services
  • 5. Evolution of Analytics What happened? Why did it happen? What will happen? How can we make It happen? Data Analytics Sophistication BusinessValue Source: Gartner How to interact with the customer? Reports  Correlations  Predictions  Recommendation s  Artificial Intelligence
  • 6. Why is Data so Important? 1500s Prin ng Press 1840s Penny Post 1850s Telegraph 1850s Rural Free Post 1890s Telephone 1900s Radio 1950s TV 1970s PCs 1980s Internet 1990s Web 2000s Social Media, Mobile, Big Data, Cloud 98,000+ Tweets 695,000 Status Updates 11 Million instant messages 698,445 Google Searches 168 million+ emails sent 1,829 TB of data created 217 new mobile web users Every 60 Seconds
  • 7. Data Analytics is your Differentiator Acquiring, analyzing and acting on data with a focus on speed to action
  • 8. Artificial Intelligence  “AI is one of the most important things that humanity is working on. It’s more profound than electricity or fire” - Sundar Pichai, CEO, Google
  • 9. The Customer Journey PR Radio TV Print Outdoor Word of Mouth Direct Mail Customer Service Physical Touchpoints Digital Touchpoints Search Paid Content email Website/ Landing Pages Social Media Community Chat Social Media Call Center Offers Mailings Survey Loyalty Programs email Agents Partners Ads Website Mobile 3rd Party Sites Offers Web self-service
  • 10. Learning the Path-to-Purchase Attribution Type Comments Single Touch Rules-Based Statistically Driven Assign the credit to the first or last exposure Assign the credit to each interaction based on business rules Assign the credit to interactions based on data-driven model Ad-Click Mailing MailingE-mail E-mailAd-Click Ad-Click 100% 33% 33% 33% 27% 49% 24% - Last touch only - Ignores bulk of customer journey - Undervalues other interactions and influencers - Subjective - Assigns arbitrary values to each interaction - Lacks analytics rigor to determine weights  Looks at full behavior patterns  Consider all touch points  Can apply different models for best results  Use data to find correlations between touch points (winning combinations)
  • 11. Data Science in Practice Source: https://www.collaberatact.com/data-science-stay/
  • 12. Data Science for the Enterprise CRISP-DM: Cross Industry Standard Process for Data Mining 1. Business Understanding • Solve a single business problem 2. Data Understanding • Discovery • Data Munging • Cleansing Requirements 3. Data Preparation • ETL 4. Modeling • Evaluate various models • Iterative experimentation 5. Evaluation • Does the model achieve business objectives? 6. Deployment • PMML; application integration; data platform; Excel Business Understanding Data Understanding Data Preparation Modeling Evaluation Deployment Data
  • 14. S3 Ingest Storage ETL Presentation VisualizationData Sources • OPRA • Equifax • CDS • Moody’s • BlackBox Relational Datasets • Barclay • Eureka • Hedge Fund Intelligence • Hedge Fund Research • Lipper • Morningstar • MF Holdings • BD/ ADV Flat File Datasets S/ FTP Push Kinesis • CAT Landing Data Lake (Tier 1) Data Lake (Tier 2) Data Science (Ephemeral) Redshift Spark (Streaming* / Batch) Lambda Data Science • Python • SQL • Scala • Predic ve Analy cs • Text Analy cs • Business Intelligence Structured Data Redshift Metadata Repository • Data Marketplace • Clean • Match • Derive • Aggregate • Mllib • CoreNLP • Prepare • Deliver Streaming Data Sets Data Analytics Innovation Ecosystem SAP Oracle Financials Marketing Relational DBs Salesforce Workday RESTful APIs Cloud DBs Bloomberg Capital IQ FactSet Quandl Alternative Data Web logs IoT Streaming Data
  • 15. Data Quality & Monitoring • Build a robust data quality subsystem: • Metadata and error event facts • Orchestration • Based on Data Warehouse ETL Toolkit • Each error instance of each data quality check is captured • Implemented as sub-system after ingestion • Each fact stores unique identifier of the defective source row
  • 16. Change Management Global economics Intensity of competition Reduce costs Move to cross-functional teams New executive leadership Social trends and changes Speed of technical change Period of time in present role Status & perks of office/dept under threat No apparent reasons for proposed changes Lack of understanding of proposed changes Fear of inability to cope with new technology Concern over job security Forces for Change Forces ResistingChange Status Quo http://www.change-management-coach.com/force-field-analysis.html
  • 18. Cloud Platform Components Cloud Component AWS Google Microsoft Scalable distributed storage S3 GCS Azure Storage Pluggable fit-for-purpose processing EMR DataProc HDInsight Compute Services EC2 GCE VMs Consistent extensible framework Spark Spark Spark Dimensional MPP Data Warehouse Redshift/ Snowflake BigQuery Azure SQL Data Warehouse Data Streaming Kenesis PubSub Azure Stream Common Interface Jupyter DataLab Azure Notebook Machine Learning SageMaker TensorFlow Azure ML
  • 20. What the Future Holds • DevOps for Analytics • Search-Based BI (NLP) • Artificial Intelligence (AI) • Virtual Reality BI (VR) • Virtual Assistant BI (Voice) • Reporting/Predictions Converge • Citizen Data Scientists Emerge

Notas del editor

  1. Reports 70s – 90s = 20 years Correlations (DW) 90s- 2000 Predictions (data mining) 2005 Recommendations (ML) 2007 Artificial Intelligence - 2017
  2. In August 2001, robots beat humans in a simulated financial trading competition. AI has reduced fraud and financial crimes by monitoring behavioral patterns of users for abnormal changes or anomalies.
  3. Teaching half-day class on this at the Data Summit in Boston in May