SlideShare una empresa de Scribd logo
1 de 15
Descargar para leer sin conexión
1
Finance and Manufacturing Big Data
Use Cases and Solutions
Sanjay Sharma
Principal Architect
August
2013
Impetus Big Data Services
2
Big Data
Platform Implementation
Operations and Visualization
Business Analytics
and Data Science
Solution Architecture, POC
and Production planning
Technology strategy, Use Case
development & Validation
BUSINESS
PROCESS
MANAGEMENT
Assessment
Solution
Modeling
Solution
Analysis
People, Proc
ess, Technol
ogy Impact
Analyze &
Optimize
Objectives &
Strategy
Model
Big Data : Value Drivers
© 2013 Impetus Technologies
Financial World: Key
Challenges
© 2013 Impetus Technologies
Business
Challenges
• Fraud
• Regulatory
Compliances
• Customer Insights
• Risk Handling
Technical
Challenges
• Reduce IT costs
• Do more with
LESS
• Unstructured data
Financial World: Some Use
Cases
© 2013 Impetus Technologies
Fraud Detection and
Analysis
Risk Management
Customer
Insights/Management
Micro targeting
Trade/Payments
Analytics
Long‐Term Storage &
Analytics
New Opportunities Reputational Risk Marketing Campaigns
Financial World: Technical Solution
Challenges
© 2013 Impetus Technologies
• Regulatory Compliances
• Machine Learning based supervised and
unsupervised analytics
Large Data Storage
and Advanced
Analytics
• Mash up structured and unstructured data
• Enrich TX with NLP/Text analytics
Unstructured Data
• Customer level Profiling/
Recommendations
• Transaction/Trade Ticker level analytics
Individual Level
Analytical
Processing
Big Data Solution: Building
Blocks
© 2013 Impetus Technologies
Big Data Use Case Solution:
Example
© 2013 Impetus Technologies
• Major Credit Card Company
• Traditionally Oracle/ DB2 + SAS +
Informatica
• Huge data
– Existing solutions/ infrastructure inadequate to
meet new business requirements and data
growth cost effectively
– Successfully revamping to Big Data Platform
Big Data Use Case Solution:
Example
© 2013 Impetus Technologies
Step 1: Hadoop POC
with Sentiment
Analysis
Step 2: Hadoop
Distribution Selection
Step 3: Production-ized
Recommendation
Engine
Step 4: Production-ized
Negative Fraud
Detection ($100+mlln)
Step 5: …
Hive
Hadoop Platform
HBas
e
Oozie
Mahout
+ NLP
Transactions
Unstructured
Sources
(Email, Surveys, CRM
notes, Social Feeds etc.)
latform
Data
a
a
er
CEP
ata
alytics
g
on
Data Visualization,
Exploration and
Discovery Tools
Tools
Reporting Tools
Analytical Tools
Business
User
Business
Analyst
Advanced
Analytics
User
Sqoop
Flume
MapRed
Unified Analytics Platform
(UAP)
Unstructured Data
Internal Data
External Data
Twitter
Facebook
Voice
to Text
Call Center
Notes
CEP
WEP
Structured Data
In-Database Analytics
Predictive
Data Mining
Segmentation
Data Visualization,
Exploration and
Discovery Tools
S
A
N
D
B
O
X
Domain Specific Tools
Reporting Tools
Analytical Tools
Applications
Business
User
Business
Analyst
Advanced
Analytics
User
BI/ Visualization
Analytics- SAS/R
Real Time Delivery
RDBMS/API
RDBMS/MPP
Informatica
ETL/ELT
L
Advantages
= PB scale
= ETL and Analytical Offloading
= Prescriptive Analytics than Predictive
Manufacturing Domain:
Challenges
© 2013 Impetus Technologies
Business Challenges
• Early preventive
maintenance/repair
• Real time/ near real time
actionable response
• Improve productivity/Margin
Recovery
• Reduce wastes, improve
efficiency
• Improve Yield
• Supply Chain
• Optimize supply chain
Technical Challenges
• High Ingestion Rates
• Sensor/ tool data with sub
second ingestion
requirements
• Millions of read/writes per
second
• Complex log formats
• Semi-structured data
• Huge amount of data
• TB/PB of data storage for
deeper analytics
Manufacturing Domain:
Architectural Building Blocks
© 2013 Impetus Technologies
NoSQL +
Search
Machine
Data
© 2013 Impetus Technologies
Machine
Data
Ingestion
Engine
(Real time +
Batch
components)
Real Time
Processing
Engine
(CEP/Analytics/
Rule Engine)
Real Time
Data
Storage
Engine
(Store +
Indexing/Searc
h)
Business Process Engine
(Business Process+ Rule management)
Kafka/ Storm Storm + Esper Cassandra + Solr
JBoss Drools/jBPM
Manufacturing Domain:
Data Ingestion/ Streaming – Customer
Example
© 2013 Impetus Technologies
NoSQL +
Search
Machine
Data
Manufacturing Domain:
Reference Architecture-Cloud– Customer Example
• Software Solutions and Services Company
• Leader in Innovation led Technology services
• 17 years of customer success, 1500 people across US/
India
• Big Data, Enterprise Mobility, Test and Performance
Engineering, Carrier Grade Large Systems
• Vendor neutral, open source contributor with Big data
accelerator products
© 2013 Impetus Technologies
Impetus Technologies
Q&A
ssharma@impetus.com
bigdata@impetus.com
http://bigdata.impetus.com
© 2013 Impetus Technologies

Más contenido relacionado

La actualidad más candente

End to End Supply Chain Control Tower
End to End Supply Chain Control TowerEnd to End Supply Chain Control Tower
End to End Supply Chain Control TowerDatabricks
 
Making Bank Predictive and Real-Time
Making Bank Predictive and Real-TimeMaking Bank Predictive and Real-Time
Making Bank Predictive and Real-TimeDataWorks Summit
 
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, ClouderaMongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, ClouderaMongoDB
 
Real-time Microservices and In-Memory Data Grids
Real-time Microservices and In-Memory Data GridsReal-time Microservices and In-Memory Data Grids
Real-time Microservices and In-Memory Data GridsAli Hodroj
 
Unlocking data science in the enterprise - with Oracle and Cloudera
Unlocking data science in the enterprise - with Oracle and ClouderaUnlocking data science in the enterprise - with Oracle and Cloudera
Unlocking data science in the enterprise - with Oracle and ClouderaCloudera, Inc.
 
How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...
How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...
How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...Databricks
 
Contact Centers Powered by Esgyn
Contact Centers Powered by EsgynContact Centers Powered by Esgyn
Contact Centers Powered by EsgynRajender K Salgam
 
Strategyzing big data in telco industry
Strategyzing big data in telco industryStrategyzing big data in telco industry
Strategyzing big data in telco industryParviz Iskhakov
 
Office 360 and Spark
Office 360 and Spark Office 360 and Spark
Office 360 and Spark Spark Summit
 
Big Data: It’s all about the Use Cases
Big Data: It’s all about the Use CasesBig Data: It’s all about the Use Cases
Big Data: It’s all about the Use CasesJames Serra
 
Use dependency injection to get Hadoop *out* of your application code
Use dependency injection to get Hadoop *out* of your application codeUse dependency injection to get Hadoop *out* of your application code
Use dependency injection to get Hadoop *out* of your application codeDataWorks Summit
 
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data ArchitectureModernizing to a Cloud Data Architecture
Modernizing to a Cloud Data ArchitectureDatabricks
 
Overview of analytics and big data in practice
Overview of analytics and big data in practiceOverview of analytics and big data in practice
Overview of analytics and big data in practiceVivek Murugesan
 
Scaling Production Machine Learning Pipelines with Databricks
Scaling Production Machine Learning Pipelines with DatabricksScaling Production Machine Learning Pipelines with Databricks
Scaling Production Machine Learning Pipelines with DatabricksDatabricks
 
Implementing and running a secure datalake from the trenches
Implementing and running a secure datalake from the trenches Implementing and running a secure datalake from the trenches
Implementing and running a secure datalake from the trenches DataWorks Summit
 

La actualidad más candente (20)

End to End Supply Chain Control Tower
End to End Supply Chain Control TowerEnd to End Supply Chain Control Tower
End to End Supply Chain Control Tower
 
Big Data Application Architectures - Fraud Detection
Big Data Application Architectures - Fraud DetectionBig Data Application Architectures - Fraud Detection
Big Data Application Architectures - Fraud Detection
 
Making Bank Predictive and Real-Time
Making Bank Predictive and Real-TimeMaking Bank Predictive and Real-Time
Making Bank Predictive and Real-Time
 
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, ClouderaMongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
 
Real-time Microservices and In-Memory Data Grids
Real-time Microservices and In-Memory Data GridsReal-time Microservices and In-Memory Data Grids
Real-time Microservices and In-Memory Data Grids
 
Unlocking data science in the enterprise - with Oracle and Cloudera
Unlocking data science in the enterprise - with Oracle and ClouderaUnlocking data science in the enterprise - with Oracle and Cloudera
Unlocking data science in the enterprise - with Oracle and Cloudera
 
How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...
How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...
How a Media Data Platform Drives Real-time Insights & Analytics using Apache ...
 
Contact Centers Powered by Esgyn
Contact Centers Powered by EsgynContact Centers Powered by Esgyn
Contact Centers Powered by Esgyn
 
Strategyzing big data in telco industry
Strategyzing big data in telco industryStrategyzing big data in telco industry
Strategyzing big data in telco industry
 
The Ecosystem is too damn big
The Ecosystem is too damn big The Ecosystem is too damn big
The Ecosystem is too damn big
 
Office 360 and Spark
Office 360 and Spark Office 360 and Spark
Office 360 and Spark
 
Infochimps + CloudCon: Infinite Monkey Theorem
Infochimps + CloudCon: Infinite Monkey TheoremInfochimps + CloudCon: Infinite Monkey Theorem
Infochimps + CloudCon: Infinite Monkey Theorem
 
Spark meets Smart Meters
Spark meets Smart MetersSpark meets Smart Meters
Spark meets Smart Meters
 
Big Data: It’s all about the Use Cases
Big Data: It’s all about the Use CasesBig Data: It’s all about the Use Cases
Big Data: It’s all about the Use Cases
 
Use dependency injection to get Hadoop *out* of your application code
Use dependency injection to get Hadoop *out* of your application codeUse dependency injection to get Hadoop *out* of your application code
Use dependency injection to get Hadoop *out* of your application code
 
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data ArchitectureModernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
 
Overview of analytics and big data in practice
Overview of analytics and big data in practiceOverview of analytics and big data in practice
Overview of analytics and big data in practice
 
Scaling Production Machine Learning Pipelines with Databricks
Scaling Production Machine Learning Pipelines with DatabricksScaling Production Machine Learning Pipelines with Databricks
Scaling Production Machine Learning Pipelines with Databricks
 
Implementing and running a secure datalake from the trenches
Implementing and running a secure datalake from the trenches Implementing and running a secure datalake from the trenches
Implementing and running a secure datalake from the trenches
 
Using Hadoop for Cognitive Analytics
Using Hadoop for Cognitive AnalyticsUsing Hadoop for Cognitive Analytics
Using Hadoop for Cognitive Analytics
 

Destacado

Hug India Jul10 Hadoop Map
Hug India Jul10 Hadoop MapHug India Jul10 Hadoop Map
Hug India Jul10 Hadoop MapSanjay Sharma
 
How can Hadoop & SAP be integrated
How can Hadoop & SAP be integratedHow can Hadoop & SAP be integrated
How can Hadoop & SAP be integratedDouglas Bernardini
 
Bringing Big Data Analytics to Network Monitoring
Bringing Big Data Analytics to Network MonitoringBringing Big Data Analytics to Network Monitoring
Bringing Big Data Analytics to Network MonitoringSavvius, Inc
 
Smarter commerce overview
Smarter commerce overviewSmarter commerce overview
Smarter commerce overviewHarikrishnan M
 
Digital Transformation and Data Protection in Automotive Industry
Digital Transformation and Data Protection in Automotive IndustryDigital Transformation and Data Protection in Automotive Industry
Digital Transformation and Data Protection in Automotive IndustryÇukur & Yılmaz Law Firm
 
Smarter commerce partner presentation final
Smarter commerce partner presentation finalSmarter commerce partner presentation final
Smarter commerce partner presentation finalBen Andre Heyerdahl
 
Case study - Automotive DMS Connection to Salesforce.com
Case study - Automotive DMS Connection to Salesforce.comCase study - Automotive DMS Connection to Salesforce.com
Case study - Automotive DMS Connection to Salesforce.comRodney Birch
 
Qrious about Insights -- Big Data in the Real World
Qrious about Insights -- Big Data in the Real WorldQrious about Insights -- Big Data in the Real World
Qrious about Insights -- Big Data in the Real WorldGuy K. Kloss
 
Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...
Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...
Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...Hortonworks
 
An Amsterdam perspective on Design Thinking
An Amsterdam perspective on Design ThinkingAn Amsterdam perspective on Design Thinking
An Amsterdam perspective on Design Thinkingicemobile
 
Social media strategies slideshare version
Social media strategies slideshare versionSocial media strategies slideshare version
Social media strategies slideshare versionKella Price
 
Barriers & Pathways to Successful Collaboration
Barriers & Pathways to Successful CollaborationBarriers & Pathways to Successful Collaboration
Barriers & Pathways to Successful Collaborationtwasserman
 
Proyecto final jonathan lara reyes
Proyecto final jonathan lara reyesProyecto final jonathan lara reyes
Proyecto final jonathan lara reyesJONLR
 
BAAL-SIG-LLT-2016-Programme-Booklet_FINAL
BAAL-SIG-LLT-2016-Programme-Booklet_FINALBAAL-SIG-LLT-2016-Programme-Booklet_FINAL
BAAL-SIG-LLT-2016-Programme-Booklet_FINALHelen Lee
 
State of open research data open con
State of open research data   open conState of open research data   open con
State of open research data open conAmye Kenall
 
HAPPYWEEK 194 - 2016.11.14.
HAPPYWEEK 194 - 2016.11.14.HAPPYWEEK 194 - 2016.11.14.
HAPPYWEEK 194 - 2016.11.14.Jiří Černák
 
ابي لا تكن سبب هلاكي
ابي لا تكن سبب هلاكيابي لا تكن سبب هلاكي
ابي لا تكن سبب هلاكيabuomar75
 

Destacado (20)

Hug India Jul10 Hadoop Map
Hug India Jul10 Hadoop MapHug India Jul10 Hadoop Map
Hug India Jul10 Hadoop Map
 
How can Hadoop & SAP be integrated
How can Hadoop & SAP be integratedHow can Hadoop & SAP be integrated
How can Hadoop & SAP be integrated
 
Bringing Big Data Analytics to Network Monitoring
Bringing Big Data Analytics to Network MonitoringBringing Big Data Analytics to Network Monitoring
Bringing Big Data Analytics to Network Monitoring
 
Smarter commerce overview
Smarter commerce overviewSmarter commerce overview
Smarter commerce overview
 
Digital Transformation and Data Protection in Automotive Industry
Digital Transformation and Data Protection in Automotive IndustryDigital Transformation and Data Protection in Automotive Industry
Digital Transformation and Data Protection in Automotive Industry
 
Smarter commerce partner presentation final
Smarter commerce partner presentation finalSmarter commerce partner presentation final
Smarter commerce partner presentation final
 
Case study - Automotive DMS Connection to Salesforce.com
Case study - Automotive DMS Connection to Salesforce.comCase study - Automotive DMS Connection to Salesforce.com
Case study - Automotive DMS Connection to Salesforce.com
 
Qrious about Insights -- Big Data in the Real World
Qrious about Insights -- Big Data in the Real WorldQrious about Insights -- Big Data in the Real World
Qrious about Insights -- Big Data in the Real World
 
Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...
Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...
Accelerating the Value of Big Data Analytics for P&C Insurers with Hortonwork...
 
An Amsterdam perspective on Design Thinking
An Amsterdam perspective on Design ThinkingAn Amsterdam perspective on Design Thinking
An Amsterdam perspective on Design Thinking
 
Social media strategies slideshare version
Social media strategies slideshare versionSocial media strategies slideshare version
Social media strategies slideshare version
 
Three Things...
Three Things...Three Things...
Three Things...
 
Barriers & Pathways to Successful Collaboration
Barriers & Pathways to Successful CollaborationBarriers & Pathways to Successful Collaboration
Barriers & Pathways to Successful Collaboration
 
Control panel
Control panelControl panel
Control panel
 
Proyecto final jonathan lara reyes
Proyecto final jonathan lara reyesProyecto final jonathan lara reyes
Proyecto final jonathan lara reyes
 
BAAL-SIG-LLT-2016-Programme-Booklet_FINAL
BAAL-SIG-LLT-2016-Programme-Booklet_FINALBAAL-SIG-LLT-2016-Programme-Booklet_FINAL
BAAL-SIG-LLT-2016-Programme-Booklet_FINAL
 
State of open research data open con
State of open research data   open conState of open research data   open con
State of open research data open con
 
Whitehead ice intro
Whitehead ice introWhitehead ice intro
Whitehead ice intro
 
HAPPYWEEK 194 - 2016.11.14.
HAPPYWEEK 194 - 2016.11.14.HAPPYWEEK 194 - 2016.11.14.
HAPPYWEEK 194 - 2016.11.14.
 
ابي لا تكن سبب هلاكي
ابي لا تكن سبب هلاكيابي لا تكن سبب هلاكي
ابي لا تكن سبب هلاكي
 

Similar a Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases

Capturing big value in big data
Capturing big value in big data Capturing big value in big data
Capturing big value in big data BSP Media Group
 
20150118 s snet analytics vca
20150118 s snet analytics vca20150118 s snet analytics vca
20150118 s snet analytics vcaVishwanath Ramdas
 
Get One Single View
Get One Single ViewGet One Single View
Get One Single ViewDhiren Gala
 
Analytics Service Framework
Analytics Service Framework Analytics Service Framework
Analytics Service Framework Vishwanath Ramdas
 
The Warranty Data Lake – After, Inc.
The Warranty Data Lake – After, Inc.The Warranty Data Lake – After, Inc.
The Warranty Data Lake – After, Inc.Richard Vermillion
 
Stay Ahead in Telecom Business
Stay Ahead in Telecom BusinessStay Ahead in Telecom Business
Stay Ahead in Telecom BusinessDhiren Gala
 
Implementing Advanced Analytics Platform
Implementing Advanced Analytics PlatformImplementing Advanced Analytics Platform
Implementing Advanced Analytics PlatformArvind Sathi
 
Analyzta_Presentation V1
Analyzta_Presentation V1Analyzta_Presentation V1
Analyzta_Presentation V1Anayzta Team
 
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder AtwalDataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder AtwalHarvinder Atwal
 
Big data project management
Big data project managementBig data project management
Big data project managementIMC Institute
 
Big Data Maturity Model and Governance
Big Data Maturity Model and GovernanceBig Data Maturity Model and Governance
Big Data Maturity Model and GovernanceIMC Institute
 
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix WebinarFuture-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix WebinarImpetus Technologies
 
SG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptxSG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptxssuser57f752
 
ADV Slides: How to Improve Your Analytic Data Architecture Maturity
ADV Slides: How to Improve Your Analytic Data Architecture MaturityADV Slides: How to Improve Your Analytic Data Architecture Maturity
ADV Slides: How to Improve Your Analytic Data Architecture MaturityDATAVERSITY
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedcedrinemadera
 
Bi presentation to bkk
Bi presentation to bkkBi presentation to bkk
Bi presentation to bkkguest4e975e2
 
Entry Points – How to Get Rolling with Big Data Analytics
Entry Points – How to Get Rolling with Big Data AnalyticsEntry Points – How to Get Rolling with Big Data Analytics
Entry Points – How to Get Rolling with Big Data AnalyticsInside Analysis
 

Similar a Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases (20)

Machine Data Analytics
Machine Data AnalyticsMachine Data Analytics
Machine Data Analytics
 
NZS-4555 - IT Analytics Keynote - IT Analytics for the Enterprise
NZS-4555 - IT Analytics Keynote - IT Analytics for the EnterpriseNZS-4555 - IT Analytics Keynote - IT Analytics for the Enterprise
NZS-4555 - IT Analytics Keynote - IT Analytics for the Enterprise
 
Capturing big value in big data
Capturing big value in big data Capturing big value in big data
Capturing big value in big data
 
Get your data analytics strategy right!
Get your data analytics strategy right!Get your data analytics strategy right!
Get your data analytics strategy right!
 
20150118 s snet analytics vca
20150118 s snet analytics vca20150118 s snet analytics vca
20150118 s snet analytics vca
 
Get One Single View
Get One Single ViewGet One Single View
Get One Single View
 
Analytics Service Framework
Analytics Service Framework Analytics Service Framework
Analytics Service Framework
 
The Warranty Data Lake – After, Inc.
The Warranty Data Lake – After, Inc.The Warranty Data Lake – After, Inc.
The Warranty Data Lake – After, Inc.
 
Stay Ahead in Telecom Business
Stay Ahead in Telecom BusinessStay Ahead in Telecom Business
Stay Ahead in Telecom Business
 
Implementing Advanced Analytics Platform
Implementing Advanced Analytics PlatformImplementing Advanced Analytics Platform
Implementing Advanced Analytics Platform
 
Analyzta_Presentation V1
Analyzta_Presentation V1Analyzta_Presentation V1
Analyzta_Presentation V1
 
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder AtwalDataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
 
Big data project management
Big data project managementBig data project management
Big data project management
 
Big Data Maturity Model and Governance
Big Data Maturity Model and GovernanceBig Data Maturity Model and Governance
Big Data Maturity Model and Governance
 
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix WebinarFuture-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
Future-Proof Your Streaming Analytics Architecture- StreamAnalytix Webinar
 
SG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptxSG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptx
 
ADV Slides: How to Improve Your Analytic Data Architecture Maturity
ADV Slides: How to Improve Your Analytic Data Architecture MaturityADV Slides: How to Improve Your Analytic Data Architecture Maturity
ADV Slides: How to Improve Your Analytic Data Architecture Maturity
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-shared
 
Bi presentation to bkk
Bi presentation to bkkBi presentation to bkk
Bi presentation to bkk
 
Entry Points – How to Get Rolling with Big Data Analytics
Entry Points – How to Get Rolling with Big Data AnalyticsEntry Points – How to Get Rolling with Big Data Analytics
Entry Points – How to Get Rolling with Big Data Analytics
 

Último

Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 

Último (20)

Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 

Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases

  • 1. 1 Finance and Manufacturing Big Data Use Cases and Solutions Sanjay Sharma Principal Architect August 2013
  • 2. Impetus Big Data Services 2 Big Data Platform Implementation Operations and Visualization Business Analytics and Data Science Solution Architecture, POC and Production planning Technology strategy, Use Case development & Validation BUSINESS PROCESS MANAGEMENT Assessment Solution Modeling Solution Analysis People, Proc ess, Technol ogy Impact Analyze & Optimize Objectives & Strategy Model
  • 3. Big Data : Value Drivers © 2013 Impetus Technologies
  • 4. Financial World: Key Challenges © 2013 Impetus Technologies Business Challenges • Fraud • Regulatory Compliances • Customer Insights • Risk Handling Technical Challenges • Reduce IT costs • Do more with LESS • Unstructured data
  • 5. Financial World: Some Use Cases © 2013 Impetus Technologies Fraud Detection and Analysis Risk Management Customer Insights/Management Micro targeting Trade/Payments Analytics Long‐Term Storage & Analytics New Opportunities Reputational Risk Marketing Campaigns
  • 6. Financial World: Technical Solution Challenges © 2013 Impetus Technologies • Regulatory Compliances • Machine Learning based supervised and unsupervised analytics Large Data Storage and Advanced Analytics • Mash up structured and unstructured data • Enrich TX with NLP/Text analytics Unstructured Data • Customer level Profiling/ Recommendations • Transaction/Trade Ticker level analytics Individual Level Analytical Processing
  • 7. Big Data Solution: Building Blocks © 2013 Impetus Technologies
  • 8. Big Data Use Case Solution: Example © 2013 Impetus Technologies • Major Credit Card Company • Traditionally Oracle/ DB2 + SAS + Informatica • Huge data – Existing solutions/ infrastructure inadequate to meet new business requirements and data growth cost effectively – Successfully revamping to Big Data Platform
  • 9. Big Data Use Case Solution: Example © 2013 Impetus Technologies Step 1: Hadoop POC with Sentiment Analysis Step 2: Hadoop Distribution Selection Step 3: Production-ized Recommendation Engine Step 4: Production-ized Negative Fraud Detection ($100+mlln) Step 5: … Hive Hadoop Platform HBas e Oozie Mahout + NLP Transactions Unstructured Sources (Email, Surveys, CRM notes, Social Feeds etc.) latform Data a a er CEP ata alytics g on Data Visualization, Exploration and Discovery Tools Tools Reporting Tools Analytical Tools Business User Business Analyst Advanced Analytics User Sqoop Flume MapRed Unified Analytics Platform (UAP) Unstructured Data Internal Data External Data Twitter Facebook Voice to Text Call Center Notes CEP WEP Structured Data In-Database Analytics Predictive Data Mining Segmentation Data Visualization, Exploration and Discovery Tools S A N D B O X Domain Specific Tools Reporting Tools Analytical Tools Applications Business User Business Analyst Advanced Analytics User BI/ Visualization Analytics- SAS/R Real Time Delivery RDBMS/API RDBMS/MPP Informatica ETL/ELT L Advantages = PB scale = ETL and Analytical Offloading = Prescriptive Analytics than Predictive
  • 10. Manufacturing Domain: Challenges © 2013 Impetus Technologies Business Challenges • Early preventive maintenance/repair • Real time/ near real time actionable response • Improve productivity/Margin Recovery • Reduce wastes, improve efficiency • Improve Yield • Supply Chain • Optimize supply chain Technical Challenges • High Ingestion Rates • Sensor/ tool data with sub second ingestion requirements • Millions of read/writes per second • Complex log formats • Semi-structured data • Huge amount of data • TB/PB of data storage for deeper analytics
  • 11. Manufacturing Domain: Architectural Building Blocks © 2013 Impetus Technologies NoSQL + Search Machine Data
  • 12. © 2013 Impetus Technologies Machine Data Ingestion Engine (Real time + Batch components) Real Time Processing Engine (CEP/Analytics/ Rule Engine) Real Time Data Storage Engine (Store + Indexing/Searc h) Business Process Engine (Business Process+ Rule management) Kafka/ Storm Storm + Esper Cassandra + Solr JBoss Drools/jBPM Manufacturing Domain: Data Ingestion/ Streaming – Customer Example
  • 13. © 2013 Impetus Technologies NoSQL + Search Machine Data Manufacturing Domain: Reference Architecture-Cloud– Customer Example
  • 14. • Software Solutions and Services Company • Leader in Innovation led Technology services • 17 years of customer success, 1500 people across US/ India • Big Data, Enterprise Mobility, Test and Performance Engineering, Carrier Grade Large Systems • Vendor neutral, open source contributor with Big data accelerator products © 2013 Impetus Technologies Impetus Technologies

Notas del editor

  1. Fraud Detection:You may not be surprised that Banks and Credit cards are monitoring your spending habits on real-time basis. One of the large credit card issuing bank has implemented fraud detection system that would disable your card if they see suspicious activity based on your past history with spending patters and trends. In addition to the transaction records for authorization and approvals, banks and credit card companies are collecting lot more information from location, your life style, spending patterns. Credit card companies manage huge volume of data from individual Social Security number and income, account balances and employment details, and credit history and transaction history. All this put together helps credit card companies to fight fraud in real-time. Big Data architecture provides that scalability to analyze the incoming transaction against individual history and approve/decline the transaction and alert the account owner.Fraud detection and analysis -­‐ Hadoop provides a scalable method to more easily detect many types of fraud or loss prevention, and perform effective risk management. Hadoop is also being used to develop models that predict future fraud events.Credit Risk – Valuations for credit risk are very computationally intensive. The number of risk factors that need to be modeled are in the 1,000s, with each risk factor taking 1,000s of stochastic paths. This problem is traditionally solved using very large grids of compute nodes. Big Data technologies offer a significantly cheaper alternative, with a huge number of loosely coupled Hadoop nodes that these computations can be offloaded onto.Risk Mitigation -­‐ Hadoop is used to analyze potential market trends and understand future possibilities to mitigate the risk of financial positions, total portfolio assets, and capital returns.Customer insights -­‐ Large financial service providers have adopted Hadoop to improve customer profile analysis to help determine eligibility for equity capital, insurance, mortgage or credit.Customer Segmentation applies in every industry from Banking to Retail to Aviation to Utility and others where they deal with end customer who consume their products and services. In Banking & Financial industry, customer segmentation is a key tool in risk scoring analysis and for sales, promotion and marketing campaigns. In addition to existing information that banks and FIs collect from day to day transactions from customers, they are also buying external data like home values, merchant records from hotels, aviation, retailers, etc. The 360 degree view of customer is still a work in progress and Big Data is enabling filling in the gaps by providing the processing power needed to mine for intelligence from underlying data.The major objectives of segmentation are:Customized product offeringCustomized and priority serviceImprove relationship with profitable customers and cut resources spent on loss making customersBetter offering to new customers based on the intelligence gained from the existing customer segment they belong New product development and bundling as per the customer segment profileCustomer Sentiment AnalysisThe bank can now respond to negative (or positive) brand perception by focusing its communication strategies on particular Internet sites, countering – or backing up – the most outspoken authors on Twitter, boards and blogs. When a company releases a new product that’s causing problems, analyzing comments in social media sites or product review sites can enable it to quickly remediate.Next Best Offer – Banks can use predictive analytics on a combination of data to create a series of targeted offers for customers, and make these offers available in real time at the next point of customer interactionMicro targeting -­‐ Banks have numerous disparate data systems (e.g., loans, mortgages, investments) that need to be aggregated in order to provide an up-­‐to-­‐date view on customer profitability, consistent CRM, and customized product recommendations and offerings. Payments Analytics – Banks can offer value-added services to their merchant customers by analysing payments cleared by them and alerting them of opportunities if they see certain patterns emergeTrade Analysis -­‐ Financial service firms use Hadoop to analyze the daily streams of transaction data in conjunction with unstructured news or social media feeds, or to back-­‐test their trading algorithmsPricing Management – Banks, Capital Markets institutions and insurance firms can use information from both types of data source to price products for individual customers, taking into account risk, capital, cost allocations, transfer pricing and a multitude of additional dimensionsLong-­‐Term Storage & Analytics – Hadoop provides a drastic cost reduction for the long-­‐term storage and analytics of transaction data. New Opportunities – Many large firms are using Hadoop to identify cross-­‐ sell and upsell opportunities by cross referencing sentiment analysis with internal customer profile data.Reputational Risk – All institutions worry about their reputation and getting feedback on newly adopted policies and newly launched products. Most of the ’noise’ around a brand comes from new data sources, and Big Data technologies can be very effective at quickly gathering  this information for analysis.Sales and Marketing CampaignsOn the customer experience side, every time you get closer to delighting your customer by showing that you understand what their real needs are, without blindly sending them emails and credit card offers, it makes the customer view their institution as caring about them and understanding what their needs are.Fraud, AML, Trader and Broker Compliance – All categories of fraud suffer from over-zealous software that generates a high number of false positives. These result in a significant operational problem as they need to be analysed manually. Tuning the software to reduce the number of these alerts results in the opposite problem, with real fraud going unreported. Using both sources of seemingly unrelated data offers the potential to catch fraudulent activities earlier than current methods allow. In the case of internal fraud monitoring, trader and broker compliance software can monitor trading activity coupled with additional data points from sources such as social media, SMS and emails, and create a graph analysis that traditional tools are unable to provide, in order to detect any patterns.Web-­‐scale Analysis -­‐ Hadoop is used to analyze what is being said on the web at large and on social networks in particular. Sentiment can apply to individual companies or products or reflect general customer satisfaction. This has potential to improve marketing to existing customers through better targeting. CrowdsourcingSome of the larger institutions have realized they can use analytics to learn about new lines of business and products, to ask customers what they think, and to get ideas. In a move to expand its utility beyond simply finding better answers to known statistical problems, data-science startup Kaggle is now letting its stable of expert data scientists compete to tell companies how they can improve their businesses using machine learning.Call Center AnalysisFor decades, companies have been analyzing call center data for staffing, agent performance, network management. But with big data age, many new interesting software are being implemented today in attempt to take unstructured voice recordings and analyze them for content and sentiment. Banks are applying text and sentiment analysis to this unstructured data, and looking for patterns and trends. Many banks are integrating this call center data with their transactional data warehouse to reduce customer churn, and drive up-sell, cross-sell, customer monitoring alerts and fraud detection.These are just few of the use cases that I have highlighted here to give a fair idea about how Big Data is being leveraged in this industry. If you have use-case that you are working with, please add it to the comment section.