9de BI congres van het BICC-Thomas More: 24 maart 2016
Data analytical platform, new generation. In this presentation Miloud Belkacem shows you how to structure your infrastructure and data sources so they can be available not to just data analysts, but also to the whole organization. It’s an insight into a modern data analytical platform.
3. Amongst the
market’s top 4
platforms
Activity
exclusively web
oriented
Significant monthly
web traffic
Insurance comparison platform
active on the French Market
Who is the Client ?
305-04-16
Startup
4. Client’s Business Model
405-04-16
Site visitors fill-in forms to compare insurances
The company sells the visitors’ forms to partners
Performance of the organisation relies on
Website Traffic
Conversion Rate
5. Project Objectives
505-04-16
Business & Decision Belgium was selected to define & execute the client’s data strategy
Two parallel tracks: Big Data & Analytics and Business Intelligence
The key high level objectives being:
Competitive Edge
Helping the client gain
competitive edge and foster its
market position
Boost Insights
Leverage analytics practices to
better understand what
happened before, what is
happening now and what could
happen in the future
Modernize the Data Platform
Set up a modern data lab based
on top-notch technological
solutions and powerful practices
6. Initial Situation
605-04-16
Platform
Site DatabaseWeb Logs
Data Scientist
Studio Excel file
Management Line
& decision takers
Deliver
Results
IT Department
Relay
Decisions
Implement
Large volumes of Raw data
Slow and heavy analysis
Limited insights and analysis capabilities
Slow cycle to market Ads and Targeting
Slow adaptations of the model
ExtractGenerate
Analyze
7. Challenges
705-04-16
Difficulty to exploit large sized web logs which is key to understanding the behavior of
users
Tedious manual data extraction to perform analysis due to performance and the need
to perform data transformations
Slow Analysis Life-Cycle as:
Data Scientist delivers information manually and irregularly to business
Decision takers assess the analysis results and take decisions
IT builds new recommendation Ads and targeting rules into the platform based on the input
of the management which creates latency
10. Approach Overview
In order to overcome the challenges detailed earlier, B&D has:
Insurance
Comparison
Platform
Business
Intelligence
Big Data
& Analytics
Selected Microsoft as the technology provider
Set up a full-featured Data platform hosted on Microsoft Azure
Define data governance to streamline reporting efforts
Design a BI solution to
Deliver traditional BI outputs (reports, Ad-Hoc, etc.)
Serve as the destination of aggregated Big Data
Set up a data lab on the cloud to
Load and make available large sized web logs and external files
Provide data scientist tools for analysis purposes
Deploy Machine Learning platform and mechanisms Plug & Play
10
11. Azure Machine Learning in a Nutshell
1105-04-16
Machine Learning cloud based component
Provides trained & enriched predictive models
Provides web service based interface to integrate with third party tools
Implement Real-Time targeting and Ads selection
Real time suggestions
Automatic referrals
Churn calculations
Customer segmentations
Next best offer ...
Azure ML
12. Empowered Insight platform
1205-04-16
Insurance Comparison
Platform
Site Database
Web Logs
Data Scientist
Studio
Generate
Business Intelligence
Load
Large volumes
storage
Machine Learning
Consume
Data Lab
Automated Real-Time Targeting and Ads selection
HDInsight
Consume
13. Single Data Platform
1305-04-16
In-House
Sources
Consumption
Platform
Data HubStaging area
ML StudioWebLogs
Dataretrieval
Reference
Files
ManualCnsolidation
External
Sources
Insurance Files
ExtractTransformLoadConsolidate
Mirror
LZ MER
Staging
BigData Stage
(Hive metastore)
TransformMergeLoad
Cube
Process
MER
DirectAccess
16. FUNCTIONAL
Benefits of the solution
16
More relevant recommendations
On-time recommendations
New requests for contact (MER)
Increase conversion rate
1
2
3
4
17. FUNCTIONAL TECHNICAL
Improved data integration (data flow)
Fully automated recommendation system
Usage of state-of-the-art ML technology
Usage of the cloud infrastructure
SocialAnalytics-Ready
Benefits of the solution
16
More relevant recommendations
On-time recommendations
New requests for contact (MER)
Increase conversion rate
1
2
3
4
18. Conclusions
Combine traditional BI & Big Data capabilities
Project hosted in the cloud
Project initiated overnight & first results presented after a few weeks
« Data Lab » solution to validate use cases, then industrialization
Machine Learning capabilities activated
17