Talk and presentation aim to share evolution journey of Banca Intesa Belgrade from sales force self-initiative to data driven sales model. CRM department was the initiator and leader in this evolution, from developing first Machine Learning algorithms in Python to nowadays when CRM campaigns based on Machine Learning algorithms are dominant source of sales in the bank. Prerequisites, challenges, solutions, milestones and results in this journey are shared highlighting key elements for the successful data driven business evolution from our first-hand experience.

1. Bank evolution to Machine Learning
driven sales model
Miroslav Grbovic
Head of CRM – Banca Intesa Belgrade
Data Science Conference – Europe 2022
Belgrade, Serbia, November 14–18th 2022

2. Company initiated vs. Customer initiated sales
Inbound
Outbound
Customer base:
→ Consumers: ~1.2 million
→ Small Business: ~120,000

3. Machine Learning based Outbound Campaigns
Targeting – selection of clients for campaigns:
→ Binary Classification Machine Learning (ML) algorithms – propensity to purchase models
(predict likelihood that a customer will purchase certain product/service)
→ Separate ML algorithm for each Product/Service campaign
Outbound

4. ML based Outbound Campaigns – Products & Services
Personal Loans
(standard, pre-approved, senior, top-up...)
Overdrafts
External
refinance
(churn)
prevention
Credit Cards Car Loans
Small Business working capital & invest loans
Mobile / Online services

5. Machine/Deep Learning algorithms evaluated, used, in testing

6. Recommender engine:
→ identification of bank clients on public website (http cookie matching)
→ collection of client’s behaviour data on public website
→ Rating for Visited product/service web pages – 1st ML algorithm (Unsupervised ML, Clustering)
→ Rating for NOT Visited product/service web pages – 2nd ML algorithm
→ implementation of commercial campaign based on Recommender
Campaigns based on client behaviour on public Website
http cookie http cookie

7. Machine Learning based Inbound Campaigns
Inbound
Next Best Offer:
→ implemented for entire consumer base
→ almost all bank products/services included
→ clients Segmentation: 1st ML algorithm (Unsupervised ML, Clustering)
→ Targeting: 2nd ML algorithm (Multi-class Classification)
→ periodically updated
Credit
Card
Account Loan Mortgage Deposit
Online
Mobi
€

8. Machine Learning Datasets
CRM system
database Payment Cards transactions
database
Core banking system
database
Digital channels
transactions & behaviour
database
Account
CRM
External sources
(e.g. credit bureau
reports...)
2017 2022
x 8
number of
Features

9. Evolution to CRM Campaigns driven Loan sales
27%
31%
39%
35%
54% 54%
27% 27%
35% 32%
51% 50%
2017 2018 2019 2020 2021 2022
Jan-Oct
Units [%] Volumes [%]
23% 21%
43%
59%
61%
64%
27% 25%
45%
62%
54%
57%
2017 2018 2019 2020 2021 2022
Jan-Oct
Units [%] Volumes [%]
CRM campaigns share in CONSUMER
loan sales [%]
CRM campaigns share in SMALL BUSINESS
loan sales [%]

10. ML algorithms Targeting results in Outbound campaigns
Customer
base
Campaign
Control
Group
7
2.5
8
Prediction Model „Lift"
Personal
Loans
External refinance
(churn) prevention
Car
Loans

11. ML based Outbound Campaigns – Process
Model Training dataset – used to develop (Train/Fit and Verify) ML algorithm:
→ single observation & target period
→ represents customer behaviour with regards to purchases of product/service of interest (avoiding biased periods)
Model Testing datasets – used to Test results of the ML algorithm:
→ multiple observation & target periods
→ encompass periods before and after training period
“Black“
Period:
- during this period (1
month or less)
Prediction Model is
implemented and
predictions are made
Observation
Period:
- is the customer behaviour data
during this X months period used as
an input for Prediction Model
Target Event
Period:
- is this Y months period, during
which model predicts Target event
(e.g. Loan purchase) to happen
1 month 1 month 1 month 1 month 1 month 1 month 1 month 1 month

12. Key CRM roles in ML driven sales model
Operational CRM:
• is 2nd half of CRM (1st one is Data Science team)
• Crucial role in ML campaign lifecycle management – from
implementation to result analysis
• Key CRM interface towards the Sales Channels, responsible
for ensuring quality of CRM campaigns execution on all
channels
Data Science Competence Centre:
• Good enough to start evolution: at least 1 person who
knows ML and willing to share experience and others
willing to learn
• Continuous learning - exploration of new Deep / Machine
Learning algorithms - academic articles, ML competition
results, data science conferences ☺ ...
• Continuous exploration and enrichment of ML datasets with
quality data from existing and new sources
New Business/Use cases:
• Continuous development of new/potential
Business / Use Cases – feasibility studies, testing,
implementation...
CRM
Value Proposition - Talk & Walk:
• ML driven sales - key value propositions:
→ Effectiveness – selling the right products (e.g. NBP)
→ Efficiency – selling the products right (ML Campaigns vs.
Sales Force self initiative)
• Convince with data, not with “opinion“ in all phases of
this evolution journey (campaign results data evaluation) -
from “not believing“ in ML campaigns to demanding too
much from ML based campaigns
• Talk in business language – in communication with other
organizational units and top management

13. Other Stakeholders roles
Marketing – key roles:
• Ensure that more complex marketing
communication - different products,
channels and customer segments
• Shorten marketing materials time to market
- more frequent campaigns
• Contribute with sales „lift“ to overall
campaign results
Sales – key roles:
• Shift own mindset from „we know what our
customers want“ to “customer behaviour is
what customer wants“
• Ensure shift of Sales Force mindset and
activities from self-initiated sales to CRM
campaign driven sales.
• Monitoring Sales Force activities on
campaigns execution (from overall level to
individual sales person level)
IT – key roles:
• Dedicate human resources for continuous
support of Business requirements – enrichment
of databases for ML datasets (ETL from various
sources)
• Validation of data quality and troubleshooting
of ETL processes
• Ensure primarily HW requirements for storage
and processing of excessive amounts of data
CRM
Sales
IT
Marketing
Top
Management
Top Management – key roles:
• Understands, believes and supports evolution
to Data driven Business
• Ensure support of all organisational units and
their engagement in the evolution journey
towards ML driven sales model.
• Evaluate and Assess ML driven sales model
results

14. Key CRM challenges in journey towards ML driven sales
Challenges in Early phase:
→ ML competence
→ Confidence and experience in real/commercial campaigns based od ML
→ Enrichment of ML datasets
→ Explanation how ML works (e.g. “we don’t tell ML algorithm what to do, he tells us…“)
→ Uneven distribution of clients in campaign among sales people
→ Communication of ML results - Important Features vs. Expectations of the audience (previous business rules/logic used)
→ Communication of campaigns results: campaign conversion rate vs. ML model lift vs. Sales lift
Challenges in Mature phase:
→ Demand for:
− too many clients in campaign based on ML
− too many campaigns based on ML prediction models (resource vs. time constrains)
− ML prediction models for target events without enough data for learning (e.g. sales of new products)
→ Data engineering, ML computational power (GPUs)
→ People resources (Data Scientists and Operational CRM)

15. Data driven sales model – In house vs Consultants/Outsource
In-house:
data science is core business – has pivotal role in modern business development
data science team is business organisational unit - resource for informed business decision
data science today is company’s market differentiator, in the future imperative for organizational structure
→ Long-term solution
Consultants / Outsource:
for strategy at the beginning of the journey towards data driven sales
for building competence in new stages of the evolution – sharing new use cases, solution architecture and
data engineering (e.g. new data sources, GPUs)...
→ Short-term solution (project based)

16. Final word
Practices and Results shared are:
→ from REAL business journey
→ acquired in last 5 years
→ from corporation (not startup) in banking industry (not e.g. gaming) ☺
We believe that our example will convince some of you:
→ to believe that you can do it in your company
→ to convince others to believe the same ☺

17. Thank you!