1615 track1 schleicher

Confidential & Proprietary • © 2017 National Consumer Panel
Thomas Schleicher, PhD
National Consumer Panel
Combining Inferential Statistics with Predictive
Modeling to Evaluate Changes in Your Business
June 21, 2017

6/21/2017
Confidential & Proprietary © 2017 National Consumer Panel
How not to do analytics…

6/21/2017
…vs. how to: The CRISP-DM Process Model
Do we understand the
problem?
How do we integrate
the solution into
existing systems?
What can we learn
about the available
data?
How must we integrate, clean,
and transform the data for
modeling?
What modeling algorithms
will be used?
Modeling
How do we know when the
model is good?

6/21/2017
Overview
Key Objectives
National Consumer Panel (NCP) Background
Data, Analytics and KPI’s
Stakeholder Consensus
Enhancing your Existing Process
Demonstrated Impact
Conclusions and Recommendations

6/21/2017
Key Objectives

6/21/2017
Key Objectives
How NCP’s experience may help to optimize the analytics
process in your organization
• How to translate and leverage knowledge from subject matter experts into
the predictive analytic process
• How to integrate inferential statistics into your predictive analytics data
system
How to improve business decisions (and operations) by
adapting to foreseen and unforeseen changes in your business

6/21/2017
National Consumer Panel
(NCP) Background

6/21/2017
National Consumer Panel (NCP)
Nielsen/IRi joint venture (2010)
• Alliance between head-to-head competitors
• Longitudinal household consumer panel

6/21/2017
Table 2: Interim Projections: Percent Distribution of Projected Population and Population Change for Regions and Divisions: 2000 to 2030
Region and Division
United States 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0
.Northeast 19.0 18.1 17.0 15.9 8.0 5.0 1.9 5.0
..New England 4.9 4.8 4.6 4.3 3.0 2.1 1.1 2.1
..Middle Atlantic 14.1 13.3 12.5 11.6 5.0 2.9 0.8 2.9
.Midwest 22.9 21.8 20.7 19.4 10.9 7.7 3.8 7.4
..East North Central 16.0 15.2 14.4 13.4 6.9 4.3 1.5 4.2
..West North Central 6.8 6.6 6.3 6.0 4.0 3.3 2.2 3.2
.South 35.6 36.8 38.0 39.4 48.5 52.1 56.5 52.4
..South Atlantic 18.4 19.4 20.4 21.5 29.2 32.2 34.7 32.0
..East South Central 6.0 5.8 5.7 5.5 3.8 3.4 3.3 3.5
..West South Central 11.2 11.6 12.0 12.5 15.6 16.5 18.4 16.8
.West 22.5 23.4 24.3 25.3 32.6 35.2 37.8 35.2
..Mountain 6.5 7.0 7.6 8.2 13.0 14.2 15.7 14.3
..Pacific 16.0 16.3 16.7 17.1 19.7 21.0 22.1 20.9
Footnote:
U.S. Census Bureau, Population Division, Interim State Population Projections, 2005.
Internet Release Date: April 21, 2005
Representative Sample of US Households (Geo’s)

6/21/2017
Representative Sample of US Households (Demo’s)
Table H1. Households by Type and Tenure of Householder for Selected Characteristics: 2015
(Numbers in thousands)
Total
Married
Couple
Male
Householder
Female
Householder Total
Male
Householder
Female
Householder
ALL HOUSEHOLDS 124,587 81,716 60,010 6,162 15,544 42,871 20,143 22,728
.SIZE OF HOUSEHOLD
..One member 34,866 - - - - 34,866 15,513 19,353
..Two members 41,881 35,323 26,847 2,541 5,935 6,558 3,673 2,885
..Three members 19,309 18,338 11,619 1,859 4,860 971 598 373
..Four members 16,464 16,142 12,518 999 2,626 322 240 81
..Five members 7,517 7,417 5,695 445 1,277 101 86 14
..Six members 2,820 2,784 2,095 198 491 37 24 13
..Seven or more members 1,729 1,712 1,237 119 356 16 9 7
Total
Family Households Nonfamily Households
For more information about ASEC, including the source and accuracy statement, see the technical documentation accessible at:
http://www2.census.gov/programs-surveys/cps/techdocs/cpsmar15.pdf.
 Sample is based on several US Census HH demographic
characteristics, including size, age, race, education, etc.

6/21/2017
Data “supply” - Consumer panelists’ “voices are heard”
 Report shopping trip data via scanned UPCs
Respond to product
preference and other surveys

6/21/2017
NCP is about data
Data Operations
• Collection, cleansing and distribution
• Multiple sources and multiple storage systems
• Reporting back to CPG and other clients

6/21/2017
Relevance of NCP’s Panelists to Industry
Data providers
• “Raw material” for our product offering
• Supply and demand
• Quality control
• Cost management
Subscribing customers
• Attrition/Churn/Retention
• CRM
• Lifecycle management
Data is our product
• Value to our clients -- and to their clients
• Importance of KPI’s
• Variety of analytics

6/21/2017
Data, Analytics and KPI’s

6/21/2017
MSci Analytics: Visualization to predictive modeling
Dashboard visualization & KPI reporting
Separate, but interrelated analytics
• Modeling - Panelist attrition and churn
• Forecasting - Panelist recruitment & scanner need
• Forecasting - Active and “static” counts
• Sample selection - Based on geo’s and demo’s
• Test & control – Incentive comparisons
– Sample size and demo’s/geo’s
Predictive modeling
• Selecting best prospective panelists
• Targeting existing panelist segments

6/21/2017
Client KPIs – Active and Static Counts
 Active count represents overall panel
 Annual static count represents most compliant households

6/21/2017
Quality of Fit Index
 How well the panel reflects actual US households
 Includes updates of census and panelist demos

6/21/2017
Stakeholder Inputs into Process

6/21/2017
Operations
S & OP Equivalent
• NCP delivers scanning panelists
• Vendor management
Panelist recruitment & replenishment
• General and targeted online recruiting
• Panelist reserves (geo’s and demo’s)
• Panelist motivations
Scanners
• Scanner shipments
• Inventory management

6/21/2017
Panelist Communications
Panelist Support Center
• Assisting panelists who contact NCP
• Contacting panelists proactively
Communications team
• Technology fit with panelists
• Compliance issues
• Special promotions and incentives
• Targeted efforts
Client-requested surveys
• Panelists voices heard (attitudinal)
• Data transmissions (behavioral)

6/21/2017
Enhancing an Existing Data Analytics
Integration Process

6/21/2017
Panelist Metadata – High Level Model Diagram
IntegratedDatabase
Trip data
Recruitment
source
Demos
Attitudinal
survey
Third-party data
Key dates
Communications
Incentives
Surveys

6/21/2017
Challenges
Getting the right data warehousing / business intelligence (DW / BI) system in
place can be a daunting task.
Critical factors for ongoing success include:
• Collaboration of cross-functional stakeholders
• Keeping up with technology advances
• Senior-level sponsorship
• Budget approval (read: value-added & demonstrated)
Once established, business objectives may evolve, requiring adaptation

6/21/2017
Continue to Demonstrate Value to Stakeholders
“Less advanced” analytics already managing KPIs
• Stable # of active panelists
• Stable # of long-term, compliant panelists
• Variable, but satisfactory “Quality of Fit“
• Assess changing priorities – knowing business landscape
Value-added opportunity lies in using advanced analytics
(including inferential statistics) to optimize operations -
ongoing

6/21/2017
Data, Analytics and KPI’s
Demonstrated Impact

6/21/2017
Overall metrics are meeting targets, but…
KPI Indicator
Sample size (Active count)
Compliance (Static count)
Tenure
Churn
Representative of US HHs (Quality of Fit)
• Demographics
• Geography

6/21/2017
…how can competing KPI’s be optimized?
Market research truisms:
• Difficult to find compliant participants <35 while easy to find them 55+
• <35 age participants in high demand
KPI Goal 1: Maximize Static count
• Opportunity: modestly oversample 55+ with high active to static
conversion rate
• Cost: over-represented 55+ and under-represented <35 static
KPI Goal 2: Maximize (Static) QOF
• Opportunity: significantly oversample <35
• Cost: low conversion rate means a modest improvement
Comes with increased churn and lower overall static counts

6/21/2017
Difficult to fully realize both goals
A balanced approach that optimizes the best mix, rather than
maximize on what might be a departmental goal is needed
Critical for cross-functional stakeholders to weigh in and
ensure that the KPI’s are optimally best for overall business
Organizational and analytical balanceChallenge of Competing KPI’s

6/21/2017
Quality of Fit KPI – Overall Measures
 Index measures how well the panel reflects actual US households

6/21/2017
Quality of Fit KPI – Select Demo
 Over-indexing Active, young households to improve Static measure

6/21/2017
Quality of Fit KPI – Select Demo
 This demo “over-indexes” due to better compliance

6/21/2017
Proprietary software selects optimal sample of reserve
panelists (i.e., recruited households) to best fill the vacancies
Along with this software, predictive model also used to identify
panelists most likely to comply
Combination of analytical approaches result in a selection of
prospective panelists predicted to optimize NCP’s KPIs
Incentive and communication tests to improve retention; way
to address the shortfall of the status quo QOF
Analytics and optimizing KPI’s

6/21/2017
Lessons Learned

6/21/2017
How to build consensus…

6/21/2017
Improve internal communications
Meet with clients & other stakeholders
• Pre-meet regarding complex assumptions
• Allow for delegation to ensure representation
Meet with the right frequency
• Monthly, bi-weekly, or weekly
• Crisis mode – could be daily
Promote culture of co-ownership
• Each department understands impact of analytics on others
• Impacts do reach bottom-line

6/21/2017
Cross-functional alignment is critical: MSci perspective at NCP
MSci
Database
Clients/
Sr.
Mgmt.
Recruit.
IT
Data
Ops.

6/21/2017
Next steps
Tracking “product” quality
• Promote panelist compliance and tenure
• Test different communications and incentive offerings
• Combine with panelist segmentation
Maintain senior management support
• Alignment of organizational KPI’s with departmental objectives
• Apply meaningful success criteria
• Demonstrate value to clients
– Financial cost efficiency
– Improved panelist compliance & other KPI’s

6/21/2017
Conclusions & Recommendations
Page 39

6/21/2017
Maximizing one (departmental) KPI over another can be sub-
optimal for the organization as a whole
• Proprietary software selects optimal sample demographics of recruited
households to replace dropped panelists
• A predictive model identifies panelists most likely to comply
The predictive analytic process was instrumental in:
• Identifying opportunities for improving existing KPI’s
• Adopting an improved client metric and improved operations
Test-control comparisons help to optimize effective and cost-
efficient panelist communications and incentives
Conclusions

6/21/2017
Recommendations
Include predictive analytics as a reflexive part of your business process
• Enable stakeholders’ business acumen to inform meaningful test-control comparisons
which in turn drive better business decisions.
Cultivate shared ownership of organizational KPI’s
• Manage attainment of departmental KPIs holistically to promote the optimal balance
for the organization
Consider that established KPI targets may need updating--or even
replacing--to continue to meet changing business needs
• Test and control comparisons provide one mechanism to leverage an existing
predictive analytic, business decision process

6/21/2017
Questions?
thomas.schleicher@ncppanel.com
Page 42

6/21/2017
Thank You!

1615 track1 schleicher

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a 1615 track1 schleicher

Similar a 1615 track1 schleicher (20)

Más de Rising Media, Inc.

Más de Rising Media, Inc. (20)

Último

Último (20)

1615 track1 schleicher