Predictive Analytics and Quality of Hire

Used correctly, predictive analytics will bolster your strategy
for attracting talent, streamline your recruitment process,
identify your workforce needs and improve Quality of Hire.
PREDICTIVE ANALYTICS AND
QUALITY OF HIRE

© 2016, Future of Talent Institute
Kevin Wheeler
February 2017
Predictive
Analytics and
Quality of Hire
2

3
Prediction is very difficult,
especially if it’s about the
future.
Niels Bohr

Technology that learns from data to predict
the future behavior of individuals.
In order to drive better decisions
What are Predictive Analytics?
4

Types of Data
5
Source: Adapted from Michael Corcoran
Information Builders
STRUCTUREDUNSTRUCTURED
Sentiment
Analysis
Predictive
Analysis
Text/Word Analysis
Data Analysis
Search
Ad Hoc Analysis
Performance Metrics
Dashboards
BREADTH OF DATA
DEPTHOFANALYSIS
Personality Analysis

Descriptive & Predictive Analytics Compared
6
Descriptive Analytics Predictive Analytics
Purpose
Understand the Past
Observe Trends
Discuss
Gain Insights
Make Decisions
Take Action
Timeframe Past and Current Future
Metrics Type Lagging Leading
Data Used Raw/Tabulated Information
Data Type Structured Structured and Unstructured
Benefits
Understanding
Efficiency
Information & Insights
Effectiveness

The Promise
7
MORE EFFECTIVE
COMMUNICATION
FASTER
PROCESSES
GREATER
CANDIDATE
UNDERSTANDING
HIGHER
QUALITY
HIRES
INCREASED
INTERACTION
ENAGING
CANDIDATE
SERVICE
PEOPLE
ANALYTICS

“When I use a word,” Humpty Dumpty said, in rather a
scornful tone, “it means just what I choose it to mean—
neither more nor less.” “The question is,” said Alice,
“whether you can make words mean so many different
things.” “The question is,” said Humpty Dumpty, “which is
to be master—that's all.”
The Illusive Definition: Quality of Hire
8
LEWIS CARROLL (Charles L. Dodgson), Through the Looking-
Glass, chapter 6, p. 205 (1934). First published in 1872.

What is a “Quality” Hire?
-Someone who does what is expected consistently
over time.
-Someone who exceeds the performance of an
“average” performer.
time
performance
L
H
L H
Average performer
High Quality Hires
Low Quality Hires

Solid Predictive Analytics Based on. . .
10
Avoiding Assumptions
Predictions based on proxies
Not questioning
The future = the past
Transparency Privacy
No Black boxes
Never using data without
candidates knowledge
Well thought out guidelines
Basics
A clear problem statement
Predicting what has impact
Data Structure
Clean data
Large enough sample size
A model neither too simple nor too
complex
Biases
Awareness of
unintentional/Inherent bias
Not favoring one set of data over
another

But Ah, the Rub!
11
Only 14% of organizations have data to
prove the positive business impact of
their assessment strategy. -Aberdeen 2014
Quality cannot be measured or predicted until
there are clear, objective and accepted
definitions.
What is expected? At what level or speed?
What is average performance?
What would above average look like?
How can performance be consistently and objectively
measured?
How is the information on performance collected?
Do you have the right data? Do you have enough
data?
Do you have enough relevant data?
Is there a large enough sample?

Can You PREDICT Who Will be Best?
12
“Talent assessments, backed by science, add a
level of objectivity to the hiring process that goes
beyond “gut feel.”
-Scott A. Scanlon
http://ow.ly/cUUU3095esy

Yes, with the right data &
evidence that can be shown
to cause a correlation in
performance.
13

Hard to show any cause/correlation for these:
Resume quality
GPA
Schools attended
Former employers
Experience
Common Ways Used to Measure QoH
14

Boolean Searching of the Resume Text
Behavioral Interviewing
Concept Mapping of the Resume Text
Ability Testing
Personality Trait Testing
Culture/Fit Testing
The Search for Magical Solution

Boolean Searching of the Resume Text
Can find resumes with specific skill/experience claims but
powerless to assess performance and risk.
Behavioral Interviewer Training
Studies prove BIs can accurately predict job performance but best
practices in the lab don’t get followed in the field.
Concept Mapping of the Resume Text
Burning Glass and Engenium are examples. Uses artificial
intelligence concepts to map resumes. Nice concept but has not
demonstrated great success in practice.
Do These Methods Work?

Ability Tests
Predict job performance on all jobs but can cause adverse impact
on protected classes; Require a confirmation scale.
Personality Trait Tests
Correlate moderately with performance for incumbents, but poorly
for candidates.
Culture fit/Job Fit Tests
May or may nor correlate with performance
Do These Work?
17

-Team-based work
-Facile communications
-Interpersonal relations
-Judgement-oriented
-Improvisational
-Reliant on expertise across
multiple functions
-Dependent on ﬂuid use of
ﬂexible teams
-Routine work
-Formal rules, procedures, &
training
-Low discretion workers or
automation
-Judgment-oriented work
-High skill/knowledge level
-Reliant on individual
expertise & experience
-Dependent on star
performers
COMPLEXITY OF WORK
AMOUNTOFCOLLABORATIONREQUIRED
Routine Interpretation &
Judgment
IndividualTeam
Transactional Experts
InnovatorsCollaborators &
Connectors
Modified from: Thomas H. Davenport, 2005
18
Different Types of Workers – Different
Quality of Hire Measures

Knowledge, Skills and Abilities (KSAs)
Role-Related Knowledge (based on experience)
Judgement (quality of decisions, success)
Past Performance (both measured and opinions)
Integrity (past activities and traits)
Fit (traits)
Roles can be Deconstructed into a Mix of
Core Components
19

Structured
Past performance
Role-Based Knowledge
KSAs
Unstructured
Risk / Background check
What Predicts Success for Transactional
Roles
20
Representative importance of factors

Structured
Role-Based Knowledge
Past Performance
Unstructured
Judgement
Citations/patents/awards
What Predicts Success for Expert or
Technical Roles?
21

Structured
Past Performance
Size of network
Volume of social media activity
Number of articles written
Role-Related Knowledge
Unstructured
Judgement
Fit
Teamwork
What Predicts Success for
Collaborator/Communicator Roles?
22

Structured
Role-Related Knowledge
Number of patents/awards
Number of ideas incorporated into products and services
Contributions that led to increased profits/sales/new
customers/better customer satisfaction
Unstructured
Cognitive ability
Mentions in press
Citations
Activities/events
Personality/fit
What Predicts Success for Innovator
Roles?
23

Job Knowledge Tests
May be predictive for Transactional and Expert roles
Virtual Job Tryouts
Effective & predictive for Transactional, Expert, and Collaborator
roles
Skills Testing
May be predictive for Transactional & Expert roles
Competency Profiling
May be predictive for Expert roles
Other Common Measures of QoH
24

So, Can Hire Quality be Measured
and Predicted?
YES! But. . .
Different roles require different measures of quality
Methods of determining quality are specific to roles
Correlation must equal causation
Unstructured data can be valuable but must be used along with
structured data.
You must collect and analyze objective data on performance and
not rely on proxies, assumptions or opinions.
Predictions are probabilities – not certainties.

By analyzing who is currently an excellent performer
through data from unstructured feedback and performance
reviews.
By discovering attributes, competencies and skills that lead
to that performance from analyzing social networks.
By analyzing potential employees for similar characteristics
via their web profiles and social media.
Emerging Ways of Predicting Candidate
Quality
26

Matching algorithms- Many providers
Accuracy? Relevance? Privacy? Discrimination?
Passive Personality Assessment- IBM Watson & others
Privacy? Relevance? Accuracy?
Chatbots- Olivia, Mya, Job Pal, Karen
Assumptions? Discrimination?
Online Assessments/games- Many providers
Privacy? Relevance? Correlation=Causation?
A Few Emerging Tools that May Help
27
But a big question: How transparent are the vendors
about their algorithms and assumptions?

Have a privacy disclosure policy that is shared with
candidates and is on your career site.
Have clear definitions of what you are looking for and
how you will know when you find it.
Let candidates know how you measure quality of hire.
Let candidates know why you have rejected them,
especially if based on social media information.
Train recruiters in what is acceptable, reliable, and
accurate.
Always use human judgement along with any AI or
other assessment or prediction.
Good Predictive Analytics Practices
28

. . .use more than one test or predictive tool.
. . .when choosing a tool, know what is in the algorithm.
Know what is being looked for, tested, analyzed, and how
each factor is weighted.
. . .make sure proxies are valid and really predictive.
. . .not adopt tools hastily and without careful thought and
knowledge.
. . .always question our own assumptions and beliefs.
You should. . .
29

30
THANKS
YOUR THOUGHTS & QUESTIONS
Follow me on Twitter: @kwheeler
Email: kwheeler@futureoftalent.org

Predictive Analytics and Quality of Hire

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

More from Cielo

More from Cielo (20)

Recently uploaded

Recently uploaded (10)

Predictive Analytics and Quality of Hire