Review of research on the effectiveness of the employement interview and the elements that lead to higher validity

1. What Do We Really Know
About the Employment
Interview?
Patrick Hauenstein, Ph.D
President, OMNI Leadership
The Right Person,
In The Right Position,
At The Right Time...
Every Time
OMNI Leadership
620 Mendelssohn Avenue North Suite 156
952.426.6100 1www.omnilx.com

2. What Do We Really Know About the Employment Interview
By Patrick Hauenstein, Ph. D
The employment interview has been around for a lot of years and has been researched
scientifically for at least 90 years. Many reviewers have reviled the interview as subjective,
biased, and little better than flipping a coin to make selection decisions. Never the less, most line
managers feel that they are able to make better informed decisions based on the interview and
consider it a powerful selection tool. What is the truth concerning interviews?
Structure Matters
There is a considerable body of research that supports the power of the interview when it is a
structured process. When interviews are structured, they have considerable reliability and
predictive value and are a powerful selection tool.
In comparison, unstructured interviews are characterized by:
• Each interviewer decides where they would like to focus their information gathering.
• Each interviewer comes up with their own questions
• Each interviewer uses their own evaluation framework for evaluating responses
• Each interviewer integrates the interviewee responses in their own way to reach
conclusions
Research suggests that when interviews are unstructured, they are little better than flipping a coin.
Structuring Interview Content
Structured interviews are organized around competencies. Competencies are simply a language
for describing the kinds of behaviors that are associated with successful performance in a job.
As an example, success as an executive is associated in part on a set of behaviors related to
strategy. Here is a common structure for a competency:
Competency Label: Strategic Thinking
Definition: Applies appropriate strategic logic to decisions and initiatives in one’s
area
Example Behaviors:
• Demonstrate understanding of key industry trends and conditions (e.g., market trends,
competitors) and their implications for one’s own area
• Demonstrate understanding of the logic behind the organization’s broader strategies
and long term direction
• Recognize and capitalize on customer/market needs and opportunities (e.g.,
geographic expansion, customer segmentation, emerging markets)
2

3. • Make decisions or pursue initiatives that leverage the organization’s strategic
resources and partnerships (e.g., brands, physical assets, core competencies) to
enhance power in the market
Used with permission from Personnel Decisions International
In a structured interview, competencies seen as important for job success are identified by
subject matter experts. Then the interview questions are constructed to obtain examples of an
individual’s past behaviors that illustrate their proficiency in each competency area. Some
examples are given below:
Examples of Structured or Behavioral Interview Questions for Common Leadership
Competencies
1. Competency - Analysis
Interview Question –
Describe the most difficult or challenging work-related problem that you have solved. What was
your approach for analyzing and solving the problem?
2. Competency - Strategic Thinking
Interview Question –
Tell me about the most difficult time you have ever had trying to develop a new strategy or
business direction in the face of worsening business results. Describe the circumstances, your
approach, and the results.
3. Competency - Innovation
Interview Question –
Tell me about the most innovative idea or work that you have ever developed. How well did it
turn out?
4. Competency - Focus on Customers
Interview Question –
Tell me about a time when you took a significant amount of time and effort to better understand
a customers business needs and provide better service.
Some other examples of common leadership competencies are provided in Appendix A.
3

4. Structuring Interview Responses
A key assumption of structured interviews is past behavior predicts future behavior. In a
structured interview, the interviewer attempts to elicit descriptions of specific behaviors. The
interviewer should not be interested in how a candidate generally behaves or would behave but
wants specific examples of actions taken in specific circumstances. Follow up questions are
used to obtain a complete behavioral response. The structure of a behavioral response can be
represented by the acronym CAR,
C- Circumstances or situation. A clear description of the situation, task, or circumstances faced.
A – Actions or behaviors. A clear description of the specific behaviors or actions taken in the
circumstances.
R – Results of actions or behaviors. A clear description of the impact or results of those actions
and behaviors.
An example of a structured interview question with a good CAR response:
Question - Describe the time that you were proudest of your ability to successfully
introduce change into an organization.
Circumstances - We had been primarily a services company and decided to offer
complementary products to enhance profit margins and meet client demands for a lower
cost solution. Since this was very new to the company, it was a major change effort and
required getting employees on board who were initially resistant.
Actions – The first thing I did was to research the literature and investigate what other
product organizations were doing as well as best practices in change management. Then,
I conducted focus groups with key stakeholder groups to surface concerns and define
criteria for success and formed a launch committee with key representatives from each
stakeholder group. We collaboratively designed the initial products for launch and
developed a comprehensive launch strategy that included field enablement,
communications, marketing, and sales enablement. We also focused initially on a small
group of clients and built success stories to gain excitement from both our internal and
external audiences.
Results – As a result of our actions we exceeded first year revenue goals for new
products by 130%.
4

5. Structuring the Evaluation Process
There are two key principles underlying how a response is evaluated in a structured interviewing
process:
• Recency – The interviewer is more interested in behaviors that have been exhibited in
recent job situations than in behaviors exhibited in the distant past.
• Relevance – The interviewer is interested in behaviors that have been exhibited in
situations or circumstances that are most similar to the challenges that will be faced in the
job under consideration.
Interviewer training is a key component for structuring the evaluation process and ensuring
accurate evaluations. Key components of interviewer training include:
• Behavior Categorization – Interviewers are given examples of candidate responses and
practice categorizing the responses into a competency framework. Interviewers then
discuss their rationale for their categorizations and reach agreement on how responses
should be categorized. This is an important skill since a planned question for one
competency may in fact elicit a response that is actually relevant for a different
competency.
• Calibration – Interviewers are shown a common video or role-play of an interview. They
independently provide competency ratings based on the responses provided. They post
their ratings, discuss the rationale for their ratings and reach agreement on how responses
should be evaluated.
• Avoiding Common Rating Errors – Interviewers are introduced to common mistakes
made by interviewers and discuss strategies for avoiding them. Some common mistakes
include:
o First Impression – For some interviewers, the hiring decision is made in the first
couple of minutes of the interview and is based solely on first impression. The
first impression is likely a combination of appearance, communication skill, and
poise.
o Halo Error – Interviewers may be so impressed with a particular accomplishment,
background, or demonstrated skill that they ignore information that may suggest
that other skills that are important for the job may be lacking. As an example, an
individual that has demonstrated he/she is superior in the execution of a strategy
may not be good at developing the strategy in the first place.
o Horns Error – Likewise, interviewers may be so unimpressed with an aspect of an
individual’s background or experience that they are blind to the positive qualities
that they may bring to the table. As an example, an individual who has had only
consulting experience may be discounted for a job internal to a company because
they have not worked solely in the company’s industry yet might bring a strong
broader perspective in successfully performing the role.
5

6. o Leniency/Severity/Central Tendency Errors – Some interviewers have trouble
differentiating among candidates. They may see all candidates as really qualified
(leniency), poorly qualified (severity), or middle of the road (central tendency).
o Managers’ judgments are affected by pressure to fill the position. Research
indicates that managers lower their standards when they’re too heavily pressured
to fill positions. They tend to rationalize poor information and rely too heavily on
training to compensate for weak skills.
o Managers’ judgments are affected by other available candidates. Managers often
make decisions relative to a group of applicants rather than to job needs. They
then find themselves taking the “best of a bad lot” rather than continuing their
search to find a really appropriate candidate.
The final component in the response evaluation process is data integration. After interviews
are completed, the interview team comes together as a group to integrate their findings and
discuss the interview responses in a systematic manner. After all the behavioral information
for a competency has been shared and discussed, the interview team comes to a consensus
rating for the competency.
Summary
Structure is the key to interview accuracy. Interview structure comes from many different
practices. Structured interviews:
• Use competencies as the framework for structuring the interview. Competencies are
based on a job analysis which is a systematic review of what it takes to be successful in a
particular job.
• Use pre-planned interview questions that are linked to the competency requirements.
This ensures that the interview questions are job related. Pre-planned questions also
ensure that the same questions are asked of each candidate to ensure consistency and
“apple to apple” comparisons. Every candidate has the same opportunity to demonstrate
they have the knowledge, skills, and background to perform the job.
• Use trained interviewers. Interviewers are trained in how to evaluate responses to
prepared questions. Training helps take the bias and subjectivity out of the evaluation
process and gives interviewers a common frame of reference for evaluating candidates.
• Ratings are integrated in a common way to arrive at an overall score as well as a profile
of strengths and weaknesses.
Omni - The Leadership Exchange is a broad provider of recruitment support and selection
solutions including its revolutionary automated interviewing support system, OASIS.
Patrick Hauenstein, Ph.D. is Chief Science Officer at Omni. Prior to that, he held senior
leadership positions at Personnel Decisions International and Development Dimensions
International. He has over 23 years consulting experience in the areas of leadership assessment,
behavioral interviewing, and talent management.
6

7. Appendix A
Some Common Leadership Competencies
BUSINESS
FRONT-LINE MID-LEVEL SENIOR
METACOMPETENCIES UNIT
LEADER LEADER EXECUTIVE
LEADER
Super Factor: Thought Leadership
Analyze Issues Make Sound Use Insightful Use Astute
1. Analysis and Solve Decisions Judgment Judgment
Problems
Understand Act Strategically Think Shape Strategy
2. Strategic Thinking
Strategies Strategically
Identify Think Creatively Innovate Display Vision
3. Innovation
Improvements
Super Factor: Results Leadership
Seek Customer Meet Customer Focus on Ensure
4. Focus on Customers Satisfaction Needs Customers Customer
Focus
Establish Plans Build Realistic Align the
5. Planning
Plans Organization
Execute Manage Ensure Optimize
6. Managing Execution
Efficiently Execution Execution Execution
Show Initiative Show Drive and Drive for Drive
7. Results Orientation Initiative Results Organizational
Success
Super Factor: People Leadership
Solicit Support Build Support Influence Use
8. Influence Others Organizational
Influence
Encourage Motivate Others Engage and Energize the
9. Engage and Inspire
Commitment Inspire Organization
Select and Develop Others Build Talent Develop
10. Talent Enhancement Develop Organizational
Talent
Relate Well to Establish Build Build
11. Building
Others Relationships Relationships Organizational
Relationships
Relationships
Super Factor: Personal Leadership
Readily Adapt Show Adapt and Demonstrate
12. Adaptability
Adaptability Learn Agility
Used with permission from Personnel Decisions International
7