Predictive Analytics World Workforce

1. Predictive Analytics World Workforce
Michael Rosenbaum
Founder and CEO, Arena
www.arena.io
formerly

2. Agenda
• Background
• Catalyte results
• Arena results
• Lessons learned

3. Background
Big Idea: use data and predictive analytics to transform the workforce
Core Thesis Multiple
ApplicationsDrawn to creating value by
applying large amounts of
data to the most subjective
areas of human endeavor
Founded two companies to
put these ideas into practice.
Catalyte in the area of
software engineering, and
Arena in health care.
1. helping employers identify the best
people for each role, and
2. helping job seekers find meaningful
work where they are exceptional
Economics &
PolicyThe thesis for this
approach was born out of
work at Harvard and The
White House

4. The Catalyte approach
Do you think you are
a talented engineer?
- recruit talent -1
- collect data -
2
- generate prediction -3
- build software products -
4

5. Catalyte Case Studies
9%
16%
20%
0%
5%
10%
15%
20%
25%
Onshore
Vendors
Offshore
Vendors
Defect Rates (%)
$510
$1,439
$0 $500 $1,000 $1,500 $2,000
Other Vendors
$ per Agile Story Point
Catalyte teams 3X more
productive with better quality
77% more cards/month & 46% more
points/month
70
75
80
85
90
0
2
4
6
0
10
20
30
40
Accenture
Commit / Deliver (%)
Tier 1 FTE
Time to Steady State (sprints)
Tier 1 FTE
25
50
75
100
2 3 4 5 6
Consistency – Velocity (%var) Time to Peak Velocity (sprints)
Tier 1 FTE
Catalyte teams deliver on their commitments more
often, get up to speed more quickly, and are more
consistent
%
increase

6. Catalyte Case Studies (2)
0
2
4
6
8
10
12
14
16
18
Avg. Story Completion Time (days)ç
Average Others
Median Others
0%
20%
40%
60%
80%
100%
120%
140%
160%
6/20 6/27 7/4 7/11 7/18 7/25 8/1 8/8 8/15 8/22
Burned Hours/Estimated (%)
PMSI Others
PMSI
2016
0
25
50
75
100
S7 S8 S9 S10 S11 S12 S13
# of Stories Completed
Consistent Productivity
# of Points Completed

7. Two big challenges in building tech
workforce
SCARCITY DIVERSITY
“How About Some
Action on Diversity?”

8. Evidence of bias in “gut” hiring process
African
American
Region Industry Silicon Valley
United States 13.2% 12.6% 3% 1 1-2% 2
Baltimore 24.5% 28.7% ? --
Portland 3.0% 3.9% ? --
Sources:
1. NSF
2. News reports Bloomberg, USA
Today
Diversity of Employee Pool Compared to
Region, Industry, and Silicon Valley

9. Evidence that data can remove bias
African
American
Region Industry Silicon Valley
United States 13.2% 12.6% 3% 1 1-2% 2
Baltimore 24.5% 28.7% ? --
Portland 3.0% 3.9% ? --
Total female: 20.4%
Sources:
1. NSF
2. News reports Bloomberg, USA
Today
Diversity of Employee Pool Compared to
Region, Industry, and Silicon Valley

10. Catalyte engineer educational
background
Selection rate: 1.5%
Implication:
There is a large, undervalued
pool of exceptional talent that
can be tapped for more rapid,
higher performing capacity to
execute the mission
50% - 4 year college
6% - graduate degree
11% - community college
32% - no college

11. Catalyte -> Arena
Disintermediate
Customer + Tech
Direct Licensing

12. Applying data & predictive analytics to
talent
We use massive amounts of data and cutting edge data science to identify the best
people for each role. They may not be who you think they are.
Better candidates Machine
learning
Guaranteed
resultsUsing data and predictive
analytics, Arena identifies
the candidates who are the
best fit for each role, in each
department, in each location
As our platform gathers
more data from your
organization, it becomes
more accurate and improves
your specific outcomes while
you sleep.
We have a 100% success rate
across all clients and a median
turnover reduction of 38%. We
guarantee a 10% reduction or
we’ll refund cost of the pilot.

13. Results for our most popular outcome
Box plot shows our results in every job category across:
REDUCTION IN EMPLOYEE TURNOVER
Median is 38%
Worst result is 13.8%
Best result is 100%
(i.e., no turnover)

14. Preliminary results for patient
satisfaction
Patient Satisfaction outcome data
available with several clients
Use HCAHPS likelihood to
recommend scores as the outcome
Results show potential for significant
improvements
- HCAHPS Likelihood to Recommend -
Pre-Arena Post-Arena

15. Results in a Level One Trauma Center
5.9%
5.9%
5.9%
21.4%
10.5%
13.5%
24.2%
29.2%
44.2%
56.5%
75.7%
26.5%
Arena Recommended Arena Non-recommended
- General Nursing-
60 Day 90 Day 180 Day 360 Day
EmployeeTurnover(Sep2014–Dec2016)
8.3%
8.3%
8.3%
11.1%
22.2%
27.8%
38.5%
28.6%
62.5%
70.0%
78.3%
61.1%
60 Day 90 Day 180 Day 360 Day
- Non-Nursing Roles-
EmployeeTurnover(Sep2014–Dec2016)

16. Results in a Major Senior Living
Organization
During the pilot period, turnover
among was:
• 35.9% lower than Arena non-
recommended for all hires, and
• 39.4% lower among resident
care hires.
- Pilot Results Apr 2016 – Jan 2017 -
35.9% 39.4%
Arena Recommended Arena Non-recommended

17. Using Data to Classify Sales
Performance
Low
High
Uber
High
Employees move between
classification buckets over
time
Uber high performers are scarce

18. Controlling for Bias when Modeling
Employee Engagement
Manager Effectiveness

19. Lessons Learned
Data Science is not the whole solution
Accurate
predictions
Change
Management
User experience
Predictions must be
accurate and consistent or
users will perceive them as
noise and stop using them
The most accurate
predictions won’t make any
difference if people don’t use
them to make their day to
day decisions
Predictions must be presented
in a way that is intuitive for
non-technical people to
understand and data must be
collected with as little friction as
possible

20. Questions?
Michael Rosenbaum, CEO
mrosenbaum@arena.io
www.arena.io