Elements of Effective Practice - Design, Management & Evaluation
Presenting the business case for an evaluator specific national mentoring program
1. Presenting the Business Case:
A National Mentoring Program for
Evaluators
CES National Capital Chapter Annual
Learning Event
Nov. 23, 2010
Core Mentoring Working Group
1
2. Core Mentoring Working Group
James Coyle
Natalya Kuziak
Dominique Leonard
Judy Lifshitz
Kathryn Radford
Lisa Styles
Jane Whynot
4. 4
Objectives of The Mentoring
Working Group
To research the feasibility of developing a
national mentoring program for evaluators
To seek the input from attendees at the NCC
Annual Learning Event
To develop a business case and model for a
pilot project or several pilot projects
To facilitate the roll out of a national mentoring
program
5. 5
Background
One new evaluator’s need for mentoring
Need for mentoring expressed at the annual
learning event for the NCC in Oct 2009
Presentation on mentoring at a L&L session in
Ottawa, Nov 2009
Breakfast sessions at the CES Conference in
Victoria 2010
Formation of the Mentoring Working Group in
2010
6. 6
Methodology
The results from three lines of evidence
have guided the development of efforts to
date. These lines of evidence have
included:
Literature review
Secondary survey data review
National on-line survey on mentoring (n=432)
Informal consultations (CES, AEA)
7. 7
Findings by Major Theme
1. The demand for a mentoring program for
evaluators
2. The advantages and disadvantages of a
mentoring program for participants
3. The dimensions of an effective mentoring
program
4. The issues and risks to consider when
developing a mentoring program
8. 1. The demand for a mentoring
program for evaluators
8
9. 9
Demand for a mentoring program for
evaluators
Year Effort and Findings/Action
2005 Evaluation practice in Canada: results of a national survey*
50% respondents indicated that lack of mentor availability is a barrier
2008 Will they join the team and stay? A study of potential and new program
evaluators*
Individuals with mentor more likely to feel that evaluation is prestigious (49%vs.31%)
Individuals with mentor felt evaluators held enviable position (43%vs.24%)
2009 Mentoring via the Independent Consulting TIG: Enhancing the Value of
Professional Affiliations*
80% expressed interest in mentoring program (73% mentee, 52% mentor)
2009 Lunch & Learns for Evaluators of Ottawa session on Mentoring
Working group struck to investigate possibilities of a national program
* Denotes references available in the bibliography
10. Demand for a mentoring program for
evaluators
72% of respondents don’t have
a mentor
69% felt they would benefit
from a mentoring program as a
mentee
10
11. Demand for a mentoring program for
evaluators
62% of respondents are not
mentoring anyone
56% felt they would benefit
from a mentoring program as
a mentor
11
12. 12
Demand for a mentoring program for
evaluators
Yes,
30%
No,
24%
Don't
Know,
42%
Other,
4%
Q33 Would you be interested
in volunteeringin another
capacity?
13. 13
Demand for a mentoring program for
evaluators
Q23. In either role, when would you be interested in
participating in a national mentoring program?
19%
52%
32%
19%20%
1%
0%
20%
40%
60%
80%
100%
Asa mentor As a mentee
Longer term
Within 1-2 years
Immediately
15. 15
Advantages and disadvantages for mentees
Top 3 Advantages
1. Source of feedback and
strategies
2. Personal development
3. New or more challenging
work projects
Top 3 Disadvantages
1. Mismatch within the dyad
2. Inappropriate behaviour
by mentor
3. Distancing or neglect by
mentor
•Screening and matching is
important
Implications
• These are therefore success
factors to monitor and evaluate
16. 16
Advantages and disadvantages for mentors
Top 3 Advantages
1. Development of
discipline/next generation
of evaluators
2. Obtain fresh
perspectives, knowledge
and skills
3. Opportunity to
demonstrate and enhance
leadership skills
Top 3 Disadvantages
1. Legal complications
(grievance, nepotism)
2. Negative reflection on
mentor (low-performing
mentee)
3. Dysfunctional relationships
• Could be focus of
messaging for recruitment
strategy
Implications
•Legal risks need to be examined
•Continuous monitoring
•Develop clear guidelines for
participants
17. 17
Advantages and disadvantages for organizations
•Legal risks need to be
examined
•Develop clear guidelines for
participants and code of
ethics
Implications
• Could be focal points for
the marketing a national
mentoring program
Top 3 Advantages
1. Lower turn-over,
employee retention
2. Organizational
cohesiveness
3. Succession planning and
organizational growth
Top 3 Disadvantages
1. Lowered morale or
grievances
2. Perpetuation of
inequalities (uncontrolled
informal mentoring)
3. Poaching of employees
19. Dimensions of an effective mentoring
program: functions
Two main functions of mentoring relationships:
a) Benefits to career
Challenging projects
Feedback
Visibility
Strategies
Protection
b) Psychosocial support
Friendship
Counselling
Role-modelling
19
20. Dimensions of an effective mentoring
program: functions
20
3.60 3.70 3.80 3.90 4.00 4.10 4.20 4.30 4.40 4.50 4.60
Support in adapting to a new
workplace, culture, language, profession
Advice on navigating work relationships
Career planning advice
Advice on avoiding mistakes
Exposure to new contacts and opportunities
in evaluation
Advice on ethical questions in evaluation
Expert skills in a subject matter
Rating Average
21. 21
Dimensions of an effective mentoring
program: participant attributes
• Very important attributes for both parties in a
mentoring relationship
– Understanding
– Availability/consistency
– Enthusiasm
– Willingness to share resources
– Personality/interpersonal compatibility
– Experience/interest in specific content areas
22. Dimensions of an effective mentoring
program: types
Informal
Relationship develops
naturally, no
organizational support
Formal
Supported and
sanctioned
Presence of
structure, guidelines, poli
cies
Provision of assistance
for mentoring lifecycle
Traditional/hierarchical
form
Dyad
Non Traditional forms
Peer/lateral
Team/group
Multiple
Functional/needs driven
22
23. 23
Dimensions of an effective mentoring
program: types
Q16 What type of mentoring relationship would you
prefer? (n=398)
49%
23%
14%
9%
5%
One-on-one relationship
Access to a network of
multiple mentors
Group mentoring
Functional mentoring for
a defined project only
Other
24. 24
Dimensions of an effective mentoring
program: communications
In-person, face-to-face
E-mentoring (email, chat or discussion
groups, E-forums)
Blended
E-communications as primary (50% on-line)
E-communications as supplemental
25. 25
Dimensions of an effective mentoring
program: communications
0% 20% 40% 60% 80%
Other
Private e-dialogue on a dedicated website
E-forum between mentor and group of mentees
Group in-person meetings
Telephone
E-mail
Face-to-face interaction
Q17 How would you prefer to communicate with your
mentor(s) or mentee(s)? (n=398)
26. 26
Dimensions of an effective mentoring
program: supports
Pairing/matching function
Monthly online newsletters
Resume database
Resources and links
Central mentorship coordination
27. 27
Dimensions of an effective mentoring
program: supports
89.8 88.3 87.3 85.2
78.1 77.3
70.4 70.4 69.1
51.0 47.0
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
% of respondents rating program component as important or very important
28. 4. The issues and risks to
consider when developing a
mentoring program
29
29. Issues and risks: lifecycle flexibility
Initiation
(6 mos -1
year)
Cultivation (2-
5 years)
Separation
(6 mos – 2
years)
Redefinition
(~years after
separation)
30. 31
Issues and risks: matching
Successful matching is critical
Strategies for successful matching:
Informal social gatherings
Face-to-face meetings
Seek out more than one mentor
Creating profiles based on matching criteria
E.g., gender, work styles, personality traits
88% of survey respondents said matching was
important to very important
31. 32
Issues and risks: critical supports
Arrange for recruitment, training & support of
mentors
Guidelines for the establishment of clear goals &
expectations for both mentors & mentees
Code of ethics that addresses:
Confidentiality & trust
Integrity & honesty
Conflict of interest
Professionalism
32. 33
Other issues and risks
Potential to become overly “bureaucratic”
Mentoring may mitigate the loss of young
evaluators from the field
Almost 53% of survey respondents were
from Ontario
Costs are unknown at this point
33. 34
Issues and risks: Pilot program
implications
1. Be flexible
2. Matching protocol
3. Training, support, goals
4. Code of ethics
5. Simple
6. Get involved
35. Conclusions
There is an appetite and demand for mentoring
for evaluators in Canada
There is a significant number of evaluators who
are interested in participating as mentees,
mentors or both
The advantages of mentoring for evaluators are
likely to outweigh the disadvantages and
disadvantages can be mitigated with a well
designed program
36. Proposed model
Proposed model is a national on-line mentoring service
with multiple forms
Capacity for on-line discussion groups or open forums
Program Coordinator(s)
Database of profiles, screening of participants, and a
matching process
Suite of support tools (e.g., participation guidelines, code
of ethics, resource information, training materials,
monitoring, evaluation)
37. Next Steps
Design a
model
program
Determine
costs and
methods
of cost
recovery
Develop a
business
case
Obtain
funding or
in-kind
support
Pilot a few
mentoring
projects in
Canada
Determine
success
and areas
for
improvem
ent
Develop
a
national
program
38. Questions for Discussion
Have we identified all the issues? What have we not
considered?
How do we keep the momentum going? Who will assist?
e.g., creating a business case? launching pilots and the
national program? Who should be approached?
Where should the program be piloted? For how long?
Are evaluators willing to pay for such a service? How
much?
Is a national mentoring program for evaluators
sustainable?
39. 40
Acknowledgements
Supporters of the Core Mentoring Working Group:
Anna Engman
Claude-Anne Godbout Gauthier
Lisa O’Reilly
Canadian Evaluation Society
Survey respondents
To contact us: MentoringWorkingGroup@gmail.com
40. 41
References
Allen, T. & Eby, L. (2007). The Blackwell Handbook of Mentoring: A
Multiple Perspectives Approach. Blackwell Publishing
Ltd, Oxford, UK.
Gauthier, B., Borys, S., Kishchuk, N., Roy, S.N. (2006) Survey of
evaluation practice and issues in Canada in The Canadian Journal
of Program Evaluation. Vol.(21)3, pgs.1-42
Kram, K. E. (1985). Mentoring at work: Developmental relationships in
organizational life. Glenview, IL: Scott, Foresman and Company.
Martinez-Rubin, Norma and Becky Melzer. Mentoring via the
Independent Consulting TIG: Enhancing the Value of Professional
Affiliations, 2009.
Roy, S.N., Kishchuk, N., Gauthier, B., Borys, S. (2008) Will they join the
team and stay? A study of potential and new program evaluators.
Paper presented at the CES Conference, Québec, May 12, 2008
42. 4343
Survey Demographics
By gender (n = 392) By age range (n = 391)
Female
73%
Male
27%
•Cross tabs for gender X preference to act as mentor or mentee probably important
1.3%
23.2%
27.6%
25.8%
19.3%
2.8%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
24 and
under
25 to 34 35 to 44 45 to 54 55 to 64 65 and
over
43. 4444
Survey Demographics
Highest level of education obtained (n = 393):
2.3%
13.2%
3.8%
60.1%
20.6%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
High school, college, or otherBachelorsGraduate certificate or diplomaMasters Doctorate
44. 4545
Survey Demographics
Province/territory of primary workplace
or place of study (n = 390):
52.8%
13.6% 11.5%
6.2% 2.8% 2.3%
10.8%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
• Observations and conclusions
•Need to discuss with Jane re. additional sampling that occurred
45. 4646
Survey Demographics
What sector do you work in? (n = 389)
22.1% 21.3%
12.6% 12.3%
10.3%
4.9%
2.8% 1.8%
11.8%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
46. 4747
Survey Demographics
How many years have you been
working in evaluation? Avg: 9.62 yrs
44.7%
21.7%
14.4%
7.6% 6.5%
3.5% 1.6%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
1 to 5 6 to 10 11 to 15 16 to 20 21 to 25 26 to 30 30+
# of years in evaluation