A Rationale to Pilot New Categories in Ontario. Feedback being obtained from conference participants.

1. Revitalizing CES
Membership
Rationale for piloting new categories in Ontario
Nikhat Rasheed
Vice-Chair CES-ON Board & Chair, Membership Committee
Tuesday, June 11, 2013
Main Mezzanine, Algonquin room

2. Agenda
What did we do? Our Process 5 min
Why change membership categories? Drivers for Change 15 min
Proposed new categories…Description, Benefit, Risks and
Mitigation Strategies
20 min
What we’re thinking about…Assumptions, Constraints
and Limitations
5 min
It’s your turn…Group feedback sessions 25-30 min
Report back from groups 10-15 min

3. What did we do?
Process Map

4. Our Process (June 2012-May 2013)
Retained expertise, went through strategic
marketing review and approved a plan
Met with CES National President at
Ontario AGM
Category rationale clarified, internal
discussions and prioritization
Data gathering and analysis, limitations
and gaps
Prepared draft rationale and shared with
CES National and CES members

5. Why change membership
categories?
Drivers for Change

6. Drivers for Change
• Membership at the Ontario Chapter is
stagnating at 375-400 members per
year (as measured since 2008).
• Members currently lapse at a rate of
approximately 20% per year (as
measured since 2008). Main
addressable reasons are:
• Lack of perceived benefits in
membership
• Cannot take advantage of event
• Not enough PD choice
• Toronto-centric
• No promotion of evaluation or
members
• Can no longer afford to pay the fee
or employer no longer willing to
cover to the fees (particularly
when the employer is a not-for-
profit)
• Both of these reasons are directly tied
to a cost/value equation problem

7. Strategic Marketing Review
Strategic Goals Strategic Actions Tactics
1. Reduce the lapse rate
from the current 20%
per year to 15% by the
end of 2013, 10% by the
end of 2014 and 8% by
the end of 2015
Review and realign membership
levels with National to reduce
lapsing and to offer better
opportunities for membership for
those associated with evaluation.
1. Full member
2. Associate member* (individual
membership for those interested
in evaluation or whose job has
changed to include less
evaluation).
3. Retired member
4. Student member
5. Student transition member**
6. Corporate member based on
employee numbers**
7. Fellow/Honorary lifetime
member ***
2. Increase new
member rate by 5%
each year for three
years (2013-2015) from
a three-year average of
82 new members per
year to 95 by the start
of 2016
Develop multi-year memberships at
a discount.
Work with the National
organization. Multi-year discounts
not reduce costs for members, but
also helps reduce the “forgot” factor
and encourages commitment.

8. Competitors – Canada and Intl
Evaluation Orgs Market Research Orgs Research/ Sector Orgs
1. The Canadian Evaluation
Society (CES) (Canada)
2. The European Evaluation
Society (EES) (Czech
Republic)
3. The American Evaluation
Association (AEA) (USA)
4. The Australasian
Evaluation Society (AES)
(Australia)
5. The U.K. Evaluation
Society (UKES) (UK)
1. Market Research
and Intelligence
Association (MRIA)
(Canada)
2. Market Research
Association (MRA)
(USA)
1. American Educational
Research Association (AERA)
(USA)
2. Ontario Public Health
Association (OPHA) (Canada)
3. The Canadian Association for
Health Services and Policy
Research (CAHSPR) (Canada)
4. Association of Educational
Researchers of Ontario
(AERO) (Canada)
5. Ontario Non-profit Network
(ONN) (Canada)

9. Competitor Analysis
1. Most associations have student discounts
2. Australasian evaluation organizations provide multi-year
discounts
3. European evaluation associations have organizational fee
structures whereas Canadian, American and Australasian
organizations do not. Could that be because evaluation is
a more established discipline in Europe?
4. Market research organizations have organizational
memberships as does the Ontario Non-profit Network,
funded by the public sector
5. AEA has a hardship provision for one-year

10. Competitor Analysis: Fee $
165
95
150
130
60
40
75
65
123.75
75 75
0
20
40
60
80
100
120
140
160
180
CES AEA AES UKES
Regular
Student
Joint (Regular)

11. Proposed new categories…
Category Description
Data
Benefits, Risks and Mitigation Strategies

12. Lifecycle Approach
Student
New
Practitioner
Associate
Member
Full
Member
Retired or
Senior
Member
Lifetime or
Honorary
Member
(by
invitation)

13. New Practitioner Category
• Recent graduates will be eligible for the New Practitioner
category for 2 calendar years after graduation at a reduced
fee of $90 per year (the same as currently paid by ‘Senior’
members).
• New Practitioners will be required to provide proof of
their graduate status which will be validated by
submission of a dated transcript or credential to allow
them to access this category.

14. New Practitioner Category - Data
• Students make up 13% of Ontario members (approx. 45 in
total in 2012-13). The current fee for students is $60 and
the full fee once they graduate is $165, an increase of
175%.
• Focus groups with students conducted
• NPS committee feedback
• Initial spot analysis done on student members from 2009
to 2012 shows very poor conversion rate – 13% (select
2009 and 2012 dates)
• Calculations being worked on.

15. New Practitioner Category
Benefits Costs/ Risks Risk Mitigation
Retain more student
members as they
transition from school to
work with lower
membership fees
Stabilize membership
lapsing rate
Increased retention would
lower the costs of
recruiting new members
CES Ontario is not able to increase its
student population over the time
period to break even
The Assumption that cost affects
membership retention does not play
out
• The NPS committee and the marketing
committees will make specific efforts to attract
students in evaluation by leveraging the
conference 2013, and dedicating resources to a
full student and faculty engagement strategy
• The NPS committee is committed to delivering
supports to New Practitioners through online
forums, discussion groups, social media, meet-
ups/ cafes and new events geared to students
and new practitioners
• CES-ON takes measures to deliver increased
value to members
The current membership system is
unable to handle the proposed
changes
• CES Ontario and CES would have to work out
the pilot options and arrangements
• A more thorough risk assessment and project
management plan would have to be prepared
Others to be identified Others to be identified Others to be identified

16. Associate Member Category
• This category was suggested by our strategic planning
consultant as individual membership for those interested
in evaluation or whose job has changed to include less
evaluation. However, because we do not have an ability to
exclude or distinguish benefit structures between
associates and full members, the board has not pursued
this category.

17. Lifetime or Honorary Member Category
• The Board has not spent much time looking at this
potential category. However, it seems to make intuitive
sense, as in, patronage can be a way of garnering
credibility.

18. Organizational Memberships
• Organizations receive bundle discounts in memberships
depending on the number of employees they register with
CES-ON.
• Memberships are transferable between different
employees.
• Memberships are for one calendar year January 1 to
December 31, and are pro-rated.
• Calculations being worked on.

19. Organizational Memberships
Benefits Costs/ Risks Risk Mitigation
• Create positive
pressure for
organizations to
take advantage of
cheaper
membership
• Increase awareness
within an
organization of
CES-ON as more
people can become
members
• Increase flexibility
for organizations
e.g. memberships
can be transferred
within
organizations
Governments are
cutting back spending
 Strong promotion of evaluation strategy to
governments at all levels tied to their platforms of
accountability, transparency and evidence-based
decision making
 Providing discounted memberships by volume would
be particularly attractive
Organizational
memberships may cost
us lost revenue
 Suggestion to pilot organizational memberships for
at least 2 years to study the effects on our
membership
 Lost revenue is a possibility, however, given most of
our members come from the broad public sector, a
marketing strategy promoting organizational
memberships within this sector might help limit
revenue loss

20. Organizational Memberships
Benefits Costs/ Risks Risk Mitigation
As in previous slide Non-profits lapse the
most
 CES-ON should become an active member
of the Ontario Non-profit Network (ONN)
to promote its mandate and training/PD
services to non-profits
 Most non-profits that apply for, or host
government funded projects, have to
provide some sort of evaluation of the
project. CES-ON could launch a campaign
to encourage them to build the cost of
membership into their budgets or into
training budgets for their staff
Others to be identified Others to be identified Others to be identified

21. Multi-year Memberships
• In order to retain members for longer periods, and to
reward our most loyal members, CES could offer them
multi-year discounts when they register or renew their
memberships.

22. Multi-year Memberships
Benefits Costs/ Risks Risk Mitigation
• Identify members who
sufficiently value CES
and intend to be loyal
members for a time-
period
• Provide financial
incentives to members
• Stabilize revenues over
time periods
• Reduce lapsing rates
• Governments are cutting
back spending
• Individuals may be
reluctant to sign-up for
more than one-year
 Strong promotion of
evaluation strategy to
governments at all levels
tied to their platforms of
accountability,
transparency and
evidence-based decision
making
 Strategy to understand
reluctance or low uptake
and then mitigate that
Others to be identified Others to be identified Others to be identified

23. We’re still planning….
Assumptions
Constraints
Limitations

24. Assumptions
It is assumed that: Effects on investment: Reliability Level:
Assumption 1
Many people are not joining CES or
retaining their memberships due to
the prohibitive cost of membership
If there are other factors that
affect retaining or applying for
membership then the revised
structure will not make
membership more attractive
Based on data
Assumption 2
The revised membership categories
will meet potential member needs
If effective, the revised
categories will increase
membership numbers.
Based on data
Assumption 3
The CES-ON and CES will promote
awareness of the revised
membership categories
Marketing will contribute to
the high number of new
members needed to join or
renew (when they would
otherwise lapse) to ensure
stable membership revenue
Based on data and
testing

25. Constraints
• While there is some survey data to support the hypothesis
that a revised membership structure will attract or retain
members, there are significant gaps in hard data,
including:
• What is the lifecycle of a member? How many students do we really
lose in the transition from student to individual membership?
• How many organizations would be attracted to organizational
memberships?
• We know memberships lapse for certain reasons (cost, evolution in
work, consultants versus employees) - however, what additional
trends in lapsing can we uncover?

26. Limitations
We have…
• Utilized strategic marketing consultant expertise, membership
data, spot analysis, competitor analysis
• Gone on our experience and “gut” to some degree
However….
• Accurate life-cycle data not tracked
• Membership processed nationally
• Membership data is static – monthly excel lists
We realize…
• Limited ability to develop full cost estimate of the proposed
changes
• Need to finalize technology, human resource, capacity costs

27. Now over to you…
Q1: Benefits & Risks of Proposed Categories
Q2: Considerations & Challenges Moving Forward
• NEW PRACTITIONER MEMBERSHIPS
• LIFETIME/ HONORARY MEMBERSHIPS
• ORGANIZATIONAL MEMBERSHIPS
• MULTI-YEAR DISCOUNTS