1. Brief Analysis of
VSD - Results Frameworks
e + i Network Meeting
Thun, May 15, 2013
prepared & presented by
Franz Kehl & Markus Engler
KEK-CDC Consultants
on behalf of SDC FP e+i & SDC QA

2. Cooperation Strategies covered
Scope of Analysis:
13 VSD Result Frameworks
out of about 44 Cooperation Strategies
• Bangladesh
• Bolivia
• Bénin
• Bosnia Herzegovina
• Burkina Faso
• Grands Lacs
• Kosovo
• Mali
• Mongolia
• Myanmar
• Niger
• OPT
• Uzbekistan

3. Result Framework: Criteria / Questions for Analysis
Column 1 Column 2 Column 3
• Quality of outcome
statement,
(methodologic and
thematic)
• Nos. of outcomes
• Comprehensibility
• Informative value of
the ‘linking’ column
• Levels addressed, i.e.
linkages, intermediate
results, risks and
assumptions
• Comprehensibility of
statements
• Traceability
• Thematic clarity
Reference for analysis: - SDC-Guidance for Result Frameworks ( Annex 2 CS-Guidelines)
- SDC PCM interactive (http://elearningpcm.ch/)

4. General Findings & Observations
4
 Quality of Results Frameworks
 Roughly 40 % comply well with quality criteria
 About 40% are fairly well developed
 About 20% can be improved in important aspects
Given that the RF has been introduced only recently
and is a relatively demanding tool, assement shows a
 Positive picture on the whole, namely SDC-outcome
column filled appropriately.
 Improvements mainly required in column 2 and 3

5. Column 1: SDC-Outcomes
5
 Disadvantaged women and
men, and especially youth,
have access to relevant
vocational skills development
offers, to decent employment
and self-employment and earn
higher income
Good examples: What makes it good practice?
 Outcome statement describes a (change
of a) situation that should be achieved
 As short as possible and as long as
required to provide sufficient detail
 Differentiated by gender and poverty
situation
 Limited number of indicators (2 – 3) but
representing key aspects of the outcome
 People trained, including the
disadvantaged, have relevant
qualifications.
Indicators
- # of qualifications developed,
upgraded, validated & adopted
- # of disadvantaged (male-female,
poor-non poor, minorities) trainees
- % of trainees availing new / better
jobs.
Well
done

6. Column 1: SDC-Outcomes
6
Examples for improvement: What is wrong with it ?
 SDC-outcome: Improved
employability
National outcome: Establish
vocational training centers
 The human resource of the
country…
 Réduction du taux
d'analphabétisme
 Hierarchy of objectives in column 1 and 3
inverted. SDC-outcome is above national
outcome
 Enseignement formel primaire
 Employability strengthened /
improved
 Rapid skills development and
improved employability of men
and women with lack of
marketable skills.
 Too general, does not specify any
particularity of SDC support
How to
improve
?
 The scope of the SDC-outcome is too
broad to be realistic or too general to be
meaningful
 Describes approach and mixes approach
and outcome
 Same indicators as for National outcomes
 No target values and baseline

7. Column 1: SDC-Outcomes: Conclusions
 50 % of the Result Frameworks fulfill methodological
requirements in column 1.
 50 % of RF have some good elements but are not satisfactory
throughout, i.e.
- comprehensibility of statements generally okay, but
- some incoherence within the result framework.
 Main improvements required:
 Outcome statements: Clarification on how to formulate the
statements properly. Includes a clear understanding of what an
outcome is.
 Provide target values and baselines for indicators

8. 8
 Improved quality of technical,
vocational and higher education,
enhanced enrolment planning
and coordination
(NDS 4.4. Strategic Goal 4)
Country level indicators:
9) Number of VET graduates,
(NSO) (f/m)
10) Ratio of VET vs. higher
education students,
(NSO) (f/m)
Good examples: What makes it good practice
 Concrete outcome statements quoted from
National Strategies
 Reference to quoted documents given
 Country level indicators mentioned.
Preferably related baselines should be
added
Column 3: Country Development and Humanitarian Outcomes
 SDC Outcome: (Column 1)
Skills development delivery
system expanded ...
 Country outcome indicator: (Column 3)
Increased employment targets
from 1.8 Mio to 1.9 Mio. jobs
 Country level outcomes and indicators are
selected such that a logical link to the SDC
outcomes is apparent.
Well
done

9. 9
Examples for improvement: What is wrong with it ?
Related to a skills
development plan, the
following indicator is given:
 Increased overall economic
growth from x % to y%
 Outcomes or indicators taken as reference
are too ambitious and/or too general
 Conducive environment for
growth and poverty
reduction
 General statement not supported by any
reference to national policies or strategies
How to
improve
?
Column 3: Country Development and Humanitarian Outcomes

10.  Availability of national policies and strategies, providing useful
country development objectives is still a frequent problem.
 In their place statements from other sources are used as
substitutes
 Main improvements required:
 If substitute statements are used, source needs to be mentioned
as well
 Even if policy papers provide poor outcome statements,
thematically they should correspond with SDC outcomes
Column 3: Country Development and Humanitarian Outcomes
Conclusions

11. 11
 Impact hypothesis
If rural young women and men
have access to high-quality
and demand-driven vocational
education and training in their
region, they will be able to
obtain suitable employment.
 Intermediary results:
1. TVET institutions have
established a system to
ensure that their training is
adapted to changing employer
and market needs.
2. Number of practical training
modules provided by private
companies.
Good examples: What makes it a good practice?
 Make use of the proposed 'levels' since
they provide useful information to
understand program /project, in particular:
 Impact hypothesis is supportive to
understand intervention logic.
 Intermediate results formulated such that it
is obvious how they relate to the outcome.
Column 2: Link SDC Contribution and Country Outcomes
Well
done

12. 12
 Risks:
1. Declining competitiveness
of … SMEs as a result of the
mining boom.
Good examples: What makes it a good practice?
 Risks and assumptions: identify the most
critical respectively relevant ones, because
they are key to the performance.
 Explaining the links within the domain as
well as with other programs helps to
identify the much looked for synergies
Column 2: Link SDC Contribution and Country Outcomes
Well
done

13. Column 2: Link SDC Contribution and Country Outcomes
13
Examples for improvement: What is wrong with it ?
 SDC-outcome:
People have relevant
qualification.
related intermediate results:
Migration cost decreased,
labour protection increased
 Statements are too short to provide
meaningful information, respectively to
show the link between outcome and
intermediate result
 Trained people might migrate  Risks and assumptions that seem
important but actually have no direct
bearing on the performance of the program
How to
improve
?

14. Column 2: Link SDC Contribution and Country Outcomes
 In about 50 % of the Result Frameworks the suggested levels/
aspects are not described:
 less clarity on how the SDC outcomes contribute to country
outcomes.
 Risks are addressed in most RFs, but not necessarily the most
pertinent risks are mentioned
 Column with most heterogenous quality (if compared across all
RFs)
 Main improvements required:
 Point out the importance of this column
 Emphasize the adequate reflection of pertinent risks and
assumptions, as these are critical for the performance
Conclusions

15. Thematic Focus of RFs
Observations:
 The thematic range is fairly broad and diverse:
 From ‘up-grading matching services’ to ‘quality of training in basic education’
 Outcomes at different levels: ‘income generation’ versus ‘training of school
drop-outs’
 Thematic ‘clusters’ identified in the 13 RFs, respectively 34 outcomes:
 Access to VT / employment, inclusion: 6 outcomes
 Employability (of young people): 5
 Framework conditions 5
 Income: 4
 Linking market & training; Market-oriented training: 4
 Skills development, qualification: 2
 Specific mentioning of young people 8
 Coherence is observed among programs working in the same region
(e.g. Western Balkan, Western Africa) respectively(more likely) same
SDC-division.

16. Common Outcome Indicators
Analysis:
 Comparison of Indicators used in Result Frameworks with Common
Outcome Indicators:
 Attribution of RF-indicators to one of the COI fields of observation,
respectively where possible to a particular COI if phrasing is similar
Observations:
 About 50 % of RF-indicators are close to the phrasing of specific
Common Outcome Indicators.
 The other 50 % RF-indicators can only generally be attributed to one of
the four fields of observation
 Distribution of RF-indicators among the 4 fields of observation:
 Income: 9
 System: 19
 Relevance: 13
 Outreach: 24

17. Summing up / Questions arising
Results Framework
Although VSD-responsibles are doing already quite well on the
Result Frameworks, some questions arise for consideration in the
plenary discussion:
How to go forward regarding the recommended improvements?
a) Disseminating the results of this analysis along with a brief hand-
out providing guidance for weak points (e.g. formulation of
outcome statement) ?
b) Referring VSD-responsibles to existing (SDC-)guidelines by
means of internet links ?
c) Rely on services of QA for guidance in methdolodical questions?

18. Summing up / Questions arising
Thematic Focus
 How do we assess the (broad) range of themes?
 Is there a rationale to go for more focus in terms of themes?
E.g. mutual exchange and learning among programs, building
specific competence in SDC?
Common Outcome Indicators
 A more detailed analysis of the indicators used in the Result
Frameworks with reference to the COI is recommended.
 The pertinent question is:
How far do the COI reflect the practice or vice versa how far the
practice should be adjusted to COIs?