Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
NTLT 2013 - Paul Maramos - How we are measured impacts upon how we teach
1. How we are
measured
impacts upon
how we teach
Paul Marambos
Southern Institute of
Technology
paul.marambos@sit.ac.nz
2. Research Topic
Will EPIs contribute to
the achievement of
key objectives for
the tertiary sector?
3. Why measure?
Nearly 700,000 New Zealanders
participated in formal tertiary education
in 2009, including industry training.
The Government spends around $4 billion
a year in operating expenditure
4. Background
Widespread increase in measurement
within organisations.
Management accounting – drive to link
research to practice
Case study approach
5. Content
Key tertiary objectives for New Zealand
Education Performance Indicators (EPIs)
Research undertaken
Results
Suggested changes to the EPIs
6. Key tertiary objectives for New
Zealand
1. provide New Zealanders of all backgrounds with
opportunities to gain world-class skills and
knowledge
2. raise the skills and knowledge of the current and
future workforce to meet labour market demand
and social needs
3. produce high-quality research to build New
Zealand's knowledge base, respond to the needs
of the economy and address environmental and
social changes
4. enable Maori to enjoy education success as Maori.
7. Government expectations of
Universities
The Government expects universities to:
enable a wide range of students to
successfully complete degree and post-
graduate qualifications
undertake internationally recognised original
research
create and share new knowledge that
contributes to New Zealand’s economic and
social development and environmental
management.
8. Government expectations of
Polytechnics
The Government expects polytechnics to:
enable a wide range of students to complete
industry-relevant certificate, diploma and applied
degree qualifications
enable local access to appropriate tertiary
education
support students with low literacy, language, and
numeracy skills to improve these skills and
progress to higher levels of learning
work with industry to ensure that vocational
learning meets industry needs.
11. Linkages between objectives
and measures …..
“The Government …...... expect to
see better course and qualification
completion and progression rates for
students as a result of higher quality
teaching and learning, and more
effective and culturally responsive
pastoral care.”
12. Guideline from management
accounting research ….
Care needs to be taken both to
observe the measures actually in
use, and also areas where
measures are absent. It is a
truism that what is measured
tends to drive out what is not
measured, and so omissions may
be as influential as measures in
use. (Ferreira & Otley, 2006)
14. Was guideline from management
accounting research followed….
There is insufficient explanation
provided as to how the TEC’s
EPIs were derived from its
strategic objectives, eg by
utilising strategy maps in order to
link objectives to measures.
15. Guideline from management
accounting research ….
it is as much the process of developing and
using the CPM system, as it is the resultant
performance measures that yield
motivational benefits. To drive motivation
the CPM system should be developed and
used in a way that enhances the
employees’ participation, psychological
empowerment, and goal commitment
(Bourne, Franco-Santos, & Lucianetti, 2012).
16. Was guideline from management
accounting research followed….
There is little evidence to suggest
that a consultative process took
place with key stakeholders
during their development.
17. Guideline from management
accounting research ….
The evidence suggests that the information
included in CPM systems is likely to increase
the understanding of individuals regarding
what is expected from them at work
(Bourne, Franco-Santos, & Lucianetti, 2012).
18. Was guideline from management
accounting research followed….
How educators have interpreted
and responded to the
implementation of the EPI’s is a
potential area for further
research
19. Guideline from education
research ….
The lack of a demonstrated causal logic
between the measures used in many of the
university league tables for first-level degree
programmes and student learning
outcomes …… encourages the gaming
behaviour and strategic mis-investments
reported in the US (Dill & Soo, 2005).
20. Was guideline from education
research followed….
An area that needs further
research is looking at whether
using EPIs to create league
tables for universities and
polytechnics has encouraged
“gaming behaviour” amongst
tertiary organisations in New
Zealand.
21. Guideline from education
research ….
The use of the results of performance
measurement in league tables assumes that
all those being measured start from the
same point; however there are usually more
sources of difference between same-service
providers than there are similarities (Adcroft
& Willis, 2005).
22. EPIs: Qualification completion
One completion:
University 1 One completion:
University 2
Was guideline from education
research followed….
23.
24. Incongruent behaviour
TEOs who select only the most
academically able students, as shown by
their secondary school achievements,
effectively deny educational opportunities
to those whose socio-economic
demographic meant that they may never
have had a chance to obtain similar
secondary school achievements.
(McClelland, 2012)
25. Recommendations
1. Outline how vision, key success factors
and strategic objectives for the tertiary
sector are linked.
2. Outline how the TEC’s EPIs were derived
from its strategic objectives
3. Consider whether alternative or
additional EPIs could be used in order
to ensure that strategic objectives are
met..
4. If EPIs are revised make use of a
consultative process that ensures all key
stakeholders are involved.
26. Recommendations (cont.)
5. Consider what levers universities and
polytechnics can use with regard to improving
the scores of their EPIs
6. Determine what expectations the EPIs
communicate to tertiary organisations and
their teaching staff, and whether these
expectations are in line with the strategic plan
for the tertiary sector.
7. Measure individual programmes, this will
provide more meaningful information that
enables effective benchmarking.
8. Consider providing information on
employment rates of graduates for individual
programmes.
Notas del editor
A key question to be asked is whether the use of the four EPIs listed above will contribute to higher quality teaching and learning, and more effective and culturally responsive pastoral care. It could be argued that the EPIs could cause the opposite to occur, ie because the EPIs focus on outputs it could put pressure on universities and polytechnics to lower educational standards in order to improve student progression / completion. Graduation rates can be independently controlled by each university and this therefore poses a problem, that is whilst graduation rates can be increased by more effective teaching and student learning, they can also be increased by lowering academic standards (Dill & Soo, 2005).
It needs to be asked if it is possible to quantify whether “high quality and excellence in tertiary education” is being delivered, for example whether a student’s ability to think critically has been sufficiently improved. Some aspects of performance are more difficult to quantify and measure, the main danger inherent in modern control systems design is that it is these aspects that get relatively less emphasis, and that we show and almost inevitable tendency towards various forms of short-termism (Otley, 2003).
It is therefore important to ask the question as to what expectations do the EPIs communicate to tertiary organisations and their teaching staff. These are that students should be retained, should pass their individual subjects / papers, and should complete their qualifications. What has not been communicated is the process for achieving this. Whilst this may be achieved by increasing the quality of teaching, it could also be accomplished by reducing standards.
For example an analysis could be done to determine whether more qualifications are being issued for the same course of studies since the introduction of the EPI that measures completion of qualifications.
Point 2: this could be achieved by utilising strategy maps which are used when developing a balanced scorecard.
Point 3: For example there is an expectation that the quality of education should improve, a consultative process could be introduced to determine whether performance measures focussing on teaching quality could be introduced
Point 5: determine if any of these could be viewed as incongruent behaviour.
Point 7: Instead of measuring individual tertiary organisations as a whole,
