Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Mindset - A Clash of Mindsets Part 1
Similar a Mindset - A Clash of Mindsets Part 1 (20)
Más de SIAAssociation
Más de SIAAssociation (7)
Último
Último (20)
Mindset - A Clash of Mindsets Part 1
- 1. A clash of mindsets finding a middle way between different impact methodologies Joy Mackeith, Triangle, Joy@triangleconsulting.co.uk Neil Reeder, London School of Economics, n.b.reeder@lse.a.uk Social Impact Analysts Association annual conference, December 2013
- 2. Aims of session • Lessons from practical experience • Outline of findings from academic research project • Initiate debate on key issues arising from different mindsets
- 3. Objectivity and subjectivity (or direct and indirect measurement) Direct measures (often end outcomes) Indirect measures (often intermediate outcomes) Things that can be directly observed, eg housing, offending Things that cannot be directly observed but have an impact on directly observable behaviour, eg confidence, improved relationships Associated with direct cost savings Associated with sustainability of outcomes
- 4. Objectivity and subjectivity: Middle ground Use both! Bring objectivity into indirect outcomes through tools for standardising professional judgement or subjective report ie psychometric tools Direct measures Indirect measures
- 10. Research on validity, reliability and practical application Dickens et al (2012) “Recovery Star: validating user recovery” in the Psychiatrist Dr Helen Killaspy (2012) “Psychometric properties of Mental Health Recovery Star in British Journal of Psychiatry Harris and Andrews (2013) “Implementing the Outcomes Star well in a multi-disciplinary environment” RMIT University, Australia (Homelessness Star, Drug and Alcohol Star and Empowerment Star) York Consulting (2013) “Family Star Evaluation” published by Family Action
- 11. Standardised systems versus case by case approaches Standardised systems Case by case approach Provides a shared framework for use across many projects or people Responds to the uniqueness of each project or person Can be aggregated Captures all impacts and reflects values and language of those being measured
- 12. Standardised systems versus case by case approaches : Middle ground Frameworks that allow case by case approaches to be analysed within a standardised system (eg BSC) Tools that are modified for different client groups (eg Outcomes Star) Standardised systems Case by case approach
- 13. Example: Big Society Capital Outcomes Matrix Outcome Area Employment Education and Training Housing and Local Facilities Income and Financial Inclusion Physical Health Mental Health Family Friends and Relationships Citizenship and community Arts, Heritage, Sports and Faith Conservation of the Natural Environment Individual Community, Sector and Society
- 14. Example: Outcome Star versions Recovery Star (mental health) Life Star (Learning Disabilities) Work Star Older Person’s Star Youth Star Empowerment Star (domestic violence) Teen Star Well-being Star (long-term health conditions) Spectrum Star (Autistic spectrum) Family Star Community Star Shooting Star (schools) Homelessness Star Alcohol Star Music Therapy Star Drug and Alcohol Star
- 15. Experimental versus exploratory approaches Experimental Exploratory Simplify complexity by isolating one variable and identifying its impact (ie RCT) Explore relationships between variables and recognise complexity of systems and relationships (eg St Mungo’s work) Can produce very persuasive evidence Cheaper and can produce richer insights into cause and effect
- 16. Experimental versus exploratory approaches : Middle ground Look when each approach is appropriate. Perhaps experimental approaches appropriate when sufficient exploration carried out and in very stable environments Experimental Exploratory
- 18. Comprehensive versus pragmatic: Middle ground Comprehensive provides good intellectual framework but measurement in practice must be pragmatic. Walk first, run later! Comprehensive Pragmatic
Notas del editor
- Neil introduces himselfJoy introduces herselfJoy speaks to the aims slideJoy speaks to her slidesNeil takes over on the RCT slide Our intention is for the presentation to last 30 mins (15 mins Joy and 15 mins Neil) and take questions of clarification as we go and then have 30 mins for discussion at the end. We thought we would start that by asking everyone to speak to their neighbour aboutWhat struck me mostThe question I would most like to ask
- Commissioners, politicians and social investors tend to be in the Direct Measures camp. Practitioners and managers are more likely to be in the Indirect Measures camp.
- Examples of tools for standardising professional judgement areStrengths and Difficulties Questionnaire (Children)Honos (mental health)Examples of tools for measuring subjective experience areCORE (counselling)SOUL record (soft outcomes such as confidence in a range of settingsOutcomes Star is a collaborative measure – incorporating both professional and client view and with scales to moderate and standardise scoringsCan use research to link intermediate outcomes to end outcomes so that routine management practice can focus on intermediate outcomes
- Star as an example of a tool that brings objectivity into assessment of indirect outcomes through standardising judgements
- Brief explanation of how Matrix works
- Importance of regulation and quality if carrying out benchmarkingLearning St Mungos have drawn from Star data
- At present many/most initiatives do not have sufficiently full understanding of key success factors to make RCT an effective use of (personal view point)Refer to Sue Holltum’s critique of a RCT on Art Therapy
- Concept of “good enough” data and analysis