Rethinking Evaluation: its value and limits in corporate learning. From Learning Technologies 2008 conference

1. Rethinking evaluation: its value and limits in learning David Wilson [email_address] Europe’s leading Corporate Learning Analysts

2. ROI Evaluation Value Measurement

3. ROI Evaluation Value Measurement Learning

4. ROI Evaluation Value Measurement Learning (ALL)

8. So What Should We Do About it? ROI Evaluation Value Measurement

9. ROI Evaluation Value Measurement Outcomes Processes

11. Orientation B&Q University “The way we work” To gain an understanding of the business context To explain the role of the university and how we can support To explain the University process and agree expectations To agree the budget and resources available To agree next steps within process Value of Learning To complete the agreed activity to measure the value of the learning solution Orientation Design the solution To design the solution using university design methodology To agree the sustainable training solution Learning Context Why Learn? To blend or not to blend? Define the desired business outcomes To define the desired business outcomes To gain a view of the current performance To analyse the learning context To agree the way we will measure the value of learning To establish the wider business / change context within which the learning will fit To agree next steps with process Planning & Set up To agree the resources required To plan the development of the team to deliver To define and agree the administration process To plan the logistics of delivery To establish reporting requirements To plan the trial / pilot and how feedback will be captured To test out the delivery of the solution To capture the feedback and learning from the trial / pilot To agree the improvements required and plan for this to take place Trial / Pilot Review To complete the PIR activity (for projects) To ensure the sustainable training is set up To establish how the learning is being put into action within the business Delivery and Action To deliver the solution as agreed Identify the learning needs To discover the learning needs of all job roles To agree the next steps in the process

14. Impact of Learning – Our Basic Model Knowledge Application Performance Activity Effectiveness Value How many KNOW it? How many DO it? How it changes PERFORMANCE How effectively they KNOW it? How effectively they DO it? How effectively does it change PERFORMANCE The value of KNOWING The value of DOING The value of the PERFORMANCE change

15. Impact Framework (Research Version) Learning Process Learning Process Motives/ Expectations Pre- Capability Post- Capability Commitment Post Knowledge/ Skills Pre Knowledge/ Skills Effectiveness Internal Factors Reward Environment Time Support Motivation Goals Application Application External Factors Competition Market Customer Time Budget Goals Performance Performance Learning Process Post Knowledge/ Skills Pre Knowledge/ Skills Value Application Performance Learning Process Post Knowledge/ Skills Pre Knowledge/ Skills Activity Application Performance Analysis Design

16. IOL Process Outline Analysis Measurement Design Influencing Factors Delivery Review

19. Measuring and Reporting Impact Defined by project Often Impact (L4) Learning (L2) Behaviour (L3) & Learning (L2) Level of Assessment Aggregated (Benchmarked) Addressed specific learning needs Individual On Demand Directly (Outcomes) Organisationally (Human Capital) Valuation Meet specific project learning goals Performing role with requisite knowledge/skills Success The Business Project Organisation Driver Project-Driven Core Competency Valuing the Learning Portfolio

20. Questions? [email_address] Research Knowledge Base: http:// research.elearnity.com