Learning Unbound: Evidence-based Design and Education’s Third Horizon Candic Thille, WASC ARC 2011
•Download as PPT, PDF•
2 likes•1,211 views
One of the most powerful features of technology-enhanced learning environments is that they allow us to embed ongoing formative assessment and feedback into instructional activities. Using intelligent tutoring, virtual laboratories, simulations, and frequent assessment and feedback, the Open Learning Initiative at Carnegie Mellon University supports flexible and responsive instruction that fosters learning. As students work through OLI courses, we use technology to collect real-time data that informs four positive feedback loops: feedback to students, to instructors, to course designers, and to learning science researchers. The results are promising. Our experience shows that educational technology can make higher education less expensive and more accessible while increasing effectiveness -- breaking the iron triangle - while serving greater numbers of students who bring enormous variability in their background knowledge, relevant skills and future goals. In fact, learning technology may be essential, if we are to meet President Obama's goal to raise the nation's college graduation rate to 60% by 2020.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (20)
Similar to Learning Unbound: Evidence-based Design and Education’s Third Horizon Candic Thille, WASC ARC 2011
Similar to Learning Unbound: Evidence-based Design and Education’s Third Horizon Candic Thille, WASC ARC 2011 (20)
More from WASC Senior
More from WASC Senior (15)
Recently uploaded
Recently uploaded (20)
Learning Unbound: Evidence-based Design and Education’s Third Horizon Candic Thille, WASC ARC 2011
- 1. Learning Unbound: Evidence-based Design and Education’s Third Horizon Candace Thille Director, Open Learning Initiative
- 2. “ Changing circumstances mandate that we shift the focus of higher education policy away from how to enable more students to afford higher education to how we can make a quality postsecondary education affordable .” - Clayton Christensen
- 3. One Complexity for Instructional Productivity: Baumol’ s “Cost Disease” “ A half hour horn quintet calls for the expenditure of 2.5- man hours in its performance, and any attempt to increase productivity here is likely to be viewed with concern by critics and audience alike.” (1967)
- 4. A false dichotomy for post-secondary education Quality Productivity Many, with Baumol assume higher education has only these two options Low Quality, Hi Productivity Hi Quality, Hi Productivity Low Quality, Low Productivity Hi Quality, Low Productivity
- 6. Basic Premise It’s not teaching that causes learning. Attempts by the learner to perform cause learning, dependent upon the quality of feedback and opportunities to use it. Grant Wiggins President, Center of Learning Assessment
- 8. Goal-Directed Practice and Targeted Feedback
- 10. Practice Synthesizing and Applying Skills & Knowledge
- 11. Feedback: Changing the Productivity of Learners and Teachers
- 15. “ The Killer App” Feedback Loops for Continuous Improvement
- 16. Learning Curve Analysis DataShop: Pittsburgh Science of Learning Center
- 21. Strategy for Educational Improvement
- 22. “ Without a complete revolution…in our approach to teaching…we cannot go beyond (current levels ) of productivity… . ” (Baumol, 1967). Our message: Such a revolution is possible. Our question: Who will lead it?
- 27. CCOER Open Textbooks Global Higher Education OER
- 29. Scientifically Designed, Interactive Learning Environments
Editor's Notes
- This is a fixed Carnegie Mellon University image. Main title: 40 pt. Times, Upper & lower case. Presenter: 16 pt. Helvetica oblique Name: 16 pt. Helvetica Bold Presenters Title: 16pt. Helvetica light
- The mini tutors used throughout OLI courses such as the ones we just saw in the Engineering Statics course, are built on the 20 years of work that has been done at Carnegie Mellon on cognitive tutors. The mini-tutors in OLI courses are not full cognitive tutors in that they do not have full production rule sets or student models but their behavior is similar to a cognitive tutor for the given problem they are intended to tutor.
- This is a learning curve graph of the knowledge components in the stoichiometry tutor (OLI Chemistry Course). The Y-axis is the “Assistance Score” (the number of hints the students requests and the number of incorrect answers they select) for the selected knowledge component and the X-axis is the opportunity number. The curve is trending downward which means that the students needed fewer hints and gave fewer incorrect answers as they progressed through the material; shows that learning is occurring. This is a graph for all knowledge components together. The tool also allows us to look at graphs for each individual knowledge component and to identify knowledge components such as “set denominator value of Avagadro’s number” which may show a learning curve that is not trending so neatly downward and may indicate a need for revision of the teaching approach.
- OLI evaluation efforts have investigated OLI courses’ effectiveness not only in stand-alone mode, but also in an instructor-led “accelerated learning” mode. This type of study owes its origins to Ben Bloom’s mastery learning concept and the subsequent accelerated schools program. The most common dependent measure used in such studies is time, i.e. the time it takes a learner to complete a particular amount of material, with proper assessment of equivalent learning outcomes. In these studies of OLI courses, we have demonstrated accelerated learning by showing that a learner can complete a semester-long course in significantly less than a semester and/or that a learner can complete significantly more than a semester’s worth of material within a semester’s time. Results showed that OLI-Statistics students learned a full semester’s worth of material in half as much time and performed as well or better than students in traditional instruction. Two studies conducted at Carnegie Mellon tested whether learners using the OLI course in hybrid mode—that is, students meeting with instructors regularly, but less frequently than in traditional courses, while also using the online modules and assignments of OLI- Statistics—would learn the same amount of material in a significantly shorter time than students in traditional class formats. Results exceeded expectations: OLI-Statistics students completed the course in 8 weeks with 2 class meeting per week, while traditional students completed the course in 15 weeks with 4 class meetings per week. Significantly, student logs showed that the OLI students spent no more total time studying statistics outside of class than the traditional students. Yet the OLI students demonstrated as good or better learning outcomes than the traditional students. Further, there was no significant difference in retention between OLI students and traditional students in tests given 1+ semesters later.[i] Usually, that kind of effectiveness or efficiency effect would be expected only as the result of individualized, human-tutored instruction. And yet in this case, students who met for less than two hours of class per week demonstrated phenomenal performance. [i] M. Lovett, O. Meyer, & C. Thille, C., “The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning ,” Journal of Interactive Media in Education (2008), http:// jime.open.ac.uk/2008/14/
- Evaluators internal and external to the project have conducted numerous studies of the effectiveness of OLI environments in supporting student learning. Several studies collected empirical evidence of the instructional effectiveness of the OLI courses in stand-alone mode, as compared to traditional instruction. In all cases, in-class exam scores showed no significant difference between students in the stand-alone OLI course and students in the traditional instructor-led course. In a study of the OLI logic course at a large state university [i] the OLI online-only students covered more material than the traditional students and there was no significant difference in performance between the two groups on exams. The most interesting result from this study is that attrition in the online course was almost non-existent, whereas the in-class condition had very high attrition (although at typical levels for this large, rigorous course at this institution). In the OLI online-only condition 84 students started the course and 83 students successfully competed it, showing a 99% retention rate. In the traditional face-to-face condition, 259 students started the course and only 105 students successfully completed it, showing a 41% successful completion rate. This result is important for two reasons: First, this differential attrition rate likely produces very significant performance biases in favor of the in-class condition on the final exam and yet the in-class condition did not perform better on the final exam. Second, it shows that another potential benefit of using OLI in the online-only mode is that it can significantly reduce attrition rates. [i] C. D. Schunn and M. Patchan, “An evaluation of accelerated learning in the CMU Open Learning Initiative course ‘Logic & Proofs,’” Technical Report by Learning Research and Development Center , University of Pittsburgh, (2009).
- Learning science is still young – while, as practitioners, we say we “know what works” our descriptions of “what works” are complex exemplars that are challenging to replicate and scale and even when replicated and scaled often do not “work” in the new context or for the new population. To replicate and scale we need to be able to describe what works as a set of underlying mechanisms that are influenced by a set of student and contextual variables. In other words we need better learning science to create better theories of learning that inform our practice. To get better theories, we need more data from more students in more contexts. Ed tech provides us with the ability to gather that data. Virtuous cycle: Ed tech -> data -> theory -> Ed tech
- The economist, William Baumol said: “Without a complete revolution in our approach to teaching there is no prospect that we can ever go beyond (current) levels [of productivity] (or even up to them) with any degree of equanimity.”[i] We believe that, using OLI strategies to develop, assess, deliver, and iteratively improve courses, we have in place the key elements to the revolution Baumol sought. This revolution will allow us to disrupt the Baumol/Bowen effect and make higher education less expensive and more accessible without sacrificing quality. [i] Baumol, “Macroeconomics of Unbalanced Growth: The Anatomy of Urban Crisis,” (op. cit.)
- Our newest project, CC-OLI, is a Collaboration with Community Colleges to use the OLI development and evaluation model to address the challenges of post secondary success.
- Here is the model of the CC-OLI structure
- Our newest project, CC-OLI, is a Collaboration with Community Colleges to use the OLI development and evaluation model to address the challenges of post secondary success.
- information and communication technologies can now be used to provide meaningful, actionable feedback to students, instructors, instructional designers, and learning scientists that simply is not available in the traditional teaching as a “ solo sport ” model. Thus far these technologies have not been widely used for such purposes. When they are, the long hoped for transformational impact of technology on education becomes a reality..
- The OER movement, largely funded by the William and Flora Hewlett Foundation spurred an international movement to equalize access to knowledge. The aims of The William and Flora Hewlett Foundation's Open Educational Resources (OER) program in 2007 were stated as to: 1. Sponsor High-Quality Open Academic Content; 2. Break Down Barriers to Open Educational Content; and, 3. Encourage People Worldwide to Use Open Educational Resources. The resulting OER movement has been successful in promoting the idea that knowledge is a public good, expanding the aspirations of organizations and individuals to publish Open Educational Resources. Source: Powerpoint presentation by Catherine M. Casserly and Victor V. Vuchic of The William and Flora Hewlett Foundation given at the Opening Learning Symposium in March 2008. I modified the slide to add 4 additional projects: CC-OER Open Textbooks, NROC, Wiki Educator and Connexions.