"Whenever a thing is done for the first time, it releases a little demon": Pilot Strategies for Blended Learning Programs
•
2 recomendaciones•1,032 vistas
Co-presented with Roxanne Russell at the 2014 Online Learning Consortium International Conference (October 30, 2014) ABSTRACT: We are in a transitional time for online learning at traditional higher education institutions- passing from a period of majority resistance to online learning and into a period of either acquiescent or enthusiastic acceptance of online learning as an inevitable disruption to traditional practices (Selingo, 2013). Generational, social and economic drivers are prompting more higher education administrators and faculty to offer traditional programs using online and hybrid models. Just as the approach to online course development requires more front-end comprehensive planning and development than traditional face-to-face courses, program level design requires a more systematic approach to cross course consistency and reliability. Pilot testing for usability and feasibility is crucial to the design process for online programs and introduces a unique set of challenges and opportunities to improve student learning and experiences. This session will share the design, experiences, interview and survey feedback, and learning analytics gathering process from a pilot conducted for a new online doctoral program, the first online doctoral program of any kind at Emory University and the first online version of this particular degree to be nationally accredited.
Recomendados
Recomendados
Más contenido relacionado
Último
Último (20)
Destacado
Destacado (20)
"Whenever a thing is done for the first time, it releases a little demon": Pilot Strategies for Blended Learning Programs
- 1. “Whenever a thing is done for the first time, it releases a little demon.” Pilot Strategies for Blended Learning Programs Emory University ~ L. Roxanne Russell ~ Timothy D. Harfield #aln14 #demonpilot
- 2. Interactive Session Prompt On Twitter use #aln14 #aln7717 & #demonpilot How have you used traditional data collection methods or learning analytics to inform course design and delivery? What is the relative importance of student perception and student behavior in informing course design and delivery? #aln14 #demonpilot
- 3. D Min Pilot Overview #aln14 #demonpilot
- 4. Process • One D Min faculty member and 10 accepted D Min students • One typical course week of activities with primary tools: Blackboard, Adobe Connect, Echo360 & Voicethread • Data Collection – Survey – Individual Interviews – Focus Group Interviews – Learning Analytics • Data analyzed for design-related program implications and to inform programmatic student support infrastructure, student orientation, administrative training and faculty development #aln14 #demonpilot
- 5. Student Perceptions #aln14 #demonpilot
- 6. Student Perceptions Wanted to work with the named Professors Needed convenience of online format and no long residency Knew if Candler was doing it, they would do it well. Had met and been impressed with high quality Candler Alums Really interested in the focus of tracks #aln14 #demonpilot
- 7. Student Perceptions How long are the synchronous sessions? Live and die by my calendar Felt so alone when audio wasn’t working in the session Made myself use the digital version of the readings #aln14 #demonpilot
- 8. Insight into fulfilling student expectations • Caliber of faculty – Originality of perspective- scholarship – Personal exposure – Charisma/personality • Prestige of institution – Quality of environment (facilities/interface/tools) – Quality of service & support – Caliber of and access to peer network • Challenge of curriculum – Relevance and level of content – Worthwhile course activities – Scaffolded project #aln14 #demonpilot
- 9. Student Behavior Access Summary • Students tend to leave assignments to last day possible DAY 1 DAY 2 DAY 3 DAY 4 DAY 5 DAY 6 DAY 7 Monday Tuesday Wednesday Thursday Friday Saturday Sunday Monday Tuesday Wednesday 5-May-14 6-May-14 7-May-14 8-May-14 9-May-14 10-May-14 11-May-14 12-May-14 13-May-14 14-May-14 Read Miller-McLemore, "Introduction" Review Materials as needed Review Materials as needed Read Moschella, "Ethnography" Review Materials as needed Create a Blackboard Profile Attend (ordinarily student-led) Synchronous Session: Colloquy Post your response on the Voicethread Discussion Post Introduction in "Getting to Know You" Discussion Continue working on "Designing a Research Question" Assignment Submit "Designing a Research Question" Assignment Attend faulty-led Sunchonous Session: What is a Practical Theologian? View Lecture: Cultivating Practical Theological Sensibilities Post your Course Roundtable Discussion Question (DQ) Response Begin "Designing a Research Question" Assignment Post your replies to classmates' DQs Begin "Designing a Research Question" Assignment Course Accesses 1 6 10 2 11 1 14 11 7 2 Unique Users 1 6 8 1 7 1 3 9 5 5 Total Item Accesses 4 68 93 6 170 3 58 94 147 514 Total Minutes in Course 1.57 622.43 1139.35 10.37 937.45 2.50 121.28 513.90 514.12 273.77 Avg Minutes in Course (per user) 1.57 103.74 142.42 10.37 133.92 2.50 40.43 57.10 102.82 54.75 #aln14 #demonpilot
- 10. Student Behavior Access Summary • Students who access the course the most, are not necessarily the most active 30 25 20 15 10 5 0 Student A Student B Student C Student D Student E Student F Student G Student H Student I Student J Accesses #aln14 #demonpilot
- 11. Student Behavior Discussion Forum • Standard personal introductions do not foster community, but decentralized assignments may lead to differences in integration DMin Pilot DMin 500 #aln14 #demonpilot
- 12. Student Behavior Discussion Forum Course Roundtable INFLUENCERS generate more posts from others than they contribute CONNECTORS may generate relatively few posts, but serve to integrate others into the community #aln14 #demonpilot
- 13. Student Behavior • Opportunity to advise students on effective strategies for interacting with asynchronous video content #aln14 #demonpilot
- 14. Insight into doctoral level program evaluation • Acknowledge that students are advanced level professionals- typical approaches to measuring student success or performance may not be appropriate • Recognize the assessment value of student expectations and perceptions for their learning experiences in doctoral level graduate programs • Prompted holistic examination of context, purpose, and questions for integrating learning analytics into program evaluation • http://www.slideshare.net/rockirussell/d-min-programeval #aln14 #demonpilot
- 15. Next time • Authentic sample useful for gathering data but also for preparing 1/3 cohort prior to orientation • False sense of security from remote students- technology dependency and anxiety much more palpable when on campus • Refine framework for data collection-survey questions, interview questions, and learning analytics #aln14 #demonpilot
Notas del editor
- http://onlinelearningconsortium.org/conference/2014/aln/whenever-thing-done-first-time-it-releases-little-demon-pilot-strategies-blended The Challenge We are in a transitional time for online learning at traditional higher education institutions- passing from a period of majority resistance to online learning and into a period of either acquiescent or enthusiastic acceptance of online learning as an inevitable disruption to traditional practices (Selingo, 2013). Generational, social and economic drivers are prompting more higher education administrators and faculty to offer traditional programs using online and hybrid models. Just as the approach to online course development requires more front-end comprehensive planning and development than traditional face-to-face courses, program level design requires a more systematic approach to cross course consistency and reliability. Pilot testing for usability and feasibility is crucial to the design process for online programs and introduces a unique set of challenges and opportunities to improve student learning and experiences. This session will share the design, experiences, interview and survey feedback, and learning analytics gathering process from a pilot conducted for a new online doctoral program, the first online doctoral program of any kind at Emory University and the first online version of this particular degree to be nationally accredited. The Context In 2013, Emory University's Candler School of Theology decided to revive the Doctor of Ministry (DMin) degree program that had run for over twenty years before it was discontinued in 1995 because of financial pressure, declining enrollment and demands on faculty time. In response to a steady stream of requests to re-institute the degree, the leadership decided to revamp the program as a 90% online degree. By taking this innovative step, Candler stands out at Emory University because the DMin program will be the first 90%online doctoral degree program on campus. This DMin degree is also the first of its kind to be accredited by the Association for Theological Schools, which approves degree programs for its 270 member association of "post-baccalaureate professional and academic degree programs to educate persons for the practice of ministry and for teaching and research in the theological disciplines." Furthermore, before making the decision to offer a new online program, Candler's faculty had only just begun to experiment with online learning components and had never offered even one class fully online. After an active year of faculty and administrative collaboration and program design and development, we knew a pilot test of the program was essential to ensure a program launch geared towards faculty and student success. The Approach To gather usability and feasibility data from an administrative, faculty and student perspective, we took the following approach: Designed a pilot course to include all activities and tools of a typical week of program courses in the pilot course Recruited faculty and administrators who would be active in the program in the first year to help build and deliver the pilot course Offered the pilot to a volunteer subset of newly admitted students to the program six months prior to the program's start Developed learning analytics based on learning management system log data to systematically examine student experience patterns and optimize program and course level design Administered a survey to and conducted follow-up individual and focus group interviews with all pilot participants, including administrators, students and faculty Analyzed the data for design-related program implications and to inform programmatic student support infrastructure, student orientation, administrative training and faculty development The Takeaways In this session, we will share experiences and results from this pilot program and ask for audience input of past experiences, reactions and suggestions. Using a collaborative digital tool, we will invite audience members to actively engage in producing a session debrief towards identifying successful strategies for implementation and evaluation of online program pilot testing. We and the audience should leave this session with a grounded understanding of programmatic level pilot design, usability data gathering techniques and cross-referencing learning analytics to inform design. Program Track: Learning Effectiveness Target audience: All audiences Institutional Level: multiple Type of session: Information Session Media: Digital projector, wireless access
- Timothy- Please post on Twitter as we get started. I will also post from my account. #demonpilot http://neoformix.com/spot/#/%23demonpilot #aln14 http://neoformix.com/spot/#/%23aln14 #aln77117 http://neoformix.com/spot/#/%23aln77117
- https://www.surveymonkey.com/results/SM-BZ9K626/ Cross-referenced with learning analytics to see what technology problems may have occurred, been under reported or idiosyncratic. When you receive feedback on a survey about frustrations, one wonders how widespread is that frustration? Not only listening to squeaky wheels. Tool failure did not extend beyond 1 or 2 people. Log file showed small # of instances of clicking more than twice on the same tool in under 5 seconds– fairly high because of loading; includes the impatients & the home bodies Tech anxiety & obstacles learning
- Program clarifications Communication Course structure Tool integration and functionality Faculty development Student preparation/orientation e.g. Adobe Connect: Adjust to Public or Password setting to access URL for past recordings Need back up method to monitor when screen sharing Need to continue to learn about troubleshooting bandwidth problems Teach patience with mics and cams in orientation Need back-up Conference Call solution
- Students may interact with the site in a number of ways: a) ‘dippers’ - students who log in frequently (averaging more than once a day), but often only check discussion board and announcements b) ‘intentioned’ - students who log in infrequently, but who interact with course resources extensively during that time. *** Students with the highest ratio of interactions to accesses (ex. Student C and Student J) are the least engaged in discussion board. May speak to temperament and/or the need to strategically schedule time to complete assignments because of other commitments. QUESTIONS: ------------------------------------------------------------------------------------ > How does ‘dipping’ and being ‘intentioned’ translate into course performance? Do these represent different, yet equally effective, learning strategies? > How can we motivate ‘intentioned’ students to interact more in the discussion board, while also acknowledging conditions that would drive a more focused approach to course interaction? LEGEND ------------------------------------------------------------------------------------ GREEN = greater than one standard deviation above mean RED = less than one standard deviation below mean
- Dmin Pilot Assignment Description: Please introduce yourselves here. Share some basics: your name, where you're serving, how you got there, and why you've decided to pursue a DMin at this point in your life and vocation. And what is the best thing about your ministry site? Faculty member produced initial post as a prompt With one exception, interactions consist in assignment fulfillment and brief acknowledgment from instructor. Replies are ‘niceties’ that do not encourage further response or connection with other members. Social network is highly volatile, as a single member (instructor) represents the sole point of connection between group members. TAKEAWAY: This is a missed opportunity for engagement. If the goal is to generate community and interaction between members of a particular cohort, and to encourage this as early as possible (and ideally prior to any kind of formal group assignment), then the assignment might be redesigned to foster a more integrated community, sooner. Dmin 500 Assignment Description Please spend about 15 minutes internet stalking your classmate and reply to this forum with a description based on what you found. Including links and images that you've found is welcome and encouraged. Feel free to play around with the features and options for customizing this post in the textbox ribbon. After posting your description, read the others posted and reply to at least 2 by Tuesday 8/20 midnight. Faculty member is taken out of discussion board entirely. A new thread is created by each introduction. Responses are unevenly distributed, with certain posts receiving far more replies than others. It is surprising that in several cases (3) the profilee did not reply to the profiler TAKEAWAY: Unlike the Dmin Pilot, the design for which results in an instable social network, the lack of coherence seen on Dmin creates a pedagogical opportunity, as the extent to which posts find replies appears to be a function of the posts themselves. It would be worth having a follow-up conversation with students, looking at the social network diagram, to talk about the characteristics of posts that saw more engagement versus posts that saw little. This would be an especially productive exercise, considering the pastoral focus of this doctoral program, which sees community building as essential. NOTE: Diagrams produced using SNAPP: Social Networks Adapting Pedagogical Practice (http://www.snappvis.org/)
- Assignment Description: This discussion space is for the purposes of engaging the resources of the course (readings, posted materials, etc.) and integrating these voices with your own wisdom and observations about your sites of ministry. Your first response should be posted by Friday at 11:59 p.m., and you should respond to at least two colleagues by Tuesday at 11:59 p.m. Visualization of discussion board interactions allows for the identification of different forms of engagement: INFLUENCERS – generate more posts from others than they contribute High ratio of In Degree (replies) to Out Degree (posts) CONNECTORS – serve important role in connecting others to the network, despite generating relatively few posts High Betweeness Centrality (the number of shortest paths between discussion participants that pass through a particular individual) - a measure of the importance of an individual to the structure of the network
- 5 of 10 watched it; 3 of 5 viewed by day it was assigned; 2 reviewed before completing final assignment By the time it’s due and in preparing- we put this into the To do list before first session as part of class prep
- Students are expected to succeed Pass/Fail program with substantive faculty feedback Credible judges Institutional impact (reputation, alumni, word of mouth)