2012 course review training (modified)
•Descargar como PPTX, PDF•
0 recomendaciones•493 vistas
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (19)
Similar a 2012 course review training (modified)
Similar a 2012 course review training (modified) (20)
Último
Último (20)
2012 course review training (modified)
- 1. Blackboard Exemplary Course Program Reviewer Training MARCH 1, 2012
- 2. 12/23/10 2012 ECP Submission Timeline Submission window: December 1, 2011 – March 2, 2012 Directors Reviews Meet to Anonymous Conducted Discuss Reviews March19 – Reviews: Sent: Mid April 6 April 10-11 May Review Winners Deadline: Notified: April 6th April 23 2
- 3. 12/23/10 Overview of the ECP Rubric • The ECP Rubric addresses four areas that are considered by this program to be essential in effective, high-quality online instruction: – Course Design – Interaction & Collaboration – Assessment – Learner Support 3
- 4. 12/23/10 1. Course Design Course design addresses instructional design, and the guidelines for this program reference the elements of: • Goals and objectives • Content presentation • Learner engagement • Technology use 4
- 5. 12/23/10 2. Interaction and Collaboration Interaction denotes communication between and among learners and instructors, synchronously or asynchronously. Collaboration is a subset of interaction and refers specifically to those activities in which groups of students are working toward a shared result. Special Note to Reviewers Please be sure to look for evidence of interaction and collaboration in special files or supplemental resources. Submitters may have provided special files and resources in order to hide the identity of students for privacy purposes. 5
- 6. 12/23/10 3. Assessment Assessment focuses on instructional activities designed to measure progress towards learning outcomes, provide feedback to students and instructor, and/or enable the assigning of a grade. 6
- 7. 12/23/10 4. Learner Support Learner Support addresses the support resources made available to students taking the course. Such resources may be accessible within or external to the course environment. Learner support resources should address the variety of topics that are appropriate for learner success in the course. The topics will vary depending on resources that are available and the uniqueness of each course. 7
- 8. 12/23/10 Rubric Layout 1. Course Exemplary Accomplished Promising Incomplete Design Goals and Objectives Content Presentation Learner Engagement Technology Use 8
- 9. What’s the difference between 12/23/10 Exemplary and Accomplished? Exemplary Accomplished Goals and objectives are easily Goals and objectives are Goals and located within the course at all located within the course Objectives appropriate places such as syllabus or the individual syllabus and Individual learning learning units; units; are clearly written at the objectives are written to reflect appropriate level and reflect desired learning outcomes, desired outcomes; are written in measureable although not all are written as outcomes (students know what measureable outcomes; they are expected to be able to do), 9
- 10. 12/23/10 A Note on Visual Appeal of Courses • There is no area of the rubric focused on course look-and-feel or on how “pretty” the course design is • Visual appeal is very personal • Provide constructive feedback on the course look-and-feel as you wish • Do not let visual appeal cloud your judgment or interfere with your review 10
- 11. 12/23/10 What Makes a Review Successful? Completeness: – Thoroughly complete Parts 1 and 2 of the Reviewer Form. Validation: – Carefully review the comments and examples of exemplary practices presented in the narrative components on the submitter’s form to determine whether or not you agree. – Base your review on the detailed criteria of the ECP Rubric. Constructive Feedback: – The ECP program is based on a premise that participation in the program as a reviewer or a submitter will enable individuals in both groups to get better at creating and delivering effective online instruction. – This positive theme should be foremost in the minds of reviewers and should be evident in the feedback you prepare for course 11 submitters.
- 12. 12/23/10 Examples of Feedback Feedback that isn’t very constructive: Too much text on the page. I didn’t like to scroll. 12
- 13. 12/23/10 Examples of Feedback Positive, constructive feedback: It is great that you provide supplementary reading materials within the course itself! This is a real time-saver for students. The materials may be easier for students to read and recall if the items were organized by theme or in chronological order and placed into dedicated folders or subfolders rather than listed on one page. While having all of the information on one page makes it easy for people to print out, it makes for an awfully long page to scroll through. In addition, because of some of the large images, the page takes a long time to load. This might be a problem for students with slow internet/dial-up connections. 13
- 14. 12/23/10 Examples of Feedback Feedback that isn’t very constructive: Learning objectives are not placed in appropriate places. Your learning objectives are not good. 14
- 15. 12/23/10 Examples of Feedback Positive, constructive feedback: Although your learning objectives (LO’s) are included in the course syllabus, it would be helpful to students to relate specific LO’s to course activities and assignments. The learning objectives (LO’s) could be improved if written in measureable terms. For example, in the LO’s below, the word “know” is vague and isn’t measureable. However, the action verb “write” is measureable. Not measureable: The student will know the Spanish alphabet. Measureable: The student will be able to write the Spanish alphabet. 15
- 16. 12/23/10 The Reviewer’s Responsibilities – Part 1 On the Reviewer Form: Rate each of the rubric components by selecting from the drop-down Provide feedback on each of the following four sections: • Course Design • Interaction and Collaboration • Assessment • Learner Support 16
- 17. 12/23/10 The Reviewer’s Responsibilities – Part 2 • On the Reviewer Form enter: – Your overall rating for the course (pull-down) – Your summary to describe your overall impressions of the course. – Your suggestions for course improvements. – A description of the top three “stand-out practices” in the course. • These practices are those that you think are worthy of being shared with the broader educational community. 17
- 18. 12/23/10 Reviewer Checklist Skim the entire submission. Note initial “gut” reactions Read the submission. Review your initial reactions and prepare a response Include the following in your response: Complete sentences Begin and end with a positive statement Include suggestions for improving or enhancing the course 18
- 19. Questions?
- 20. 12/23/10 Agenda • Steps for reviewing and rating courses 20
- 21. 12/23/10 Overview of the ECP Submission Form • There is an online ECP Submission form for each course • This form is a key resource Reviewers use in their evaluation of the course. • The Submission form: – Sets context for the course – Provides evidence as to how the course meets each requirement in the ECP Rubric – Contains course login information 21
- 22. 12/23/10 New ECP Reviewer Form • Each course reviewed must have a completed reviewer form – If you are reviewing as a team please submit one review with all your notes consolidated in it • The new online form contains: – The ECP Submission form – Reviewer Information – Part 1: Feedback using the Rubric: • Course Design • Interaction & Collaboration • Assessment • Learner Support – Part 2: Overall Feedback 22
- 23. 12/23/10 Ready to Review? 1. On March 19th you will receive your course assignments from Sheryn Anthes. Be on the lookout and check your spam filter 2. Create a login and find your assigned courses. Access the online Reviewer Form and review the instructions 3. Evaluate your assigned course (if you have trouble logging in, please email us ASAP) 4. Save your feedback, constructive comments, and evaluation information in the Reviewer Form 5. Deadline to complete reviews: April 6th 23
- 24. 12/23/10 Create an Account and Log In 24
- 25. 12/23/10 Select the Name 25
- 26. Review Their Submission Form and then Select Add Review
- 27. 12/23/10 Access the Course Details to access the course can be found on the “Reviewer Form”. 27
- 29. 12/23/10 Overall Comments 29
- 30. 12/23/10 Help – I’ve got questions! • I can’t login to the course! Email us at community@blackboard.com immediately. • Any other questions - Email us at community@blackboard.com 30
- 31. Thank you!
Notas del editor
- Here is the high-level timeline of the Review Process.You will be sent an email on March 19th with your course assignments. You have 3 weeks to review the courses that have been assigned to you.The deadline is April 6th. You will be sent weekly reminders.In Mid April, the ECP Directors will meet to finalize the list of ECP winners.Winners will be notified later in April. Late May, reviews are sent back to the Course Owners.You will be sent a Certificate of Appreciation for your efforts as a course reviewer. Please be sure to fill out your address in the Reviewer details of the Reviewer form.
- Let’s talk about the Exemplary Course Program Rubric. The rubric addresses four areas of a course that are essential for effective, high-quality online instruction. We’ll cover each of these in more detail in the following slides. Course design Interaction and Collaboration Assessment Learner Support
- Course Design covers the goals and objectives of the course – are they clearly stated for the students? Content presentation – is it well organized and straightforward to find and progress through the materials?Learner engagement – is the course designed to be engaging and appealing?Technology use – online learning can take advantage of a variety of technologies, does the course use these technologies appropriately and effectively?
- Interaction and collaboration can be a significant portion of as well as a critical driver of the learning experience whether it happens between and among learners or between students and their instructor. Interaction can occur synchronously or asynchronously. The rubric asks you as reviewers to determine whether there is sufficient and effective interaction within the online course.Collaboration is a subset of interaction and refers specifically to those activities in which groups of students are working towards a shared result. This may occur as a group project, team wiki page, video presentation, to name just a few examples. Because there are so many different ways interaction and collaboration especially can be incorporated into a course it is very important for you to look for evidence of it. Oftentimes this evidence is submitted as special files or supplemental resources. In addition, student privacy concerns often mean that evidence of student collaboration has been anonymized and provided in a location separate from the course or included within the course in a specially marked folder. This information will be provided to you within your reviewer assignment email.
- The Assessment portion of the ECP Rubric focuses on examining instructional activities designed to measure progress towards the learning outcomes. Assessments also give instructors the opportunity to provide feedback to the student and enable the assigning of a grade. Assessments can be in many forms so please refer to the course submission form for evidence that the course submitter specifically called out as examples of Assessment.
- The Learner Support portion of the rubric addresses the support resources made available to students taking the course. These resources may be included within the online course itself, or linked to from the course, for example a link to the student writing center or library. The types of resources available to the student will vary depending upon the topic taught and the technologies incorporated into the course.
- Each area of the Rubric has a set of sub-areas. In our example the Course Design area has four sub-areas. Each of those sub-areas has a detailed definition of the level of achievement - Exemplary, Accomplished, Promising or Incomplete. Download the ECP Rubric from the website to see the detailed definitions for each area and sub-area. You will use these detailed definitions as you review the course. Let’s go through a quick example.
- Here is the actual text from the ECP rubric for Course Design: Goals and Objectives. We’re just looking at the definitions for Exemplary and Accomplished to give you an idea of how the definitions of achievement become more detailed and demonstrate greater mastery as one moves from Incomplete to Promising to Accomplished to Exemplary.Let’s start with the first sentence in the Exemplary column. Now compare it to the text in the Accomplished column. Do you see the differences between the two? To achieve an exemplary rating the bar is set higher for the course designer. An exemplary course will have goals and objectives that are easily located within the course at all appropriate places, whereas an Accomplished course will just have the goals and objectives somewhere within the course syllabus or individual learning units.If we walk through the sentences below you will see similar differences between the definitions in the two columns. These differences may be subtle. You will need to familiarize yourself with the rubric in advance of your review and keep it close to you as you are reviewing the course so that you can effectively determine how to rate each aspect of the course using the rubric.
- This is a screenshot of a 2009 Exemplary Course Award Winner. Not all ECP Exemplary Courses look like they were designed by experts in Photoshop. Do not let your reaction to the colors selected or icons used sway your review of the course based on the elements of the ECP Rubric. There is no area of the rubric focused on how “pretty” the course design is or how cool the images are. Visual appeal and design sense is very personal. The ECP Directors deliberately left out of the ECP Rubric an evaluation of the prettiness of the course. Instead, we want you to focus on the design of the course from the ability to navigate through the course content, how the course design aids in content organization, and so forth. That said, if you want to provide constructive feedback to the course creator as part of your overall feedback please do so.
- What makes a review a successful one? Three things:Completeness: There are two parts to the Reviewer form – your feedback on each section of the rubric and then your overall feedback on the course Validation: The course submitters will have provided their evidence or examples of what makes their course exemplary as part of their course submission. It is important for you to review these items and decide whether you or not you agree with them. You should base your review of their course using the detailed criteria of the ECP rubric. We’ll cover this in more detail shortly. Constructive Feedback: This is one of the most important aspects of the Exemplary Course Program. The Exemplary Course Program is based on the premise that participation in the program – whether as a reviewer or course submitter, will enable participants to get better at creating and delivering effective online instruction. Constructive feedback is at the heart of this. Please be positive and constructive as you provide feedback to the course submitters.
- If there is one thing to stress here is that your feedback is the most critical element of the review. It provides the justification for your ratings, but most importantly it provides the submitters with necessary information on the types of improvements they need to consider to move their courses to an exemplary standard. Submitters will know where to focus their efforts. The above comment really doesn’t tell us anything. It is framed from a negative viewpoint. The same comments can be made much more positively – keeping comments collegial and tactful – by clarifying with suggestions and examples such as chunking the text into sections, having separators using color coded headings, even the use of page templates to provide consistency. It is easy to be critical – it takes more thought to be constructive.
- Notice the difference in this feedback – starting with a positive comment and then offering suggestions on how to improve. Having it presented in such a manner provided a benefit to the learners – but here are some other suggestions that would provide even more benefits…. so it’s constructive, it’s positive, and here’s why.
- Again, these statements are quite vague – what do you mean by “objectives are not good” or “appropriate places”. These statements lead us only to assumptions – more definitive comments need to be included. In a case such as this, you might want to start out by saying, “while the learning objectives were readily available in the course syllabus…then lead into the suggestions. You also need to keep in mind that not all faculty are instructional designers – and not all of the courses submitted were created by instructional design teams – there are still some lone rangers out there. That’s why sometimes having a team consisting of an instructional designer, faculty expert, etc. is a good approach to the review process. Let’s see what the next slide reads….
- One of the best practices we have noted over the years is the placement of learning objectives in areas other than just the syllabus – allowing for Making them available on content module pages – within assessment activities, etc. as noted above. These types of strategies help students relate the objectives to the activities required of them. And measurable objectives also helps align student expectations – helping them understand what they are going to be required to do. The next comment clarifies the improvements needed in the learning objectives. And an example provided. In this case the reviewer re-wrote an objective as an example.
- You will be required to download the reviewer’s form from the website Jan will provide more instruction on that later. The next couple of slides relate what your responsibilities are with respect to completing that form. One form for each course you review. If you do a team review – then all comments from the individual members of the team need to be compiled into one review form. Select the rating for each component in the rubric. In the example here you are rating the Goals and Objectives section based on your observation and notes you’ve made. At the end of each section, you will be required to enter overall comments on the section summarizing your observation, positively making comments and offering further suggestions where necessary.
- Review and reflect on your ratings and comments for each section – this will help you determine your overall rating. If the majority of your ratings are less than exemplary, then it is unlikely you would give an overall rating of exemplary to the course. Provide your overall concluding comments about the course. Reiterate what the submitter needs to take away – the good and the weak areas noted. Remember – positive and constructive. Then add your overall suggestions for how the course (or various areas) can be improved. Look at the areas you’ve noted as exemplary – they will help inform your best practices for the course. Most courses will have at least three areas that could be considered as being well done. This is the area where you have a chance to praise what is done well that might be worth sharing with the online community. So, consider what was done really well. It does not necessarily mean it will make the final list of best practices once compared to others.
- Carefully review the submitters form what have they said about their course highlight areas you want to validate especially those noted as exemplary by the submitter. But also highlight anything that stands out – whether it’s a design element or a certain adaption of a tool, or certain piece of technology, etc. Note the course level teaching methods varies based on level of course blended delivery may affect level of communication/collaboration the choice of tools and how used will be impacted depending on totally online vs blended knowing this information will influence your choices. Remember: you are essentially a student in the course how oriented do you feel so taking some time to become acquainted with the course prior to start reviewing is important. Can make some initial jot notes during this process. Some courses will take less time – most will take 3 to 4 hours to complete to do a good constructive job. You may need less time as you become more familiar with the rubric and you’ve developed a system for yourself. Sometimes you need to leave a course and come back to it – especially if you are struggling with some of the ratings. You will likely find you need to look at a course a couple of times.
- Each course submitted to the ECP program has a completed nomination form. This form contains their contact information, the ECP Rubric with their notes as to how their course meets each part of the rubric. We’ll take a moment to walk through this rubric together.[switch to live demo of rubric]
- For each course you review you must complete the reviewer form. If you are reviewing courses as a team, please compile your individual reviews into one review form. We want only one review form per course.[switch to live demo of reviewer form]
- Enter your login information.Select the submission you would like to review by pressing "Review" and press "Add Review" to complete the review form.Any attachments will appear next to the "Review" tab. They appear in order of the upload field on the submission form.Press "Save" to submit the review. You can not save drafts.You can then move to the next submission and repeat the process. "Done" will appear next to the submissions you have already reviewed.