National university assessment process
•Descargar como PPTX, PDF•
1 recomendación•747 vistas
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a National university assessment process
Similar a National university assessment process (20)
Último
Último (20)
National university assessment process
- 1. Assessment Mary Hazzard, Ph. D. Assessment Fellow
- 2. The learner will be able to: Describe the NU Assessment Plan Identify if a PLO is to be introduced, developed, or mastered in a course. Differentiate between course evaluation/grading and program assessment. Identify valid signature assignments Align signature assignments with Program Learning Outcomes (PLO) Learning Outcomes
- 3. Ensure evidence-based decision making Ensure high quality programs Ensure students are learning Ensure high quality graduates Encourage programmatic improvement Demonstrate adherence to adopted standards Provide evidence of accountability to the public and the profession Provide evidence for resource and budget allocation Purpose of Assessment
- 4. To insure that we know what are students learn Provide alignment across the mission and through the Institutional Learning Outcomes, Program Learning Outcomes, to the alignment of institutional resources Includes activities designed to achieve the National University’s educational objectives Transparent and collaborative process (AMS) NU Assessment Plan
- 5. What do we want students to learn? (PLO’S) What evidence do we use to assess their learning? (Plan) How well are they learning? (Findings) So what? (Reflection and recommendations) Now what? (Plan for Improvement) Reflection on Assessment Process Closing the Loop (Status of proposed recommendations) Program Annual Report
- 6. 1. Apply information literacy skills necessary to support continuous, lifelong learning. 2. Communicate effectively orally and in writing, and through other appropriate modes of expression. 3. Display mastery of knowledge and skills in a discipline. 4. Demonstrate cultural and global awareness to be responsible citizens in a diverse society. 5. Demonstrate professional ethics and practice academic integrity. 6. Utilize research and critical thinking to solve problems. 7. Use collaboration and group processes to achieve a common goal. Catalog 2011 page 20 Institutional learning Outcomes (ILO)
- 7. Identify the knowledge, skills, and dispositions (values) expected of all graduates of a program Stated in behavioral, measurable terms Describe general expectations for graduates Consistent with the University Mission and Institutional Learning Outcomes Consistent with state/national standards for discipline, if appropriate Revised or developed in CurricUNET Program Learning Outcomes
- 8. Included in course syllabus May be introduced, developed or mastered in a specific course. Introduced in an early required course Developed through several required courses Mastery expected prior to graduation Program Learning Outcomes
- 10. BS in Information Technology Management
- 11. Triangulation of Data Description Example Process of using three points of data that say the same thing when making educational decisions about a PLO. Any single assessment score is subject to environmental or other influences which can affect its accuracy. 2 Direct measures I indirect
- 12. Key assessment instruments used to make decisions about graduate performance/program effectiveness. Provide aggregated data from the identified assessments. Disaggregate for onsite and online and other appropriate options Assessment: Direct Measures
- 13. Portfolio of student work throughout program Evaluation of fieldwork/practicum/clinical practice Demonstrations/presentations Performance assessment Signature assignments embedded in courses Oral presentation Virtual assignments Case analysis Capstone project Types of Direct Measures
- 14. Degree to which an assessment instrument accurately assesses the specific concept/PLO that the individual is attempting to measure. Types of validity Face Validity Criterion related validity Construct Validity Content Validity Validity
- 15. Assessments that gather data on perception of program effectiveness Post program surveys (Alumni surveys) Employer feedback Exit survey (Last class in program) Focus groups Interviews Reflection journal Assessment: Indirect Measures
- 16. Assignment embedded into a course and used for assessment purposes Must be used in all classes ( online, on site, hybrid) the same way. Method for evaluation must be consistent and reliable across evaluators. Two ways to handle: 1. Use different rubric for assessment 2. Use same rubric for both grading and assessment. Signature Assignments
- 17. Remove identifying student and faculty information Reviewers analyze work for specific PLO. Develop specific criteria for targeting the PLO. Focus is on assessment rather than grading. Assessment Only
- 18. Develop common scheme for assessing the elements of signature assignments Conduct assessment as assignment graded. Pool assessment data across courses. Grading and Assessment
- 19. Check with the lead faculty first Obtain any description of the signature assignment Obtain the rubric if this has been developed Provide instructions in the course about the signature assignment for students so they know how the assignment will be used. Provide students with the rubric so they can self assess their work and know how it will be graded/assessed. Provide direction for faculty who will be teaching the course about use of rubric. Course Developer Action
- 20. Make sure the assignment and rubric are in online, eCompanion and Hybrid class. Work with OIRA and SPL to implement LOM Map the assignment and rubric to the PLO and CLO if appropriate Provide direction to the faculty who will be teaching the course about how to manage the gradebook in eCollege. Review the rubric module if you will be the one to develop the rubric if one has not been developed. Additional Action
- 21. Inter-rater reliability Extent to which two or more individuals (coders or raters) agree. Addresses the consistency of the implementation of a rating system. Training, education and monitoring skills can enhance inter-rater reliability. Test how similarly people categorize items Test how similarly people score items.
- 22. Determine if a PLO is introduced, developed or mastered in the course. Check with lead faculty about signature assignment and rubric Provide student with information about the use of the signature assignment and rubric Provide faculty with information about signature assignment and rubric. Ensure the signature assignment and rubric are included in all online, eCompanion, and Hybrid classes. Work with OIRA and SPL to implement LOM. Summary
Notas del editor
- This module will discuss program assessment of the student’s ability to demonstrate mastery of the program learning outcome.
- Learning outcomes have been developed for this module and include: The learner will be able to: Describe the NU Assessment PlanIdentify if a PLO is to be introduced, developed, or mastered in a course.Differentiate between course evaluation/grading and program assessment.Identify valid signature assignmentsAlign signature assignments with Program Learning Outcomes (PLO)
- The purpose of assessment includes the following among other:Ensure evidence-based decision makingEnsure high quality programsEnsure students are learningEnsure high quality graduatesEncourage programmatic improvementDemonstrate adherence to adopted standardsProvide evidence of accountability to the public and the professionMany have stated that if institutions do not provide evidence of accountability then external stakeholders may proscribe what will be used to demonstrate this accountability. We have seen this occur in K-12 education with the advent of required testing.
- The program annual report is designed to assess the program learning outcomes and provide evidence of closing the loop with recommendations and activities. Questions that guide the PAR review include:What do we want students to learn? (PLO’S)What evidence do we use to assess their learning? (Plan for assessment)How well are they learning? (Findings)So what? (Reflection and recommendations)Now what? (Plan for Improvement)Reflection on Assessment ProcessClosing the Loop (Status of proposed recommendations). Each year the PAR includes what has been done about recommendations from the prior year as well as progress on the MOA.
- The institutional Learning Outcomes IILO) were developed to define outcomes expected of graduates from National University. The ILO were developed from the Mission statement as well as from items found in many programs. The University has had ILO’s since 2008. There are 7 ILO’s and they are:1. Apply information literacy skills necessary to support continuous, lifelong learning. 2. Communicate effectively orally and in writing, and through other appropriate modes of expression.3. Display mastery of knowledge and skills in a discipline.4. Demonstrate cultural and global awareness to be responsible citizens in a diverse society.5. Demonstrate professional ethics and practice academic integrity.6. Utilize research and critical thinking to solve problems. 7. Use collaboration and group processes to achieve a common goal.The key concepts have been highlighted in red and include: information literacy; orally, writing, appropriate modes of expression; knowledge and skills; cultural and global awareness; professional ethics and academic integrity; research and critical thinking; and collaboration and group processes. Program leads have mapped the PLO’s to the ILO’s.
- Program learning outcomes (PLO) identify the knowledge, skills, and dispositions (values) expected of all graduates of a program and are stated in behavioral and measurable terms. The PLO describe the general expectations for graduates and are consistent with the University Mission and Institutional Learning Outcomes (ILO). The PLO should also be consistent with state/national standards for the discipline, if appropriate. The PLO must be revised or developed in curricUNET, be approved by all appropriate parties and then are included in course syllabi.
- Program learning outcomes are included in the course syllabus and may be introduced, developed, or mastered in a specific course. Normally the PLO is introduced in an early required course and then is developed through several required courses. Mastery is expected prior to rgaduation and is typically expected in the capstone or final course at the end of the program.
- Here is an example of the curriculum map for the MA in Strategic Communication and you can see that the PLO’s are introduced in early courses and then developed in additional courses and mastery is expected near the end of the program. There should be sufficient opportunities in a program for the student to be able to develop and demonstrate mastery of the knowledge, skill, or value identified in the PLO. This example has been taken from a PAR report in AMS. A curriculum map is developed each time a program is developed or revised in curricUNET. The next slide will provide an undergraduate example.
- In this baccalaureate example you will note that in the first course the PLO are all introduced and then developed through succeeding courses in the program until toward the end of the program mastery is expected. All courses have at last one or more PLO in the introduced, developed, or mastered level.
- All PLO’s should be have a triangulation of the data used for assessment purposes. The three sources of data should say the same thing when educational decisions about a PLO are being made. Any single assessment score is subject to environmental or other influences which can affect accuracy. NU requires the triangulation of data and this must include 2 direct measures and at least one indirect measure.
- Direct measures are those key assessment instruments used to assess student ability to meet the PLO and results are used to make decisions about the graduate performance and program effectiveness. Date is obtained from a sample of courses across teaching modalities and the data is aggregated, analyzed, and decisions and recommendations made. In addition, data may be disaggregated for onsite and online and other appropriate options dependent on where and how the program is taught.
- There are a variety of types of direct measures that may be identified by the prorgam lead faculty and some include:Portfolio of student work throughout program Evaluation of fieldwork/practicum/clinical practice Demonstrations/presentations Performance assessment Signature assignments embedded in coursesOral presentationVirtual Assignments Capstone projectAny assessment measure must be a valid instrument and meet the research requirements for validity.
- Assessment instruments should be valid measures of the PLO. Valid measures accurately assess the specific concept/PLO that the individual is attempting to measure. The types of validity most commonly used include face, criterion related, construct, and content validity.
- Indirect measures gather data on the student/graduates perception of the program effectiveness. Some types include but are not limited to: Post program surveys (Alumni surveys)Employer feedbackExit survey (Last class in program)Focus groupsInterviewsReflection journal
- Signature assignments are assignment embedded into a course and used for assessment purposes. A signature assignment must be used in all classes ( online, on site, hybrid) the same way and the method for evaluation must be consistent and reliable across evaluators. Normally a rubric is used in evaluating the student work.There are two ways to handle and this is generally determined by the program lead. The ways are:1. Use different rubric for assessment. Here the actual student data is aggregated and a different group of faculty will evaluate.2. Use same rubric for both grading and assessment. In this instance the student work is assessed once by the instructor and this is then used for both grading and assessment.
- When the student’s work is to aggregated for assessment purposes only both student and faculty information should be removed. Reviewers analyze the work for the specific PLO using an assessment only rubric. In this instance the focus is on assessment rather than grading.
- If the assignment is to be used for both grading and assessment and this is generally what faculty are using then a common scheme for assessing the elements of the signature assignment are developed in a rubric and the assessment is conducted at the same time as the assignment is graded. Assessment data is then pooled across courses.
- What should the course developer do to learn if there is a signature assignment in the course they are to develop and then what action should they take. First Check with the lead faculty if you are uncertain if a signature assignment should be in the course. If there is to be a signature assignment then obtain the description of the assignment but you may have to refine and provide some additional information for the student. Provide instructions in the course about the signature assignment for students so they know how the assignment will be used. Obtain the rubric to be used for assessment and grading if it has been developed and provide students with the rubric so they can self assess their work and know how it will be graded/assessed. You should also provide direction for the faculty who will be teaching the course about the use of the rubric. The rubric should be used consistenly by faculty and faculty should be calibrated on the use and inter-rater reliability established.
- Addition actions include making sure the assignment and rubric are in online, eCompanion and Hybrid class templates. You will also need to work with the Office of Institutional Research and Assessment (OIRA) and Spectrum Pacific Learning (SPL) to implement the Learning Outcome Management (LOM) which will allow for the data to be aggregated and used for assessment in an orderly fashion. Map the assignment and rubric to the PLO and CLO if appropriate as this will allow the assessment data to be aggregated and disaggregated for the PLO and the results used in the Program Annual Report. Provide direction to the faculty who will be teaching the course about how to manage the gradebook in eCollege.Review the rubric module if you will be the one to develop the rubric if one has not been developed. For assessment purposes the faculty who will be using the rubric to assess should be calibrated on use of the rubric and inter-rater reliability established.
- Inter-rater reliability is the extent to which two or more individuals (coders or raters) agree. It addresses the consistency of the implementation of a rating system and tests how similar people categorize items and score items.Training, education and monitoring skills can enhance inter-rater reliability.
- In summary:Determine if a PLO is introduced, developed or mastered in the course.Check with lead faculty about signature assignment and rubricProvide student with information about the use of the signature assignment and rubric Provide faculty with information about signature assignment and rubric. Ensure the signature assignment and rubric are included in all online, eCompanion, and Hybrid classes.Work with OIRA and SPL to implement LOM.