Dr. Diane Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy LIS Program University of Hawaii

1. Assessment Tools for Information
Competency
Dr. Diane Nahl
LIS 665 Teaching Information Technology Literacy
Library and Information Science Program
University of Hawaii
Spring 2013

2. Assessment Philosophy
1. Assessment is a continuous feedback process aimed
at understanding and improving student learning.
1. Assessment is designed to enable measuring student
learning outcomes (SLOs) to demonstrate and
provide evidence of learning.
2. Assessment is a strategic planning tool that enables
reporting of progress on short-term and long-term
goals and objectives.
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
2

3. The Assessment Mandate
WASC has identified information literacy as both a
learning ability and a competency:
“These programs also ensure the development of core
learning abilities and competencies including, but not
limited to, college-level written and oral
communication; college-level quantitative skills;
information literacy; and the habit of critical analysis of
data and argument.”
[Italics added] Western Association of Schools & Colleges
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
3

4. The Assessment Mandate
Through its accreditation process, WASC asks each of its
member institutions of higher education to explain how
its library contributes to the learning process , specifically
to developing student information literacy skills, in
addition to providing information services.
Western Association of Schools & Colleges
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
4

5. Foundations Requirement: UHM
Information Literacy Hallmark
To satisfy the Written Communication requirement, a
course will:
Help students develop information literacy by teaching
search strategies, critical evaluation of information and
sources, and effective selection of information for
specific purposes and audiences; teach appropriate
ways to incorporate such information, acknowledge
sources and provide citations. (2002)
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
5

6. Information Literacy as a
Student Learning Outcome (SLO)
Outcomes measure what a student learned in terms of
skills acquired (e.g., distinctions between Boolean
operators, difference between scholarly and popular
information sources, applying Web source evaluation
criteria, etc.) or in terms of attitudes acquired or reinforced
(preferences and interests, values in information seeking
and use, self-efficacy beliefs, ethical uses of information,
etc.).
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
6

7. The Threefold Self in
Information Skills Assessment
• Attitude (affective skills)
o Values, Perceptions, Preferences, Perseverence
o Coping skills, Self-efficacy, Emotional Intelligence (EI)
• Knowledge (cognitive skills)
o Understanding content and processes
o Critical thinking and information evaluation skills
• Performance (A-C-S skills)
o Applying EI and knowledge in information tasks
o Kinesthetic learning activities, noticing skills
o Embodied learning
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
7

8. Threefold Biological
Mental System
 Sensorimotor System
 Identifying, recognizing, noticing, ignoring, verbalizing,
executing, performing, writing, doing
 Cognitive System
 Interpreting, appraising, planning, comparing, contrasting
differentiating, recalling, defining, critically evaluating, problem-
solving
 Affective System
 Coping, evaluating, value-attaching, prioritizing, persevering,
regulating, intending, engaging
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
18

9. Assessment Phases
• Preliminary: Prior to teaching
o Needs assessment, baseline pre-tests and
questionnaires before instruction
• Formative: Concurrent with teaching
o During a process or session
• Summative: Recollection after teaching
o Immediately after a session, specific later time
interval, or much later (longitudinal)
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
9

10. Formative Assessment
• Concurrent and situated in context
• Process-oriented
• Activity-based
• Performance assessment
• Knowledge assessment
• Attitude and perception assessment
• Authentic assessment
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
10

11. Formative Assessment Types
• Worksheets (or other products) for exercises
• Real-time Surveys and Polls
• Real-time Rating Scales (Evaluation)
• Structured Self-Reports
• Real-time Open-ended Feedback
• Email, Text Chat, Threaded Discussion
• Real-time Rubrics
• Minute writing, Muddiest Point, Free Write, Writing to Learn
• Creative products (posters, stories, personal accounts)
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
11

12. Summative Assessment
• End-point assessment
• Retrospective reflection
• Recall-based encoding
• Performance assessment
• Knowledge assessment
• Attitude and perception assessment (session or
course evaluation)
• Authentic assessment
• Creative products (posters, stories, personal
accounts)
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
12

13. Summative Assessment Types
• Knowledge Tests & Exercises
• Performance Tests & Exercises
• Surveys
• Interviews
• Focus Groups
• Post-Rating Scales (Evaluation)
• Content Analysis of Open-ended Questions
• Rubrics
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
13

14. Knowledge Tests & Exercises
• Matching
• Multiple Choice
• Checklists, identifying appropriate or relevant elements
• Fill-in number, concept words or phrase
• Short answer, a sentence or two
• Concept maps
• Classification, ordering of steps or codes
• Criterion-referenced ranking or prioritizing
• Essay
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
14

15. Performance Tests & Exercises
• Hands-on activities
o Worksheets filled-in during activity
o Screen capture evidence of an activity
o Structured self-report forms during activity
o Research and search logs & journals
• Collaborative exercises
o Peer Learning
o Problem-Based Learning, Inquiry-Based Learning, Jigsaw, etc.
• Portfolios and Presentations
• Reflection Exercises
• Research Assignments
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
15

16. Authentic Assessment
 Authentic Assessment, e.g., concept mapping, minute
writing, portfolios, logs, journals, creative products
(e.g., poster, story, personal accounts), rubrics, etc.
 Use model student responses to build descriptions and
criteria (e.g., LILO rubrics; BID)
 Degree to which a skill has been mastered
 Description of the elements and levels of performance from
low to moderate to high
 Establishes criteria enabling students to understand how to
improve
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
16

17. Composing Assessment Questions
 Avoid bias, leading questions
 Avoid yes/no responses
 Include open-ended responses
 Accounted ratings
 Please explain your number; Please state why you chose that
number
 Accounted knowledge
 Please explain your answer; Please state why you gave that answer
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
17

18. Outcome Measures & Evaluation
Questions
 All ACS Outcomes must be measured to provide evidence of degree of
student learning, and they must be scored (% correct or ideal rating) and
include an Answer Key for the instructor
 Identify the specific Outcome Measures you will use for each PI and set
of ACS Outcomes
 Worksheet(s)
 Pre-Post quizzes
 Poll data
 Observation data (classroom or online meetings)
 Rating scales and open-ended questions
 Rubrics
 Include a Session & Instructor(s) Evaluation Form (like course evaluation)
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
18

19. Mapping Assessment Items
to Learning Outcomes
Cognitive Outcome: Students will accurately estimate the time it takes to
do the library research for an assignment.
Assessment Item:
1. Once you are in the online catalog, how long do you think it will take to find the
call number for the book Tongue Fu?
A. A few seconds.
B. Less than 2 minutes.
C. More than 2 minutes.
• Please state why you think it will take that amount of time:
See examples in LIS 665 student work on Nahl site
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
19

20. Mapping Assessment Items
to Learning Outcomes
Affective Outcome: Students are willing to allocate sufficient time to the
information research activity.
Assessment Item:
1.How willing are you to allot the following amounts of time to library
research for a 10 page paper?
a. 1 hour Gladly Only if I have to Not at all
b. 5 hours Gladly Only if I have to Not at all
c. 10 hours Gladly Only if I have to Not at all
See examples in LIS 665 student work on Nahl site
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
20

21. Exercise: Peer-Review of
Active Learning Exercises
• Teams 1 & 3, Teams 2 & 3; & Teams 4 & 5 meet to
usability test and to audit alignment of active learning
exercises, SPIOs and ACS outcomes.
o First do an exercise created by one Team, after which do an
exercise for the other Team.
o Map the hands-on exercise to the Team’s stated outcomes.
o Identify the form of evidence it will produce to demonstrate
students have learned new knowledge and skills, or completed an
assignment to meet specific ACS Outcomes.
o Determine whether certain ACS Outcomes are not mapped to an
assessment measure (exercise, test, scale, etc.).
o Include criteria for passing each Outcome and the form of
assessment to be used
• Revise exercises based on testing and reviews.
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
21

22. Instructional Sequence
 Linear document showing the detailed order and
schedule for a session
 Each activity is marked with time limits
 Some activities are marked to skip or shorten if needed
(classroom settings)
 Each activity indicates who is teaching, observing,
facilitating, time keeping, troubleshooting, etc.
(classroom settings)
 Lists all equipment, materials and items needed in the
session and who is bringing what (classroom settings)
 Includes procedures for capturing data (paper, online,
observed)
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
22

23. Next Week
• Ch 6; Burkhardt et al.
• Final Draft of Instruction Unit Assessment
documents and forms (Hands-on Exercises, Ratings,
Polls, and Session Evaluation items, etc.)
• Draft of Instructional Sequence with time in
minutes each portion takes to complete
Nahl Spring 2013 LIS 665 Teaching Information Technology Literacy Literacy
23