Keynote at MOBIUS 2015: Articulating Our Impact: Strategies for Holistic Library Assessment

1. A R T I C U L A T I N G
O U R I M P A C T

2. A R T I C U L A T I N G
O U R I M P A C T

3. H O L I S T I C
library assessment
s t r a t e g i e s f o r

4. s t r a t e g i e s f o r
@ c h a r b o o t h
s l i d e s h a r e . n e t / c h a r b o o t h
H O L I S T I C
library assessment

5. s t r a t e g i e s f o r
@ c h a r b o o t h
s l i d e s h a r e . n e t / c h a r b o o t h
S A R A L O W E S E A N S T O N E
N A T A L I E T A G G E D A N I B R E C H E R
H O L I S T I C
library assessment

6. holistic assessment
of student IL learning
*

7. holistic assessment
of student IL learning
*

8. holistic assessment
of student IL learning
*
a u t h e n t i c

9. holistic assessment
of student IL learning
*
a u t h e n t i c
differentiated

10. holistic assessment
of student IL learning
*
a u t h e n t i c
differentiated
i n t e g r a t e d

14. a c c e s s
f r e e d o m
a d v o c a c y
i n q u i r y
o p e n n e s s

15. re e d c o l l e g e

20. ✓ PROCRASTINATION

21. ✓ PROCRASTINATION
✓ COMISSERATION

22. ✓ PROCRASTINATION
✓ COMISSERATION
✓ GRAFFITING

23. ✓ PROCRASTINATION
✓ COMISSERATION
✓ SLEEPING
✓ GRAFFITING

24. O W N E R S H I P

25. R I T U A L

26. C O M M U N I T Y

27. V A L U E

29. C O N T E N T

30. C O N T A I N E R
C O N T E N T

31. C O N T A I N E R
C O N T E N T
C O N C E P T

32. V A L U E

33. { }

34. Academic libraries have long enjoyed
their status as the “heart of the
university.” However, in recent decades,
higher education environments have
changed… Community college, college,
and university librarians no longer can
rely on their stakeholders’ belief in their
importance. Rather, they must
demonstrate their value.
-Megan Oakleaf, 2010

35. { }D A T A

36. D A T A

37. D A T A

38. D A T A

39. D A T A

40. ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_summary.pdf - p12
D A T A

41. D A T A
ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_summary.pdf - p12

42. { }V A L U E

43. { }I M P A C T

44. { }Wh a t ’s In It Fo r Th e m ?

45. I N S T I T U T I O N A L / E X T E R N A L
A D M I N S T R A T I V E
D E P A R T M E N T A L
I N D I V I D U A L
A U D I E N C E

46. R E S O U R C E S & S T A F F
R E C O G N I T I O N
C O L L A B O R A T I O N
G O O D W I L L
O B J E C T I V E

47. R E P U T A T I O N
E F F E C T I V E N E S S
S T U D E N T S U C C E S S
S A T I S F A C T I O N
O U T C O M E

48. holistic assessment
of student IL learning
* assessment

49. holistic assessment
of student IL learning
*

50. holistic assessment
of student IL learning
*

51. holistic assessment
of student IL learningR U B R I C S
*

52. holistic assessment
of student IL learningR U B R I C S
*

53. holistic assessment
of student IL learningR U B R I C S
*
M A P P I N G

54. { }R U B R I C S
M A P P I N G

55. { }c u s t o m i z e d
R O A D M A P S
o f l e a r n e r
C O M P E T E N C I E S
R U B R I C S

56. { }
T H E C L A R E M O N T C O L L E G E S

57. { }
T H E C L A R E M O N T C O L L E G E S

58. L I B R A R Y I L D E F I N I T I O N

59. S T U D E N T & F A C U L T Y S U R V E Y S

60. I L P E R F O R M A N C E P E R C E P T I O N S G A P
0%
20%
40%
60%
80%
100% Faculty
Student
Faculty
Student
Faculty
Student
Faculty
Student
Faculty
Student
Faculty
Student
Write
annotated
bibliographies
Provide
proper
attribution to
source
materials in
their academic
work
Use sources
to further an
argument/
thesis
Evaluate
sources to
determine if
they are
authoritative
Differentiate
between types
of information
sources (e.g.,
scholarly v.
popular
literature, fact
v. opinion)
Effectively use
Library
databases,
catalog(s),
and other
information
resources to
find relevant
source
material
Excellent/Very High
Above Average/High
Average/Moderate
Below Average/Low
Poor/None

61. 0%
20%
40%
60%
80%
100% Faculty
Student
Faculty
Student
Faculty
Student
Faculty
Student
Faculty
Student
Faculty
Student
Write
annotated
bibliographies
Provide
proper
attribution to
source
materials in
their academic
work
Use sources
to further an
argument/
thesis
Evaluate
sources to
determine if
they are
authoritative
Differentiate
between types
of information
sources (e.g.,
scholarly v.
popular
literature, fact
v. opinion)
Effectively use
Library
databases,
catalog(s),
and other
information
resources to
find relevant
source
material
Excellent/Very High
Above Average/High
Average/Moderate
Below Average/Low
Poor/None
* * * * * *
* * * * *
I L P E R F O R M A N C E P E R C E P T I O N S G A P

62. How much have your instructors (students)/you (faculty)
emphasized the following in the courses you teach?
Student v. Faculty Mean Percentage Difference
-10.38%
-9.97%
-9.12%
-2.86%
-0.97%
-12% -10% -8% -6% -4% -2% 0%
Using peer-reviewed or scholarly
sources in assignments
Using practices (terminology,
procedures, writing style, etc.) of a
specific major or field of study
Appropriately citing the sources used in
a paper or project
Questioning the quality of information
sources
Not plagiarizing another author's work

68. C A M P U S - L E V E L I M P L E M E N T A T I O N
S C R I P P S
P I T Z E R
P O M O N A
C G U
H M C
K G I
C M C
Grassroots/
consensus
governance
with robust
committee
structure
Pitzer College
Consensus-
based
governance
without much
committee
structure
Scripps College
Faculty
committees
with strong
departmental
autonomy
Pomona
College
Top-down/
federated
states
governance
Claremont
Graduate
University
Faculty-
committee-
based
governance
Harvey Mudd
College
Small/agile/
progressive
committee of
core faculty
Keck Graduate
Institute
Top-down
governance/
assessment
driven
Claremont
McKenna
College

69. Grassroots/
consensus
governance with
robust
committee
structure
Pitzer College
Consensus-
based
governance
without much
committee
structure
Scripps College
Faculty-
committee-
based with
strong
department
decision making
Pomona College
Top-down/
federated states
governance
Claremont
Graduate
University
Faculty-
committee-
based
governance
Harvey Mudd
College
Small/agile/
progressive
committee of
core faculty
Keck Graduate
Institute
Top-down
governance/
assessment
driven
Claremont
McKenna
College
SU 2012: Director of
Assessment meets
with librarians to
discuss IL
assessment options
SU 2012: Librarians
edit Carleton College
Rubric to create
Rubric v1.1
SU 2012: Librarians
score First Year
Seminar papers (FYS)
using Rubric v1.2
FA 2012: Faculty
incorporate Rubric
v1.4 into FYS
FA 2012: Librarians
score capstone
papers with Rubric
v1.4
SU 2014: Library
uses AiA Rubric to
evaluate student IL
skills
FA 2012: Writing 50
(first-year seminar)
faculty score papers
using Rubric
2012/2013
SU 2014: Library
uses 5-part Rubric to
evaluate student IL
skills
Future: Hopeful
integration of
2013-14 IL Rubric
into capstone
evaluation
SU 2011:
Environmental
Analysis (EA) Rubric
created based on
Drake Univ. Rubric
SU 2011: Discussions
with faculty improve
rubric & EA program
FA 2011/FA 2012:
EA Rubric used to
develop IL instruction
for EA senior thesis
students
SP 2013: Librarians/
faculty use Rubric
2012/2013 to
evaluate sample EA
theses from three
prior years, submits
to WASC
SU 2013: Library
uses AiA Rubric to
evaluate student IL
skills
SU/FA 2012: Rubric
v1.4 discussed at all
departmental
meetings
SP 2013: Dean
requests Rubric
2012/2013 as
foundation for an IL
instruction session in
methods course /
leads to development
of IL Labs pilot
SP 2013: 5-Part
Rubric provides
foundation for
developing five core
learning outcomes for
IL Labs pilot
Future: Planning for
potential integration
of Rubric into a
Canvas course pilot
as a sample
evaluation object, and
a Thesis/Dissertation
evaluation project
FA 2012: Librarians
and ALO attend
WASC Accreditation
Retreat on Core
Competencies:
Critical Thinking and
Information Literacy
FA 2012: Librarians
present Rubric
2012/13 to
Assessment
Committee
FA 2012: Committee
interested in training
upper class “peers” in
using rubric to
evaluate clinic work
SU 2014: Library
uses AiA Rubric to
evaluate student IL
skills
Future: Hopeful
integration of
2013-14 IL Rubric
into clinic evaluation
FA 2012: Librarians
and ALO attend
WASC Accreditation
Retreat on Core
Competencies:
Critical Thinking and
Information Literacy
FA 2012: Map and
integrate Rubric v1.?
into existing rubrics
for some programs
SP 2013: Begin using
updated rubrics as
part of an overall
program to increase
rubric use in more
classes
Future: IL Rubric data
needs to be gathered
and assessed.
Rubrics are being
adopted/developed
for the new
BioPharmacy
program
FA 2012:
Assessment
committee adopts
Rubric 2012/2013
FA 2012: Keck
Science Dept. adopts
Rubric 2012/2013
SP 2013: Library
scores senior theses
using Rubric
2012/2013
SP 2013: Keck
Science Dept. scores
senior theses using
Rubric 2012/2013
SU 2014: Library
uses AiA Rubric to
evaluate student IL
skills
Future: Hopeful
integration of
2013-14 IL Rubric
into capstone
evaluation

70. S P E C I F I C R U B R I C A P P L I C A T I O N S
• F i r s t - y e a r s e m i n a r p a p e r e v a l u a t i o n
• S e n i o r c a p s t o n e / t h e s i s e v a l u a t i o n
• C a m p u s e s a c c r e d i t a t i o n a s s e s s m e n t
• A s s e s s m e n t i n A c t i o n ( 2 0 1 3 - 1 4 )

71. S P E C I F I C R U B R I C A P P L I C A T I O N S
• F i r s t - y e a r s e m i n a r p a p e r e v a l u a t i o n
• S e n i o r c a p s t o n e / t h e s i s e v a l u a t i o n
• C a m p u s e s a c c r e d i t a t i o n a s s e s s m e n t
• A s s e s s m e n t i n A c t i o n ( 2 0 1 3 - 1 4 )

72. 5 C O L L E G E S = 5 F I R S T Y E A R
S E M I N A R P R O G R A M S

73. Research Question
“What impact (if any) does librarian intervention in first-year courses have on IL
performance in student work?”
Methodology
1. None = no faculty collaboration with librarian
2. Low = Traditional One-Shot
3. Moderate = multiple sessions, moderate syllabus/assignment collaboration
4. High = multiple sessions, online tutorial & quiz, significant syllabus/
assignment collaboration
Collaboration Levels
• First-Year Student Papers coded by Level of Librarian Collaboration in Course
• Rubric Evaluation (http://bit.ly/ccl-ilrubric) of papers
• 4 Liberal Arts College First-Year Seminar/ Experience programs > Claremont
McKenna College, Pitzer College, Pomona College, Scripps College (coded in
results for anonymity) ; Results from fifth college, Harvey Mudd, still being
coded
• 416 papers; 14 interrater pairs (Jan-May, 2014)

75. report and data - libguides.libraries.claremont.edu/AiA

76. { }R U B R I C S
M A P P I N G

77. { }
C O N T E X T U A L / C U L T U R A L I N V E S T I G A T I O N

78. { }
C O N C E P T / M I N D M A P P I N G
m i n d o m o . c o m

79. { }
C O N C E P T / M I N D M A P P I N G
K N O W L E D G E M A N A G E M E N T

80. { }
C O N C E P T / M I N D M A P P I N G
K N O W L E D G E M A N A G E M E N T

81. C U R R I C U L U M V I S U A L I Z A T I O N

82. c a t a l o g s s c h e d u l e s w e b s i t e s
C U R R I C U L U M V I S U A L I Z A T I O N

83. c a t a l o g s s c h e d u l e s w e b s i t e s
C U R R I C U L U M V I S U A L I Z A T I O N

84. C U R R I C U L U M V I S U A L I Z A T I O N

85. C A M P U S

86. P R O G R A M

87. R E Q U I R E M E N T S

88. O B J E C T I V E S / O U T C O M E S

89. A N N O T A T I O N S

90. I N S I G H T
+
S T R A T E G Y

91. S C A L I N G U P : I M L S S P A R K S ! G R A N T

92. T E M P L A T E S , S U P P O R T , & T R A I N I N G

93. I D E N T I F Y I N G S C A F F O L D E D I L S E Q U E N C E

94. D I S C I P L I N A R Y F A C U L T Y I L F O R U M S

95. { }R U B R I C S
M A P P I N G

96. h o l i s t i c
a s s e s s m e n t
D E M O N S T R A T I N G I M P A C T
I T E R A T I V E I M P R O V E M E N T
I N V E S T I G A T I N G V A L U E

97. { }V A L U E

98. { }I M P A C T

99. R E S O U R C E S
ea curriculum map: ea.pomona.edu/ea-5c-program-map/
ccl il program documents: libraries.claremont.edu/informationliteracy/
A i A p r o j e c t a n d d a t a : l i b g u i d e s . l i b r a r i e s . c l a r e m o n t . e d u / A i A
Degrees of Impact: Analyzing the Effects of Progressive Librarian Course
C o l l a b o r a t i o n s o n S t u d e n t P e r f o r m a n c e . W i t h M . S a r a L o w e , N a t a l i e
T a g g e , a n d S e a n M . S t o n e . C o l l e g e & R e s e a r c h L i b r a r i e s .
Accepted: June 26, 2014; Anticipated Publication Date: July 1, 2015.
Slides from Tagge, Natalie; Booth, Char; Chappell, Alexandra; Lowe, M. Sara; and
Stone, Sean M., "Choose Your Own Adventure: Integrating an Information
L i t e r a c y R u b r i c i n t o S e v e n ( V e r y ) D i f f e r e n t
C o l l e g e s " ( 2 0 1 3 ) . L i b r a r y S t a f f P u b l i c a t i o n s a n d R e s e a r c h . P a p e r 1 7 .
h t t p : / / s c h o l a r s h i p . c l a r e m o n t . e d u / l i b r a r y _ s t a f f / 1 7
S l i d e s a d a p t e d f r o m L o w e , M . S a r a ; B o o t h , C h a r ; S t o n e , S e a n M . ; T a g g e ,
N a t a l i e ; C h a p p e l l , A l e x a n d r a ; a n d B u r r o w , G a l e , " L i b r a r i a n s M a t t e r !
I m p a c t o n F i r s t - Y e a r I n f o r m a t i o n L i t e r a c y S k i l l s a t 5 C o l l e g e s " ( 2 0 1 4 ) .
L i b r a r y S t a f f P u b l i c a t i o n s a n d R e s e a r c h . P a p e r 2 2 .
h t t p : / / s c h o l a r s h i p . c l a r e m o n t . e d u / l i b r a r y _ s t a f f / 2 2

100. c h a r b o o t h @ g m a i l
i n f o m a t i o n a l . c o m
@ c h a r b o o t h
C H A R B O O T H
D I R E C T O R
r e s e a r c h , t e a c h i n g
&
l e a r n i n g
S E R V I C E S
c l a r e m o n t c o l l e g e s
L I B R A R Y
s l i d e s h a r e . n e t / c h a r b o o t h