Acknowledgements
• CSALT – USC Learning and Teaching Grant Scheme
• Reference group
• Student life and learning – Margot Reeh
• Librarian – Roger Carter
• C-SALT – Theresa Ashford
• NAPA – in kind support
2
Who am I & where is USC?
3
Smart Steps Case based simulation Evidence based practice
Why the fuss over IL?
4
Changing student
intake
• the suite of
competencies and
attributes of students
entering university
has changed over the
last 2 decades
Research
• universities in
Australia are not
successfully
equipping their
graduates with
information literacy
skills to enable life-
long learning.
Changing
teaching practice
• How do we teach our
students to move
beyond reproducing
current thinking to
critical thinking?
Ambulance and Paramedic Industry Skills Forecast
Industry Reference Committee; 2018
• Certificate II to diploma
• VET sector
• Bachelor PhD
• Tertiary sector
• National registration for
paramedics in Australia begins
2018
• Degree now baseline qualification
for paramedics
5
The IRC ranked 12 generic skills
1
• Information literacy
2
• Communication
3
• Leadership…
6
The project
Literature review
External Data Collection
Network of Australian
Paramedic Academics
Internal Data Collection
USC Bachelor of
Paramedic Science
7
We asked NAPA participants (n=29) to define
what Information Literacy meant to them...
Theme % of comments Skill level
Find / collect 84.6 Lower
Critique / evaluate 53.8 Higher
Integrate 38.5 Higher
Understand 38.5 Lower
Identify knowledge required 34.6 Higher
Communicate 19.2 Higher
Manage data 3.8 Lower
9
The benefits of developing Information Literacy
skills in Paramedics
Students who develop information literacy skills Generally
agree (%)
can engage in independent learning through constructing new
meaning, understanding and knowledge.
94.7
can search for and use information for decision making and
problem solve.
94.7
can demonstrate social responsibility through a commitment to
lifelong learning and community participation.
73.7
derive satisfaction from using information wisely. 68.4
10
“Standard” . “Element” . “Learning Objective”
Standard 1. Recognises the need for information and
determines the nature and extent of the information needed
Importance Point in
program
1.1.1 Explores general information sources to increase
familiarity with the topic
High Early
1.1.4 Identifies key concepts and terms in order to formulate
the focus questions
High nc
1.2.1 Understands how information is organised and
disseminated
High Early
11
Points of consensus
Points of consensus
1. Recognises the need for information and determines the nature and extent of
the information needed
1.1 Defines and articulates the information needed
Activities/context: Explore information sources, identify key concepts and terms
1.2 Understands the purpose, scope and appropriateness of a variety of information
sources
Activities/context: Topic context is discipline bound; differentiates between a
variety of information sources; identifies primary and secondary information
sources
12
Making sense of student understandings of
Information Literacy:
Sources of information students use for assignments
• Peer Reviewed Journal 97.3
• Textbooks 82.0
• Government reports 64.9
• Online reference 60.4
• Collaborative Encyclopaedia 12.6
• News Media 2.7
• No statistically significant changes across the years
14
• Use of diverse resources to develop assignments increased with year
in program:
Entry year 24.3% final year 65.0%
• HOWEVER - no associations were found for year in program
regarding:
• checking whether a concept is still current while developing an answer for an
assignment.
• use a systematic process to help judge the relevance of the information.
• use of bibliographic software to manage collected information.
15
Making sense of student understandings of
Information Literacy:
• Associations were found for year in program
• Use of multiple sources to reach a conclusion
• Early year 10.8% Final year 60%
• Use the USC style guide to ensure accuracy in citations
• Early year 9.4% Final year 20%
• No associations were found for year in program -
• Working out how to explain a new idea effectively
• Synthesis - setting out the main ideas from each source and then combine
them to generate new knowledge for an assignment
• Use and understanding of statistical/research data
16
Making sense of student understandings of
Information Literacy:
The ‘big easy’
a solution or strategy that can
be implemented using your
current resources and
network.
17
Our Big Easy
Our solution was based on
developing tools to help
recognise and map how our
program would support the
development of a student’s
ability to recognise, find,
critique and synthesise
18
Solution 1
Critical alignment within a program, informed by the data collected externally and internally
Vertical integration decision tool Horizontal integration map
19
IL Standards
Standard 1
Year 1 outcome
Year 1
Course 1
Element
outcomes
Activities
Assessment
Year 1
Course 2
Element
outcomes
Activities
Assessment
IL Standard
USC standard
Program standard
defined by CAUL
defined by university
defined by program
Year 1
Standards Assessment
Target courses
Year 2
Standards Assessment
Target courses
Year 3
Standards Assessment
Target courses
20
Standards and
elements
Year standard Scaffolds
1. Recognises the
need for information
and determines the
nature and extent of
the information
needed
Year 1 - With direction the student interprets the
nature and extent of information the task requires.
Teacher led deconstruction of task requirements
Year 2 - To work with minimal direction to interpret
the nature and extent of information the task
requires.
Student led deconstruction of task requirements
Year 3 - To work independently to interpret the
nature and extent of information the task requires.
Independent deconstruction of task requirements.
2. Finds needed
information
effectively and
efficiently
Year 1 - With direction find information required to
answer a task using an appropriate search strategy
Teacher led session to develop the skills necessary to
use multiple strategies to find information
Year 2 - To work with minimal direction to find the
information the task requires.
Student led session to develop effective strategies to
find information required.
Year 3 Independently finds needed information using
a structured approach
Independently uses appropriate search tools to
efficiently find required information.
3. Critically evaluates
information and the
information seeking
process
Year 1 - With direction evaluates the relevance of
information.
Teacher led session on assessing the usefulness and
relevance of the information.
Year 2 - Identifies themes within the information and
constructs a logical argument.
Student led session on defining and applying criteria for
evaluating the information.
Year 3 - Critically explores themes and critiques the
evidence discovered
Student led session on critical evaluation of information.
Solution 2
A structure informed by the literature review and USC resources
21
Working
Party (me)
Program
reference group
Course
coordinator
Program
coordinator
Academic
developers
Solution 3
Develop a scientific communication rubric for the program that shows
increasing complexity of the pass mark across consecutive years.
Criteria & Weight Year 3 pass (Developing) Year 2 pass (Developing) Year 1 pass (Developing) Year 1 fail (inadequate)
Formation of an argument
Content / Scientific Merit
The assessment piece should
be well-researched, drawing
on a suitable range of quality
sources of information to
answer the question posed
Sound theoretical base with
all material wisely selected
Sound theoretical base with
no irrelevant material
included.
Theoretical base mostly
relevant.
Inadequate theoretical base
with mostly irrelevant or
low-quality material
included.
Analysis
The assessment piece should
show evidence of
exploration of important
themes identified in the
scientific literature
Themes developed in a
balanced way; developing
logical relationships and
internal consistency;
Logical relationships are
indicated; consistent
internally.
Themes are described
without being connected;
internal inconsistencies are
present.
lack of breadth in identified
themes, absence of logical
relationships or many
internal inconsistencies
Synthesis
The assessment piece should
show evidence of critiquing
the scientific evidence and
weaving together of ideas to
support a logical argument.
All arguments are fully and
clearly supported by
evidence based on research
which is presented showing
comparisons and synthesis.
Most arguments are
generally supported by
appropriate research-based
evidence with some
synthesis of the evidence
cited.
Some arguments are
supported by evidence
however there is infrequent
critical analysis of evidence
from sources used.
Little, if any of the evidence
cited supports the
arguments or evidence from
sources is treated
uncritically.
22
Solution 3
Develop a scientific communication rubric for the program that shows
increasing complexity of the pass mark across consecutive years.
Criteria & Weight Year 3 pass (Developing) Year 2 pass (Developing) Year 1 pass (Developing) Year 1 fail (inadequate)
Presentation of the argument
Structure
The assessment should be
structured to meet the
specifications contained in
the task information.
Elegant and imaginative
presentation
Structure appropriate to
assignment task.
Some inconsistencies in
style; adequate statement of
intention appropriate layout.
Appropriate introduction
with clear statement of
intention; well-structured
overall with appropriate
conclusion;
Structure and style
Badly designed;
Grammatical Conventions
Your work should be well-
presented in a style that
meets the prescribed criteria
or contemporary
conventions
All presentation criteria met
with no spelling, syntax or
grammatical errors.
All presentation criteria met
but with a more than five
spelling, syntax or
grammatical errors.
Appropriate style that was
consistent with the task
Inappropriate style that was
inconsistent with the task.
Referencing and citations
Your work is original and the
sources of ideas are
acknowledged where
appropriate and citations
reflect the prescribed style
guide
Consistent and complete
Vancouver style referencing
techniques with a few errors
Consistent and complete
Vancouver style referencing
techniques with moderate
number of citation errors
There are inconsistencies in
the Vancouver style
referencing technique.
In adequate referencing style
and technique.
23
I have a nursing and paramedicine background
I am a lecturer in the Bachelor of Paramedic Science program at USC, Qld
The program I work in is a science based paramedicine program that teaches evidence based practice mainly through case based learning
My interest in ILE stems is because IL is the backbone of evidence based paramedicine and clinical practice.
Changing student intake
the demographic attributes of university students in Australia has shifted significantly over the last two decades changing suite of competencies of students who enter the classroom.
Research
universities in Australia and North America are not successfully equipping their graduates with the requisite skills in information literacy to enable life-long learning.
Our research showed a lack of consensus among educators on critique and synthesis
Teaching & Learning
How do we teach our students to move beyond reproducing current thinking to critical thinking?
Education of paramedics -
In 1996 the first university course for entry to practice paramedicine began.
Now, in 2018 a national registration scheme is being introduced and the base line qualification for a paramedic is a bachelor degree.
How does the industry rate ILE?
SkillsIQ. 2018 Industry Skills Forecast for Public Consultation. Ambulance and Paramedic Industry Skills Forecast Industry Reference Committee; 2018
In this changing environment the Industry Reference Committee met to discuss amongst other things 12 generic skills taught in courses.
Information literacy was voted as the top skills
IL is defined by the skills council as the ability to:
identify a need for information, and to locate, evaluate, and effectively use and cite the information.
develop a working knowledge of new systems.
work without direct leadership and independently.
2 – Communication
3 – Leadership
4 – Technology and application
5 – Design mindset/thinking critically/ Problem solving
6 – Language, literacy and numeracy
7 – Data analysis
8 – Customer service
9 – STEM
10 – Entrepreneurial
11 – Financial
12 – Environmental / sustainability
Our project consisted of three stages
Literature review- A systematic review of the literature was performed to identify information literary skills are described as critical for the health disciplines, and this search also sought to identify validated survey tools in the area of testing IL skills.
Delphi study – (I think this may be too much for your time and audience)
A modified Delphi process13 with three iterative rounds were conducted online
We used the ANZIL model as the frame to collect data
The ANZIL model has 6 standards and each has a subset of elements. These are on the notes on the tables
We provided learning outcomes from major universities to seek consensus on important attributes for a paramedic program to focus on and where in the program that the attribute should be achieved by.
We surveyed Paramedic academics to achieve consensus.
Participants were all members of the Paramedics Australasia Network of Australian Paramedic Academics (NAPA).
This group was chosen as the reliability of data from Delphi studies are improved when subjects are drawn from the specialized area of knowledge related to the target issue.14
USC student survey
Custom designed online survey as the two validated surveys developed by CAUL were deemed by the reference group to not be useful in the context of this project
Some of the questions seemed ambiguous, open to social identity issues, or irrelevant due to changes in technology.
The survey was piloted with a small convenience sample of four academic staff and fourteen students from Bachelor of Nursing and Bachelor of Sport and Exercise Science programs.
We asked our external partners – paramedic academics from around the country
NAPA – national association of paramedic academics
We asked
How they defined IL
How they taught IL
To rate 72 learning outcomes based on the ANZIL standards
The Australian and New Zealand Institute for Information Literacy definition
a set of abilities enabling individuals to; recognise when information is needed and have the capacity to locate, evaluate and use effectively the needed information
We asked NAPA participants in round one of the Delphi to define IL as a skill.
This question elicited responses that emphasised lower order skills of finding or collecting information rather than critique and synthesis.
This aligned with the ILOs that reached consensus as learning objectives pertaining to Standard Three: critically evaluates information and the information seeking process.
The ILOs for critical thought were the predominant learning outcomes that did not achieve consensus.
The findings show that the lower order IL skills appear to be more valued than higher order skills.
This is unusual because IL is considered the basis for evidence-based-practice2 and is a necessary skill for paramedic-led healthcare and continuous professional development.
The lack of consensus in this area is consistent with previously cited studies showing that IL is often underdeveloped in university graduates.
Benefits of attaining information literacy skills
CAUL make 4 statements about the benefits of attaining IL skills. All the NAPA participants indicated that students undertaking an entry-to-practice paramedic program need to develop information literacy skills.
NAPA participants generally agree with the benefits of ILE as described in the ANZIL framework. (shown here) and perceived it as important in paramedic undergraduate entry-to-practice programs and of value to students.
However, the comments describing how ILE is taught indicated that ILE is not systematically embedded in many paramedic programs.
Four overarching principles:
1Engage in independent learning through constructing new meaning, understanding and knowledge;
2Derive satisfaction and personal fulfilment from using information wisely;
3Individually and collectively search for and use information for decision making and problem solving; and
4Demonstrate social responsibility through a commitment to lifelong learning and community participation.
There are 6 ANZIL IL standards.
Each has a number of elements.
In the NAPA Delphi we presented learning outcomes for each element to the participants.
We asked the participants the level of importance of each learning outcome where in the program these should be achieved.
We showed participants how others responded and gave an opportunity to change their answer.
Consensus was if 80% of participants agreed.
This slide shows an example of ILOs that reached consensus on importance – high, moderate or low. Note that not all learning outcomes that reached consensus on level of importance reached on consensus for place in program.
Participants indicated that many outcomes should be taught at the commencement of a program of study.
Front loading the foundations of IL skills makes sense as skills of finding, describing, storing and managing information, support the higher level skills such as information analysis and evaluation.
The other surprise was that critical thinking outcomes of did not reach consensus. It is my position after this project that this may tie into why universities are poor at developing IL skills in graduates and why this attribute was only mentioned by about half of the NAPA participants in their definitions.
The first two numbers in the numbering of the ILO shows which standard and element it is assigned to. The third number is added to provide the ILO with a unique identifier.
From these findings we developed the important themes or activities and context for teaching in an entry to practice paramedicine program.
What is shown here is an example of two elements for standard 1.
This has allowed us to make better choice around curriculum development.
Peer Reviewed Journal 97.3
Textbooks 82.0
Government reports 64.9
Online reference 60.4
Collaborative Encyclopaedia 12.6
News Media 2.7
Statistical values for association Fishers exact test=9.455, P=0.027
Q1: I use diverse sources of information to develop my assignment. Choose one response only.
Q2: I check whether a concept is still current while developing an answer for an assignment.
Q3: I use a systematic process to help me judge the relevance of the information
Q11: Do you use bibliographic software such as Endnote to manage collected information?
Regarding the use of multiple sources to reach a conclusion
Early year 10.8% C/- final year 60%; FET=23.467, P=0.002
Use the USC style guide to ensure accuracy in citations
Early year 9.4% C/- final year 20%; FET=13.330, P=0.022
We also found in the data that
Students perceive that basic statistical information such as P value, Confidence Intervals, Odds Ratio, Limitations of study design, Level of evidence, Number needed to treat, is important to consider when choosing an article to use for an assignment
But only about a quarter of students reported using this data often or always when it was provided,
Even less were able to correctly identify the correct definition for these concepts from a range of options. .
Concept
Important informationΩ (%)
n=74
Actively
considerΨ (%)
n=101
Able to
define concept (%)
N=76
P value
66.7
28.7
19.5
Confidence Intervals
41.4
23.8
17.1
Odds Ratio
62.7*
18.6*
14.3
Limitations of study design
84.1*
50.6*
Level of evidence
84.5
72.2
Mean
81.6
Median
76.3
NNT
14.5
The big easy – is a strategy that can be implemented using your current resources and network.
These are the start of change
These actions start momentum for change.
We shouldn’t assume that understanding scientific communication and information literacy occurs automatically with a program of study
A lot of service teaching occurs for our program, so it is not just individual programs with responsibility to develop IL skills of students
So the question became how wouldour program scaffold the teaching of students to move beyond reproducing current thinking to critical thinking.
We are moving away from course-centric to a program centric view.
We are looking at the CAUL standards and defining within our program what students should achieve at each year level. We are then mapping this to courses.
I will work through this slide to demonstrate what I mean.
Vertical integration
For each CAUL IL standard we must define it at program level and then year level within the program and decide what assessment tasks would best detect evidence of achieving that standard. The next decision is to decide which courses are best suited to target the teaching.
Horizontal
We take the information from the decision tool and apply it to the selected courses.
The student data has shown us where the problem areas are
The Delphi data provided information on what Paramedic Educators think are the important aspects to be concentrating on.
We will also be concentrating on critical thinking, although no consensus was reached.
We are used this template to map ILE across the program and courses.
Wang L. Developing peoples information capabilities: Fostering Information Literacy in educational workplace and community contexts. Library and Information Science. 2013;8:31-49.
As a team we produced a concensus statement on the ANZIL standards for each year level.
PAR101, PAR102, PAR103
PAR211, PAR202, PAR203, PAR222, PAR212
PAR301, PAR311, PAR302
To guide our staff the following flow of information and assistance will be implemented.
Roles within the framework
Program reference group: expertise external to the program that can advise on guidelines and pedagogy to enable information literacy education with a program. external
Program coordinator: the program lead for the design, alignment and quality assurance of curricula activities within a program of study
Course coordinators: responsible for the design and delivery of individual courses within a program of study
C-SALT: USC body responsible for assisting program and course coordinators to deliver quality programs of study.