Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build Teamwork Skills and Reflective Practice for Medical Science Students: Stories and Strategies from UNSW Australia - Patsie Polly & Jia-Lin Yang, University of NSW
During their course of study, medical science students are generally unaware that they are developing professional skills related to graduate capabilities. Interestingly, at a program level the institution finds it difficult to view the development of these capabilities. In this session we will discuss our own learning journey as discipline specific teachers who have worked collaboratively to implement ePortfolios and rubrics across courses and within the medical science degree program at UNSW Australia. Our approach to supporting student learning and development of reflective practice and professional skills in teamwork by cross-discipline alignment of assessment coupled with ePortfolio thinking and doing will be presented.
Similar a Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build Teamwork Skills and Reflective Practice for Medical Science Students: Stories and Strategies from UNSW Australia - Patsie Polly & Jia-Lin Yang, University of NSW
A Workshop: Promoting Student Access and Success Through ResearchTanya Joosten
Similar a Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build Teamwork Skills and Reflective Practice for Medical Science Students: Stories and Strategies from UNSW Australia - Patsie Polly & Jia-Lin Yang, University of NSW (20)
Python Notes for mca i year students osmania university.docx
Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build Teamwork Skills and Reflective Practice for Medical Science Students: Stories and Strategies from UNSW Australia - Patsie Polly & Jia-Lin Yang, University of NSW
1. School of Medical Sciences
Patsie: Lead facilitator: ePortfolios pedagogy/rationale/skills in the Medical Sciences and beyond
Jia-Lin: Lead Career learning and ePortfolios, Data/statistics / spokesperson
Thuan: Project manager/ spokesperson - Aligning Assessment Project implementation across
disciplines, 2015
Annie: Project manager/ spokesperson - Fellowship implementation across programs, 2016
Richard, Trevor, Nicole, Cristan, Thomas, Julian, Sue:
Discipline Specific Experts: will lead table discussions and discuss their course and task specific
approaches
Stories and Strategies from UNSW Australia
Collaborative, Program-wide Alignment of Assessments and
ePortfolios to Build Teamwork Skills and Reflective Practice for
Medical Sciences Students:
2. ePortfolios Australia: 29/09/16 Workshop 2 Room 2174, 2-4:30pm
Title: Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build
Teamwork Skills and Reflective Practice for Medical Science Students: Stories and
Strategies from UNSW Australia
Themes: Working across disciplines, building teamwork skills, scaffolding reflective
practice, ePortfolios for future use beyond university , extracting data from ePortfolios
Schedule
• Workshop overview: 15 min
• Group discussion: 30 min
• Group presentation: 30 min (5 min per group/table)
• Discussion: Design, Development, Implementation (30 mins)
• Q & A (30 min)
• Summary (15 min)
3. AIMS: Workshop
• To engage colleagues in thinking about how to align
assessment tasks across programs and disciplines
• Implementation of ePortfolio use linked to assessment tasks
• Discuss the idea of ePortfolios, rubrics and data to recognise
attainment of skills
4. ePORTFOLIO / Reflective Blogging
Select
Collect
ReflectCurate
Present
Self appraisal
Gathering
occupational
information
Goal setting and
selection
Planning and
problem solving
Self
awareness
Decision
making
Opportunity
awareness
Transition
learning
Yang et al, 2014
Polly et al., 2015
5. UNSW – Institutional Priorities for Graduates
Global Citizens
Scholars
Leaders Professionals
UNSW, Strategic Intent, 2013
UNSW, Key Priorities, 2012-2016
UNSW Strategy, 2025
The ‘Why’
P. Polly et al, 2013
Understanding of
their discipline in its
Interdisciplinary context
Capable of
initiating and
embracing
innovation and
change
Capable of ethical, self-
directed practice and
independent lifelong learning
Graduates who are
culturally adept and
capable of respecting
diversity
7. 2:10-2:40pm
Group discussion: 30 min
• What is your understanding of a student ePortfolio?
• What is your experience in ePortfolio study and/or practice?
• What are 3 important issues in current ePortfolio study and
practice from your points of view?
• What are your 3 suggestions improving ePortfolio study and
practice in educational institutes in Australia?
8. 2:40-3:15pm (35 min, 5 min/table)
Group feedback
ePortfolio definition
Three important issues
Resolving issues
11. 3:30-4:00pm (30 min) What have we done in resolving
the issues? UNSW Stories and strategies
PATSIE: Overview (3-5mins)
PATSIE: Design:
Top down from school incorporating teamwork skill development and ePortfolio in curriculum of science program
PATSIE: Development
• Teamwork, Modified AAC&U Rubric
• ePorfolio, template questions, reflection rubric
JIA-LIN: Career development learning, CDMSE survey
Implementation:
THUAN (APP; 3mins): Moodle workshop, self-/peer assessment, WordPress
ANNIE (TF; 3mins): Moodle as an LMS, central data collection
JIA-LIN (3mins): Statistical analysis, ePortfolio reflection analysis (N Vivo)
Discipline Expert Stories: 3 mins each – suggestion, assessment strategy linked to ePortfolio slide, data slide
TREVOR: Neuro Physiology - 3rd Year
RICHARD: Neuro Physiology – 2nd Year
NICOLE: Pharmacology – 3rd Year
THOMAS: Anatomy - 3rd Year
CRISTAN: Pathology – 2nd Year
SUE: Medical Science – 1st Year
12. ISSUE and WHY we use ePortfolio
• Students find teamwork activities challenging
• Assessment tasks coupled with reflective practice are critical
for building professional skills such as teamwork
• Address graduate attributes to launch into professional skills
and career learning
• Capturing, tracking and extracting data from ePortfolios is a
challenge
13. Longitudinal,Program-wideuseofePortfolio
Professional
Readiness
Pathways
Further studies
Research
Industry
Year 1
Year 2
Year 3
Year 3
Year 4
Career
Awareness
ePortfolio Use in the Medical Sciences
Cross-Discipline Approaches – WordPress 2015, Moodle 2016
Skills Building
Anatomy Pathology Pharmacology Physiology
Transverse, Cross-context and cross-discipline
UNSW L&T Innovation and Development Grant, Round 2 2014
Aligning assessment to track development of graduate capabilities, professional skills and life-long
learning in the biomedical sciences
UNSW Ethics Approval
HC15134
P. Polly et al, 2014
Identity
Narrative
Employability
14. Teacher Involvement – Medical Science Program
Aligning Assessments
P. Polly et al, 2014
15. ePortfolio: a high impact practice in higher education:
Statistical analysis and ePortfolio assessment
Jia Lin Yang
ePortfolio Forum Workshop
28 September 2016
16. ePortfolios impact on learning: Statistical analysis
• Scholarship practice (educational publications)
• Hypothesis testing and errors
• Descriptive and inferential statistics
• Selection of correct statistical analysis methods (Yang et al.
2010 eSTATS-CHOICE:
http://dev.adaptiveelearning.com/demos/DTv3/)
19. NVivo - http://www.qsrinternational.com/nvivo-product
NVivo is software that supports qualitative and mixed methods
research. It’s designed to help you organize, analyze and find
insights in unstructured, or qualitative data like: interviews,
open-ended survey responses, articles, social media and web
content.
A systemic approach:
• Converting data into digital transcripts
• Coding/recoding
• Constant comparing
• Clustering
• Storing
• Facilitating the connection
• Pilot inductive and interpretive analysis
• Within/across cases
• Commonalities and exceptionalities
20. Badging
A badge is a visual representation of an accomplishment,
achievement or skill acquisition—more granular than a formal
degree, but helps to make incremental learning more visible.
(http://www.educause.edu/badging)
A digital badge is a validated indicator of accomplishment, skill,
quality, or interest that can be earned in many learning
environments. Open digital badging makes it easy for anyone
to issue, earn, and display badges across the web—through an
infrastructure that uses shared and open technical technical
standards.
(https://www.hastac.org/initiatives/digital-badges)
21. Career Development Learning
across Year 3 Science Courses
Jia Lin Yang
Research article: Jia-Lin Yang, Patsie Polly, Thomas Fath, Nicole Jones, John
Power.2016. Int J Science Math Tech Learning. 23(3), 33-46.
http://ijlsmtl.cgpublisher.com/product/pub.266/prod.111
22. Pilot study: Educational interventions on career
development learning (2012, PATH3208)
Yang et al. Int J Adult, Commun Prof Learning Vol 22 (1), 2015, pp 1-17.
Table 4: Results of CDSME Survey Pre- and Post the Study Intervention
Total (n=32) Female (n=21) Male (n=11)
mean SD p mean SD p mean SD p
Self-appraisal pre 18.1 3.1 0.014 18.3 2.6 0.019 17.8 3.9 0.240
post 19.8 2.6 19.8 2.0 19.7 3.6
Occupational
information
pre 15.9 2.8 0.001 16.2 2.4 0.002 15.4 3.5 0.122
post 18.2 3.1 18.7 3.0 17.3 3.1
Goal selection pre 18.4 3.1 0.507 18.9 3.4 0.794 17.6 2.5 0.438
post 18.8 2.6 19.1 2.4 18.3 3.0
Planning pre 17.0 2.9 0.002 17.8 2.7 0.048 15.4 2.6 0.009
post 19.2 3.1 19.3 3.0 18.9 3.4
Problem
solving
pre 17.1 3.0 0.016 17.6 3.4 0.162 16.1 1.9 0.024
post 18.3 2.5 18.5 2.8 18.1 1.9
Source(s): Yang, Coleman, Das and Hawkins, 2013
23. Aim
1. To enable more students in Stage 3 courses at the School of
Medical Sciences to be engaged in a career development
learning process in order to develop career awareness,
employability and professional skills.
2. To perform a controlled study in large sample populations.
1. There would be no improvement in CDMSE scales within
individual courses, pre- and post-course.
2. There would be no difference in variation of CDMSE
scales in CDL intervention or control group.
Null Hypotheses
27. Results 1: CDMSE scales within individual courses
0 5 10 15 20 25
occupational info
goal selection
planning
self appraisal
problem solving
ANAT3212 (n=11)
post-course pre-course
0 5 10 15 20 25 30
occupational info
goal selection
planning
self appraisal
problem solving
PATH3208 (n=37)
post-course pre-course
0 5 10 15 20 25
occupational info
goal selection
planning
self appraisal
problem solving
PHAR3101 (n=52)
post-course pre-course
0 5 10 15 20 25 30
occupational info
goal selection
planning
self appraisal
problem solving
PHAR3202 (n=35)
post-course pre-course
0 5 10 15 20 25
occupational info
goal selection
planning
self appraisal
problem solving
NUER3221 (n=30)
post-course pre-course
Control group
* p<0.05, ** p<0.01
**
28. Result 2: CDMSE scales obtained pre- and post-course
for CDL intervention (n=135) and control (n=30) groups
**
* p<0.05, ** p<0.01
29. Conclusion
• The introduction of this pedagogy and model has
proved successful in evidencing graduate learning
outcomes for improving student confidence in
CDMSE.
• This study supports further extension of this approach
to senior undergraduate courses at UNSW and other
universities.
• Using ePortfolio to deliver CDL will result in directive
and structural reflection by learners.
32. Acknowledgements and Outcomes
UNSW Australia funding
• 2015 : SEF#4 1x LNT Innovation and Development
• 2016 : SEF#3 UNSW Teaching Fellowship
Presentations
• 2015 Yang et al, MedEd Forum, Faculty of Medicine, Dec
• 2015 Yang, Jia-Lin, and Patsie Polly. ‘Career development learning across year 3 science courses – Pedagogy and
curriculum: Improving student self-efficacy in employability skills’ 22nd Learners Conference, University of St Paulo,
Madrid.
• 2016 Polly et al, AAEEBL International ePortfolio Conference, Boston, US, Jul
• 2016 Polly et al, ePIC, International ePortfolio and Identity Conference, Invited Panelist, Invited Speaker, Bologna, Oct
Published Journal
• Naumann…Polly et al, (2016). Virtual Patient Consultations and the use of ePortfolios to support student learning of
integrated professional skills. (Focus on Health Professional Education, under final review).
• J.L. Yang, P. Polly, T. Fath, N. Jones, J. Power (2016). ISA Model and Integrative Career Development Learning in Year 3
Science Courses. The International Journal of Science, Mathematics and Technology Learning (Vol 23, Issue, 3, p33-46).
Book Chapters
• P. Polly, et al (2016). ePortfolios, Assessment and Professional Skills in the Medical Sciences at UNSW Australia. In
ePortfolios in Australian Universities. Springer.
• P. Polly, et al (2015). Creative teaching, learning and assessment in Medical Science: ePortfolios to support skills
development in scientists beyond just knowing their own discipline content. Capturing Creativity through Creative
teaching. The Learner Series.
Notas del editor
This project started with asking ourselves about why we would use ePortfolio at UNSW. We wanted to build graduate capabilities and address UNSW institutional priorities for UNSW graduates in Medical Science……we started our process in embedding and aligning graduate attributes (scholarship, leadership, professionalism, global citizenship for professional skills development, career awareness and employability building) into programs, improving program coherence (cross discipline integration of ePortfolio), and improving the quality of both informal and formal learning (e-learning and e-assessment). All of this relied in evidencing effects of students transitioning from passive to active learners….ePortfolio was the perfect way to capture how students built their professional skills
It will also address the most recent UNSW key priorities of L&T (2012-2016), improve the student learning experience and student learning outcomes in addition to enhancing curriculum by increasing the level of effective use of technology to enable and support learning and teaching (UNSW Blueprint to Beyond). Teaching and learning beyond the walls of our institution should encompass new thinking about pedagogies and new ways of developing courses for students for impact beyond graduation. In addition, the warranty of learning outcomes has largely been overlooked. The use of ePortfolio will address this need not only through course use but ultimately at a program level in order to align graduate attributes and TLOs with graduate employability (Figure 2).
Further, this project aligns specifically with current review of the Medical Science Program by its Management Committee (chaired by Cox), in terms of the adoption of TLOs proposed by CUBEnet, a national discipline-based community of practice, working under the auspices of the Australian Council of Deans of Science (http://www.acds.edu.au/tlcentre/wp-content/uploads/2013/05/Biomedical-Science-Draft-TLOs-and-Development.doc).
Figure 2. Dual strategies for ePortfolio implementation and use in the Medical Science program at UNSW Australia. A longitudinal program-wide approach has been piloted from years 1-4, encouraging students to develop reflective practice over time. A transverse cross-context, cross-discipline approach has been implemented in year 3 courses to develop professional skills and career capabilities and awareness. This resulted in successful institutional funding to further develop grad attributes in team work
The following tables show 2nd and 3rd year courses within SOMS that have research focussed activities that can lead into these research intensive courses
But most importantly it shows that we so a lot of team work skills building acrtivities that can be aligned to develop this graduate attribute
This meant more teacher involvement and more effective student engagement in developing skills when studnets are made aware of our teaching intention