These are the slides for our free course. You can find the course on Udemy at:
https://www.udemy.com/academic-program-development-and-accreditation/
and the YouTube Course Playlist at:
https://www.youtube.com/playlist?list=PLXa3JWoXGD0VhgBZxVBfZUmt49heXPnhh
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Academic Program Development and Accreditation Course
1. Academic Program Development
and Accreditation
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
2. 4 Course Series in Udemy
4. Faith and
Learning
Integration
2. Instructional
Design for
Online and
Blended
Courses
3. Academic
Program
Development
1. Disruptive
Innovation in
Higher Education
3. Course Outcomes
After completing this course, you should be able to:
Design courses in Udemy (or in an unaccredited organization) for academic
credit
Design an effective higher education academic degree or certificate program
Collect stakeholder feedback and conduct online research of similar programs
to determine effective program design
Develop program outcomes and an outcome map to courses
Develop values integration for those in Christian higher education, ministry,
missions agency or churches providing high-quality unaccredited, semi-
accredited and non-formal education
Apply lean startup principles and agile methods to academic program design
to adapt to resource-constrained environments
Complete the documentation needed to submit an academic program for
review to an accreditor (or design program documentation to support
alternative methods to accreditation in contexts where that is needed)
4. Target Audience
Udemy course designers that would like to create courses for credit
Educators that would like to get their programs accredited
Department chairs or other designers of accredited degree or certificate
programs
Udemy course developers wanting their students to get academic credit for
their courses
Educational program directors at non-accredited institutions wanting their
programs to be able to be accepted as transfer credit to accredited institutions
A particular focus will be on Christian educators and/or those serving the poor
or in developing countries
Educators interested in developing online or blended learning for adult learners
Those who value pragmatism and cost-effectiveness in education
Potential partners with City Vision University that would like to co-develop a
program to submit for accreditation
5. Course Requirements and Corequisites
Course requirements. It would be helpful to:
◦ Have some experience with developing educational programs and a general
idea of goals and potential courses that would make up your academic
program
◦ Have some experience with teaching or developing courses
6. What Makes this Course Unique
1. This course uses proven disruptive innovation models in
academic program design.
2. This course is designed around constructivist learning
philosophy so that you will learn program design through building
a program to meet accreditation requirements.
3. Second, this course will provide very practical Creative Commons
worksheets and templates for you to use and reuse in building
your program and seeking accreditation.
7. How You Can Help
Our goal is to significantly expand access to higher education
globally
◦ This course is largely a volunteer driven project of Christian Higher
Education Innovation Alliance (cheia.org), which is a charity-driven
collaboration of 100+ leaders to enable the global growth of post-secondary
education serving the majority world and the poor.
Ways you can help and participate
◦ Contribute templates, resources and suggestions to improve the course
◦ Spread the word about this course
◦ If interested, join CHEIA’s Google Group
◦ Take our other free courses
8. How to Design Courses in Udemy
for Academic Credit
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
9. MIT’s edX Micromasters in Supply Chain Management as a Model
243,000 Free edX
non-credit MOOC Participants
1,900 Certificate
Students
(pay $1,350,
622 complete)
40 Degree
Students
10. City Vision MOOC Certificate Strategy and Funnel
Free MOOC Students in Udemy
Accredited Instance of Udemy
Courses in LMS
Complete Certificate
Degree
Programs
11. Design Tips for Developing Accredited Courses in Udemy
Design so the accredited version is as similar as possible to Udemy version
◦ View Udemy course as courseware assigned in your accredited course in your Learning
Management System (LMS)
◦ Design Udemy course with the same number of sections as you have weeks in your LMS
course, and assign one Udemy section per week in LMS.
Weekly design in LMS
◦ Assign Udemy section for that week as required materials
◦ Mirror Udemy assignments in LMS for each week
◦ Enter grades in LMS with comments or links to comments
Assessment Design
◦ Project-based courses work best, given Udemy’s design
◦ Consider using Google Classroom in parallel if needed to assist students in creating
templates and to allow for collaborative editing
◦ Use peer response assignments rather than forums since Udemy does not support forums
12. Strategy for Accredited Institutions
Submit courses to accreditor (as required) explaining that the
Udemy instance of your course is courseware
◦ Many schools use courseware like Pearson and McGraw Hill, so accreditors
and government regulators have a framework for handling courseware
Timing Consideration
◦ Students can take Udemy course at any time
◦ Students sign up for accredited course like they would any other course and
need to follow timeline for that course
Government Aid Consideration
◦ US Federal Student Aid (Title IV) will not allow federal aid for self-paced courses. To
address this, only allow students to access each week’s assignment after that week opens.
◦ Competency-based or self-paced courses will allow for students to submit entire course
assignments at once.
13. Strategy for Unaccredited Organizations
Design your course so it looks like an accredited course (syllabus, hours
requirements).
◦ US: Carnegie hour definition 45 hours of activity per credit hour.
◦ ECTS: one credit typically 25 to 30 hours of work.
When possible, design your course to be project-based or portfolio-based.
◦ Most schools will have a prior learning portfolio process, so design the final project to be
able to serve as the portfolio.
Provide institutions accepting your credit with documentation similar to what
would be provided to an accreditor.
◦ Use standard course names and course outcomes. Show that your course meets standard
course outcomes.
Seek out partners that will provide articulation agreements for your credit.
Consider alignment with ACE Credit Recommendations, NCCRS, CLEP, AP or
other credit exams
14. Proven Unbundling/Rebundling Models to Get
Credit for Unaccredited Courses & Programs
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
15. Tools and Models for Transnational and Alternative Credit Paths
Non-Accredited
Programs
1. Non-
Government
Recognized
Accreditation
(i.e. ICETE)
2. Articulation
Agreements
3. National
Vocational
Qualifications
4.
Transnational
Validation
and
Franchises
5. Prior
Learning
Portfolios
6-8. MOOCs,
Standarized
Exams, Etc.
See my Clayton Christensen Institute article (Part 1, Part 2)
16. Open Source Software & Open Courseware Business Models
Open Source
Software
(Linux, Drupal)
Customized Products &
Consulting
(Red Hat/IBM, Acquia)
Open Courseware
(ThirdMill, Saylor
City Vision MOOCs)
Customized Academic
Programs & Consulting
(City Vision)
17. ThirdMill
Courses
City Vision
MOOCs
City Vision
Accredited
Courses
Custom Program Design Collaboration with City Vision
and Partner Submitted to Accreditor as Needed
Partner
Developed
Courses
Other
Partners
Standard
ThirdMill
Programs
EMI
Program
St. Matthew’s
Program
Culturally
Contextualized
Custom Programs
Partnering &
Consulting
Courses
Unbundling, Rebundling and Custom Program Development
Saylor
Academy
Degree Path
Saylor
Courses
18. City Vision’s Ministry Certificate Path Partnership with ThirdMill
600,000 Free ThirdMill
non-credit MOOC Participants
100 Certificate
Students
(pay $1,200 in total)
Degree
Students
19. City Vision’s Degree Path Partnership with Saylor Academy
Saylor Academy Students
(millions)
Saylor Associate’s
Degree Path
(pay $2,000 in total)
Bachelor’s
Path
($5,000 in
Total)
20. ThirdMill
Courses
City Vision
MOOCs
City Vision
Accredited Courses
Collaborative Custom Program Design by City Vision and EMI
Partner
Developed
Courses
12 Credit Custom Graduate Certificate
in Christian Studies for
Engineering Ministries International
Culturally
Contextualized
Custom Programs
Partnering &
Consulting
Courses
Custom Certificate Path for Engineering Ministries International
• Bible
• Theology
• Vocation, Calling & Purpose of Work
• Theology of Technology
• Cross-Cultural Management & Ministry
Non-Credit Students
Taking Free Versions
21. ThirdMill
Courses
City Vision
MOOCs
City Vision
Accredited Courses
Collaborative Custom Program Design by City Vision and Wheeler
Prior Learning Assessment Process
Partner
Developed
Courses
Custom Degree Path for
Wheeler Mission
Culturally
Contextualized
Custom Programs
Partnering &
Consulting
Courses
Custom Certificate Path for Wheeler Mission
• Biblical Counseling Program
22. ThirdMill
Courses
City Vision
MOOCs
City Vision
Bible Courses
Collaborative Custom Program by City Vision and St. Matthew’s
Designed for Blended Learning with Local Facilitation
Partner
Developed
Courses
Custom Certificate Path for St.
Matthew’s House in Ministry or
Addiction Counseling
Culturally
Contextualized
Custom Programs
Partnering &
Consulting
Courses
Custom Certificate Path for St. Matthew’s House
(in development)
• The 12 Steps: a Spiritual Journey• Vocation, Calling & Purpose of Work
• Addiction Counseling Certification
Non-Credit Students
Taking Free Versions
23. Who City Vision Partners with on Joint Program Development
Based on our mission, we typically seek out faith-based partners
that are serving
◦ The poor, addicted and underserved
◦ The developing world
Three Options for partnering
◦ Option 1: You develop unaccredited training for evaluation for credit by City
Vision (Wheeler Mission, YWAM UofN, TUMI, Mission Year)
◦ Option 2. You co-design a program path using a combination of City Vision
courses and new custom courses we get accredited (Engineering Ministries
International, St. Matthews)
◦ Option 3. We co-design an entirely new program for accreditation (Thirdmill
Seminary)
If interested in partnering, contact us at
web.cityvision.edu/partnering/
24. Academic Program Planning
and Assessment
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
25. Organizational Strategic Planning Process
Situation
Assessment
• Organizational Assessment
• Analysis (SWOT, etc.)
Goals &
Outcomes
• Mission, Vision & Values
• Institutional Level Outcomes: high level goals
• Measurable Detailed Goals (SMART) & KPIs
Implementation
Plan
• Strategic Steps & Milestones
• Work Plan & Specifications
26. Relationship between Strategic Plan & Institutional
Effectiveness Plan
Source: Nichols, J. O., & Nichols, K. W. (2005). A Road Map for Improvement of Student Learning And Support Services Through Assessment. Agathon Pr.
27. Hierarchy, Theory of Assessment & Backward Design
Mission
Administrative &
Educational
Support Outcomes
Institutional
Learning
Outcomes
Program Learning
Outcomes (PLO)
Degree A
Course Learning
Outcomes (CLO)
Course A
Course Learning
Outcomes (CLO)
Course B
Unit Learning
Outcomes (ULO)
Assessment
Assessment
Signature
(Summative)
Assessment
Unit Learning
Outcomes (ULO)
Assessment
Assessment
Signature
(Summative)
Assessment
Program Learning
Outcomes (PLO)
Degree B
General
Education
Learning
Outcomes
Each Level Needs to:
• Fit within the scope of the
level above it
• Have Outcome Measures
to Prove that the Outcomes
Above it Were Achieved
Proof
Content
Content
Content
Content
• Backward design means
Outcomes first, then
assessment, then
content
28. Program Learning Outcomes (PLOs)
Often “flow” from Institution Level Outcomes (ILO) and/or mission.
Should be clearly defined and measurable.
Should be reasonably attainable.
Include the development of skills, providing job-related training, the
imparting on knowledge and information, the training in the
application of knowledge and skills and the development of
desirable habits and attitudes.
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
29. Course Learning Outcomes (CLOs)
Describe what learners should be able to know or do at the
conclusion of a specific course. They include what the learner will
not only be able to do, but also how well they can do it and under
what conditions
Support the mastery of PLO
Should link up (“map”) to the PLO – and ILO/Mission
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
30. Academic Program Design Models
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
31. ADDIE Model
Image Source: https://commons.wikimedia.org/wiki/File:Skema_ADDIE.jpg
YouTube Video on ADDIE Model
32. Importance of Adaptive Design:
Traditional ADDIE (Waterfall) Backward Program Design
Audience is Traditional Students
Outcomes for Well-Defined Fields
Assessments Based on Known Outcomes
Instruction with Known Content Available
Evaluation
Feedback
Iteration
Is Years
33. Technical vs Adaptive Challenges
Technical Challenges
Clearly defined
Can be solved by experts
Can be resolved in short(er)
time
Can be solve through authority
Requires information learning
without paradigm shift
Adaptive Learning
Problem and approach
undefined
Not necessarily solvable
Long-term process solution
Solved in partnership of leader
and community
Must develop new paradigm
Source: Leadership on the Line, 2002, by Martin Linsky, Ronald A. Heifetz
34. Successive Approximation Model (SAM)
Provides a more iterative, lean approach with faster feedback cycles than traditional ADDIE
Image Source: https://ilite.wordpress.com/2015/03/11/iterative-e-learning-development-with-sam/
YouTube Video on SAM - Successive Approximation Model
35. Importance of Adaptive Design: Optimal Design Methodology
Changes Based on Environment
DESIGN APPROACH
PROBLEM:
We know what customers
want
(the field doesn’t change much)
SOLUTION:
We know how to deliver it
(tech & resources don’t change
much)
Waterfall √ √
Agile √ no
Lean Startup no no
Source: http://www.slideshare.net/NatalieHollier/lean-strategymeetup-small/
36. Project Management: Waterfall vs. Agile vs. Lean Design
Source: http://www.slideshare.net/NatalieHollier/lean-strategymeetup-small/
(Traditional ADDIE & Assessment Plans)
37. Lean Startup Process
Build
MeasureLearn
Product
(start with MVP)
Data
Pivot
Maximize
Loop
Iteration
Speed
Ries, E. (2011). The Lean Startup: How Today’s Entrepreneurs Use Continuous
Innovation to Create Radically Successful Businesses (First Edition). Crown Business.
38. The Challenge: Rapid Growth of Higher Education Globally
100 Million
Students
in 2000
263 Million
Students
in 2025
(84% of growth in
the developing world)
Sources Karaim, R. (2011). Expanding higher education: should every country have a world-class university. CQ Global Researcher, 5(22), 525–572.
Lutz, W., & KC, S. K. (2013). Demography and Human Development: Education and Population Projections. UNDP-HDRO Occasional Papers,
(2013/04). Retrieved from http://hdr.undp.org/sites/default/files/hdro_1304_lutz_kc.pdf
137 Million New Students in Developing Countries by 2025
4.9 billion middle class globally by 2030
39. Priorities for Project Management Triangle
for Course Design Vary Based on Environment
Practical Application/
Adaptive Revision Frequency (time)
Cost Quality (scope)
Developing
Countries
Western
Countries
1. Fixed Cost Design vs. Fixed Quality Design
How does design change if cost must be 1/10th or
1/100th (as in Ethiopia)? 3. Quality and Cultural Contextualization
Since the foundation of most materials is for
middle-class Western audiences, cultural
contextualization is a much larger part of
the definition of quality outside of Anglo
contexts.
2. More Adaptive Design & Practical Application
Cultural minorities with limited resources often
face environments with more unknowns than
Western educators who are in a more homogeneous
globally dominant culture, thus requiring more adaptive
design.
40. How Do You More than Double Access to Higher Education in 25 years?
Design for 4 Interrelated Uncertainties
Changing
Students
Changing
Goals
Affordable
Content
Availability
Costs
Different students based on different
goals, content and costs.
$1,000 degree vs. $10k degree
vs. $100k degree
What goals are realistic
given the students,
costs and content?
Different costs, goals and students
will present different content options +
content & platforms are rapidly changing.
Different content availability,
goals and students will allow
radically different costs.
41. Academic Program
Needs Analysis and Research
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
42. Methods of Needs Analysis
Mostly done by larger institutions in large degree markets
◦ Purchase industry research reports
◦ Local government or industry agencies and research studies
◦ Surveys, Focus Groups, Working Groups
Smaller Institutions or Adaptive Problems
◦ Lean Startup Methods
◦ Online market research
◦ Interviews
All Institutions
◦ Comparable programs
◦ Assessment models
◦ Licensing standards
43. Program Needs Analysis
and Research Process
Program
Outcomes,
Courses &
Curriculum
Map
1. Org
Mission,
Objectives,
Faculty Input
&
Institutional
& GE
Learning
Outcomes
2. Needs of
Students &
Employers
(Advisory
Board)
3. National,
Industry and
Licensing
Standards
4. Research
Comparable
Programs:
Courses,
Outcomes,
Concentration
s &
Sequencing
External
Stakeholder
Needs
Internal
Stakeholder
Needs & Priorities
External
Standards
Learning
from
Field
44. Needs Analysis:
Aligning Program Outcomes with General Education,
Institutional Learning Outcomes and Standards
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
45. Program Needs Analysis
and Research Process
Program
Outcomes,
Courses &
Curriculum
Map
1. Org
Mission,
Objectives,
Faculty Input
&
Institutional
& GE
Learning
Outcomes
2. Needs of
Students &
Employers
(Advisory
Board)
3. National,
Industry and
Licensing
Standards
4. Research
Comparable
Programs:
Courses,
Outcomes,
Concentration
s &
Sequencing
External
Stakeholder
Needs
Internal
Stakeholder
Needs & Priorities
External
Standards
Learning
from
Field
46. Hierarchy, Theory of Assessment & Backward Design
Mission
Administrative &
Educational
Support Outcomes
Institutional
Learning
Outcomes
Program Learning
Outcomes (PLO)
Degree A
Course Learning
Outcomes (CLO)
Course A
Course Learning
Outcomes (CLO)
Course B
Unit Learning
Outcomes (ULO)
Assessment
Assessment
Signature
(Summative)
Assessment
Unit Learning
Outcomes (ULO)
Assessment
Assessment
Signature
(Summative)
Assessment
Program Learning
Outcomes (PLO)
Degree B
General
Education
Learning
Outcomes
Each Level Needs to:
• Fit within the scope of the
level above it
• Have Outcome Measures
to Prove that the Outcomes
Above it Were Achieved
Proof
Content
Content
Content
Content
• Backward design means
Outcomes first, then
assessment, then
content
47. Alignment of Outcomes with Standards
US: Lumina’s Degree Qualifications Profile provides standard
learning outcomes for all Associate’s, Bachelor’s and Master’s
degrees
Europe: Bologna Tuning Process is an agreement between
European countries to ensure comparability of higher education
qualifications
National Qualification frameworks
Accreditors standards
Goal should be to align with standards to the extent required
and where they support your mission and outcomes
51. National Vocational Qualifications
150 countries around the world now have national qualifications
frameworks or NQFs
Often are divided into vocational and university level qualifications
Much more modular and flexible than degrees
Typically tied to international standards dominated largely by the
UK and Europe
Source: http://www.universityworldnews.com/article.php?story=20170328000916417
52. European Qualification Framework (EQF)
Source: https://en.wikipedia.org/wiki/European_Qualifications_Framework
Level Knowledge Skills Example
In the context of EQF, knowledge is described as
theoretical and/or factual.
cognitive (involving the use of logical, intuitive and creative thinking) and
practical (involving manual dexterity and the use of methods, materials,
tools and instruments).
Level 1 Basic general knowledge basic skills required to carry out simple tasks (UK) RQF entry level 3.
Level 2 Basic factual knowledge of a field of work or study
basic cognitive and practical skills required to use relevant information in
order to carry out tasks and to solve routine problems using simple rules
and tools
(UK) GCSE Grades D-G, RQF Level 1,
Lower secondary school
Level 3
Knowledge of facts, principles, processes and general
concepts, in a field of work or study
a range of cognitive and practical skills required to accomplish tasks and
solve problems by selecting and applying basic methods, tools,
materials and information (UK) GCSE Grades A*-C, RQF level 2
Level 4
Factual and theoretical knowledge in broad contexts within
a field of work or study
a range of cognitive and practical skills required to generate solutions to
specific problems in a field of work or study
(UK) A-level, RQF level 3, vocational
school
Level 5
Comprehensive, specialised, factual and theoretical
knowledge within a field of work or study and an awareness
of the boundaries of that knowledge
a comprehensive range of cognitive and practical skills required to
develop creative solutions to abstract problems
(UK) HNC, HND, Foundation Degree,
RQF levels 4 & 5, Certificate of Higher
Education, Diploma of Higher
Education,
Level 6
Advanced knowledge of a field of work or study, involving a
critical understanding of theories and principles
advanced skills, demonstrating mastery and innovation, required to
solve complex and unpredictable problems in a specialised field of work
or study
(UK) Bachelor's degree, RQF level 6,
Graduate Certificate, Graduate
Diploma;
Level 7
Highly specialised knowledge, some of which is at the
forefront of knowledge in a field of work or study, as the
basis for original thinking and/or researchCritical awareness
of knowledge issues in a field and at the interface between
different fields
specialised problem-solving skills required in research and/or innovation
in order to develop new knowledge and procedures and to integrate
knowledge from different fields
(UK) Master's degree, Postgraduate
Certificate, Postgraduate Diploma, RQF
level 7;
Level 8
Knowledge at the most advanced frontier of a field of work
or study and at the interface between fields
the most advanced and specialised skills and techniques, including
synthesis and evaluation, required to solve critical problems in research
and/or innovation and to extend and redefine existing knowledge or
professional practice
Doctorate, PhD, Professional
Doctorate, (Italy) Dottorato di ricerca,
RQF level 8.[2]
53. Writing Effective Program Outcomes
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
54. Writing Effective Learning Outcomes
Remember that learning outcomes describe what learners should
be able to know or do at the conclusion of a lesson/unit, course, or
program.
When writing effective learning outcomes, consider the following:
◦ Identify the concept you want students to learn (e.g., scientific method).
◦ Identify the level of knowledge you want students to attain (e.g., application
of scientific method)
◦ Identify the verb that describe the observable behavior you want students to
demonstrate (e.g., apply the scientific method)
◦ Identify any additional context or criteria for the learning outcome (e.g.,
apply the scientific method to the explanation of life in outer space)
◦ Identify the level of student you are developing these learning outcomes for
(e.g., first year undergraduate, master’s, doctoral)
55. Using Bloom’s Taxonomy for Writing Learning Outcomes
Bloom’s Taxonomy is a
classification of educational goals.
Intended as a tool for curriculum
planning, delivery, and assessment.
There are six different levels,
starting with lower order skills like
remembering, understanding to
higher order skills like evaluating
and creating.
Each level of learning has a
corresponding set of actionable
verbs aimed at assessing student
learning.
56. Bloom’s Taxonomy with Action Words
Remember
(recall facts
and basic
concept)
Define
Duplicate
List
Memorize
Repeat
State
Understand
(explain ideas
or concepts)
Classify
Describe
Discuss
Explain
Identify
Locate
Recognize
Report
Apply (use
information in
new situations)
Execute
Implement
Solve
Use
Demonstrate
Interpret
Operate
Schedule
Analyze (draw
connections
among ideas)
Differentiate
Organize
Relate
Compare
Contrast
Distinguish
Examine
Experiment
Evaluate
(justify a
position or
decision)
Appraise
Argue
Defend
Judge
Select
Support
Value
Critique
Create
(produce new
or original
work)
Design
Assemble
Construct
Conjecture
Develop
Formulate
Author
Investigate
57. Writing Effective Learning Outcomes
Verb Noun Context
(optional)
The actionable
verb corresponding
to the different
levels of Bloom’s
Taxonomy
The noun refers to
the knowledge,
skills, and abilities
you want students
to learn.
The context
indicates where,
when, or how the
outcome will be
applied.
Example: Identify (verb) the seven steps of the research
process (noun) when writing a research paper (context).
58. Using Bloom’s for Measuring Learning at the Varying Levels
Based on story of Goldilocks and Three Bears
Remember: Describe where Goldilocks lived.
Understand: Summarize what the Goldilocks story
was about.
Apply: Construct a theory as to why Goldilocks went
into the house.
Analyze: Differentiate between how Goldilocks reacted
and how you would react in each story event.
Evaluate: Assess whether or not you think this really
happened to Goldilocks.
Create: Compose a song, skit, poem, or rap to convey
the Goldilocks story in a new form.
63. Western Bias in Faith-Based Academic Institutions in Bloom’s Domains
Tendency of Western faith-based institutions to significantly
overvalue cognitive domains of learning and to devalue affective
and psychomotor domains
◦ Strongly reinforced by accreditation structures that focus on assessment of
outcomes that are more easily measured
◦ Resulting process teaches objectivism
Need to be intentional at both program level and institutional level
to offset this bias
◦ Affective: Invest in mentoring, practicum, community, service learning,
spiritual formation, journaling, creative expression, affective questions
◦ Psychomotor: Invest more in and grant credit for internships, practicum,
service learning, coops, field experience
Source: Shaw, P. (2014). Transforming Theological Education: A Practical Handbook for Integrative Learning. Langham Global Library.
64. Quotes on Challenges of Faith Integration
“While we teach orally ‘the Word became flesh’, too often we teach
psychologically and methodologically ‘the Word became text’
- Éla 1988
“In theological education, the easier something is to assess, the less
important that something is likely to be.”
- Graham Cheesman
“No truth is taught by words or learned by intellectual means… Truth
must be lived into meaning before it can be truly known.”
- Horace Bushnell 1979
Source: Shaw, P. (2014). Transforming Theological Education: A Practical Handbook for Integrative Learning. Langham Global Library.
65. Importance of Balancing Domains of Learning
Result of Over-Emphasizing One Domain of Learning
Overemphasizing the cognitive domain: pride and irrelevance
Overemphasizing affective domain: ignorant pietism &
emotionalism
Overemphasizing behavioural domain: empty technical excellence
Christian Integration needs to not only integrate faith, but it also
needs
to provide a holistic balance across all three domains of learning to
shape healthy individuals.
Shaw, P. (2014). Transforming Theological Education: A Practical Handbook for Integrative Learning. Langham Global Library.
67. Faith Integration in Course and Program Review Rubrics
Best practice is to have faith-integration items based on an
institutions values in their course and program evaluation rubrics
◦ Typically it is much more important to have deep faith integration at the
program level for every individual course
Can inventory courses and programs by level of faith integration
desired and compare to what level is actually there
◦ See OSCQR+ Rubric Tab
◦ Should tie course review rubrics to Institutional Learning Outcomes and
General Education Outcomes
68. Faith Integration in General Education & Institutional Learning Outcomes
Best practice faith-based institutions often put many of their faith-
integration learning outcomes in
◦ General Education
◦ Institutional Learning Objectives (or Ideal Graduate Profile)
See sample research links
Other Key Tools
◦ International Council for Christian Higher Education
◦ Spiritual Assessment Tools
Spiritual Transformation Inventory (STI)
ABHE Exams for Bible Knowledge or AP Bible Exam from ExamYard
69. Design: Academic Faith Integration
Dr. Michael Truong
Executive Director, Office of Innovative Teaching & Technology
Azusa Pacific University
www.apu.edu
70. What is Academic Faith Integration?
“Faith integration is informed reflection on and
discovery of the relation(s) between the Christian faith
within the academic disciplines, professional
programs, the arts, and lived practice resulting in the
articulation of Christian perspectives on truth and life
in order to advance the work of God in the world.”
– Azusa Pacific University
71. What is Academic Faith Integration?
“Faith integration is informed reflection on and
discovery of the relation(s) between the Christian
faith within the academic disciplines, professional
programs, the arts, and lived practice resulting in the
articulation of Christian perspectives on truth and life
in order to advance the work of God in the world.”
– Azusa Pacific University
72. What is Academic Faith Integration?
“Faith integration is informed reflection on and
discovery of the relation(s) between the Christian
faith within the academic disciplines, professional
programs, the arts, and lived practice resulting in
the articulation of Christian perspectives on truth and
life in order to advance the work of God in the world.”
– Azusa Pacific University
73. What is Academic Faith Integration?
“Faith integration is informed reflection on and
discovery of the relation(s) between the Christian faith
within the academic disciplines, professional
programs, the arts, and lived practice resulting in the
articulation of Christian perspectives on truth and
life in order to advance the work of God in the
world.”
– Azusa Pacific University
75. Strategies for Academic Faith Integration
Compatibility
Strategy
Emphasizes areas of
harmony or compatibility
between discipline and faith.
Applications include 1) using
biblical examples to illustrate
disciplinary concepts (e.g.,
Moses as a leadership
model) or 2) showing
Christianity’s relevance to
disciplinary knowledge (e.g.,
human nature, beauty,
history)
Transformationa
l Strategy
Remakes or transforms the
discipline to align with
Christian worldview.
Applications include 1)
advocating the existence of
truth, reason, and meaning
(e.g., creation of the world)
and 2) upholding biblical
authority in the discipline
(e.g., sin as source of social
ills).
Reconstructioni
st Strategy
Replaces faulty foundation of
the discipline with Christian
worldview. Applications
include 1) employing
Christian worldview as the
organizing principle to
interpret the discipline (e.g.,
replace relativism with
absolute truth) and 2)
replacing disciplinary
assumptions (e.g., replace
random chance with God’s
sovereignty)
*Adapted from Beers and Beers (2008). “Integrating of Faith and Learning” in The Soul of a Christian University: A Field Guide for Educators.
76. Reflective Questions
1. THE WHAT
(the investigation)
• What are the foundational
assumptions of the issue,
course, or discipline?
• How do I know that this
knowledge claim is true?
• Is there an agenda or
ideology behind this
conclusion?
• What is the worldview
behind or implied by the
claim?
2. SO WHAT?
(the interpretation)
• How does the claim fit in
with the Christian faith?
• How could my worldview
act as a filter to evaluate
this subject?
• What ethical questions
does the
knowledge/expertise in this
subject raise?
• What does this subject tell
us about God?
3. NOW WHAT?
(the application)
• Where is the hope here?
• How might we reclaim this
area for the glory of God?
• How could the restoration
of this issue be a signpost
for the kingdom of God?
• How could learning in this
subject affect my faith
development?
*Adapted from Beers and Beers (2008). “Integrating of Faith and Learning” in The Soul of a Christian University: A Field Guide for Educators.
77. Integrative Questions
English: What are the similarities and differences in interpreting the biblical
texts and interpreting other literature texts?
Political Science: What is the role of forgiveness in international relations?
Fine Arts: What are the limits, if any, on the freedom for human creative
expression?
History: How do alternative views on the “direction of history” (e.g., linear,
cyclical, teleological) fit or not fit with the Christian narrative?
Economics: What is the relationship between the quest for profitability and the
Christian call for compassion and justice?
Physics: What are the similarities and differences between the use of models
in scientific inquiry and the use of models in theological inquiry?
Education: What is the relationship between subject-centered and student-
centered teaching pedagogies in light of a Christian perspective on
personhood?*Adapted from Beers and Beers (2008). “Integrating of Faith and Learning” in The Soul of a Christian University: A Field Guide for Educators.
80. Curriculum Maps in the Assessment Logic Model
Mission
Administrative &
Educational
Support Outcomes
Institutional
Learning
Outcomes
Program Learning
Outcomes (PLO)
Degree A
Course Learning
Outcomes (CLO)
Course A
Course Learning
Outcomes (CLO)
Course B
Unit Learning
Outcomes (ULO)
Assessment
Assessment
Signature
(Summative)
Assessment
Unit Learning
Outcomes (ULO)
Assessment
Assessment
Signature
(Summative)
Assessment
Program Learning
Outcomes (PLO)
Degree B
General
Education
Learning
Outcomes
Curriculum Maps Show the
Linkage Between Program
Learning Outcomes and
Course Learning Outcomes
Proof
Content
Content
Content
Content
Syllabus Typically Shows the
Linkage Between Course
Learning Outcomes and
Unit Learning Outcomes
Signature or Summative
Assessments Are Foundational
to Curriculum Maps
81. Goals of a Curriculum Map
Demonstrate Alignment (within a program, between general
education and institutional goals, etc.)
Identify where and how particular outcomes are expected, explicitly
taught for, and assessed (Ewell, 2013)
Allow for backwards design of the curriculum
Identify role of course pre-requisites
Identify gaps in outcomes/levels (initially)
Identify optimal sequence
Be used as advising tool—identify pathways
Be incorporated in program reviews, a part of
refinements/successes
Serve as Key Tool toward Institutional Improvements
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
82. Signature Assignment/Embedded Assessment
Embedded in courses (shouldn’t require “double grading”)
Aimed at measuring Program Learning Outcomes as well as
course outcomes
Often the same assignment and instrument for each course
◦ Rubric
◦ Exam
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
83. Curriculum Mapping Process
Identify PLO’s
Identify courses where PLOs are demonstrated by students
Identify signature assignments and assessments
Identify levels of learning (where does “mastery” occur?)
Create matrix (the Curriculum Map)
Identify (and fix) gaps in outcomes, assignments, assessment
Ensure scaffolding/alignment
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
84. Curriculum Map Example
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
85. Questions to Evaluate Your Curriculum Map
Are all outcomes presented, in a logical order?
Do all the key courses address at least one outcome?
Do multiple offerings of the same course address the same
outcomes, at the same levels?
Do some outcomes get more coverage than others?
Are all outcomes first introduced and then reinforced?
Do students get practice on all the outcomes before being
assessed, e.g., in the capstone?
How are electives/pre-reqs figured in?
Source: Nuts and Bolts of Curriculum Mapping. Presentation at DEAC Fall Workshop October 22, 2018. Dr. Errin Heyman
86. Developing Program Assessment Plans in
Formats Used by Accreditors
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
87. Institutional Effectiveness Paradigm & the Five Column Model
Source: Nichols, J. O., & Nichols, K. W. (2005). A Road Map for Improvement of Student Learning And Support Services Through Assessment. Agathon Pr.
88. Sample 5 Column Model for Program Assessment
Undergraduate Nonprofit Management Program Five Column Model
Expanded Statement of
Institutional Purpose
Program Intended Educational
Outcomes
Means of Assessment and
Criteria for Success
Summary of
Collected Data
Use of Results
Institutional Mission Statement:
to provide radically affordable
Christian education to underserved
communities via distance learning
Institutional Goal:
to provide practical undergraduate
education to Christians to equip
them in their field in social service
and social entrepreneurship
careers.
1. Understand and apply the roles
and responsibilities of a nonprofit
board of directors and the
management team to provide
governance and leadership to the
nonprofit organization.
2. Understand and apply basic
accounting and budgeting principles
in order to successfully manage the
finances of a nonprofit organization.
3. Understand and apply basic
marketing, communication and
fundraising principles in operating a
successful nonprofit organization.
4. Understand and apply nonprofit
management principles related to
program development, ethics,
decision-making and nonprofit legal
and regulatory requirements.
5. Understand and apply the
essential elements of nonprofit
human resource management
including volunteer management,
hiring, firing, supervision and legal
considerations.
80% of students shall receive a
pass rate score of 70% for final
projects and final exams tied to
assessing educational outcomes.
(direct)
Job placement rate of graduates.
(indirect)
72.3% of students have achieved a
pass score of 70% for final projects
and final exams tied to assessing
educational outcomes.
95.5% were placed in training
related jobs (77.3%), other jobs
(4.5%) or went on for advanced
degrees (13.7%).
Reviewed pass rates for each
courses. Identified problems with
one faculty and one courses Joy at
Work. Course is being rewritten and
considering replacing faculty.
Developed formal survey and
interview questionnaire for
graduates and employers to provide
more detailed data in future.
89. Course and Program Evaluation Rubrics
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
92. Developing Key Program Processes, Policies &
Documents Needed for Accreditation
Dr. Andrew Sears
President, City Vision University
www.cityvision.edu
andrew@cityvision.edu
93. Final Project: Prepare a Degree Proposal for an Accreditor
Recommend using the Distance Education Accrediting
Commission’s application, but may use templates from other
accreditors
◦ Assignment: Download and read through Change in Educational Offerings
(Degree Programs Part 2) from https://www.deac.org/Seeking-
Accreditation/Applications-and-Reports.aspx
Key Components of Proposal
◦ Development of program outcomes
◦ Curriculum map between courses and program outcomes (attachment)
◦ Degree Program Comparison (attachment)
◦ Understand and apply educational theory needed to answer key questions in
the accreditation proposal
◦ Develop other key attachments needed in the proposal
94. About the Distance Education Accrediting Commission (DEAC)
The oldest and premiere US accreditor focused on distance and
online education (www.deac.org)
DEAC will accredit institutions outside the US
Why use DEAC in this course?
◦ DEAC is a global model for best practices in accreditation of online education
that could be translated in other accreditation contexts
◦ DEAC has extremely thorough documentation and training material needed
◦ Could still use another accreditor if desired
Note the scope of this course is to focus on the degree proposal
component of overall accreditation
◦ The full Self Evaluation Report for accreditation is beyond the scope of this
course (but this course could get you halfway there)
95. Key Resources from DEAC
Assignment download and Read through Change in Educational
Offerings (Degree Programs Part 2) from
https://www.deac.org/Seeking-Accreditation/Applications-and-Reports.aspx
DEAC’s Self Evaluation Resource Template at:
https://www.deac.org/Seeking-Accreditation/Applications-and-Reports.aspx
DEAC’s Preparing for Accreditation Free Online Course
◦ https://www.deactraining.org
◦ There is also a free Course for Accreditation Evaluators
Editor's Notes
73% of college students are non-traditional.
Increasingly, we are preparing students for jobs that don’t exist.
How can you prepare students if you don’t know what fields you are preparing them for. Teach how to think. That is some of it, but also need to stay current.
90% of the world’s data was generated in the last 2 years. Each year I do my courses, there are radically different content and platforms available.
Pace of change is accelerating
Top 1% of global income means you make more than $32,400.
Cost plus pricing (traditional) vs. target costing
Tertiary Education Stats from: https://www.youtube.com/watch?v=tOTjtsrKOqI
By 2050, between 1 and 2.5 billion people will have a tertiary education.
Market: $900 billion market in 2005, $1.5 trillion in 2012, $2.5 trillion 2017
# students outside Western countries: 30 million in 1980, 140 million in 2010
84% of growth from developing countries from 2000 to 2010
Mexico 1.9 million to 2.8 million in past decade
India: under 10 million to over 20 million in past decade
Gangel, Kenneth o. 1978. Christian higher education at the end of the twentieth century, Part 2: Integrating faith and learning: Principles and process, Bibliotheca sacra 135.0538, 99-108. Available from ATLA Serials 2002. Retrieved 7 November 2003 from http://66.77.30.29/pls/elil pshow?lcookie=915583&pid=1000474&lmode=ME&lfirstp=99.