This document discusses evaluating creativity programs. It recommends using both quantitative and qualitative data from a range of measures to assess outcomes. Evaluations should include pre- and post-comparisons, and consider both short-term and long-term outcomes. Specifically, evaluations should focus on a range of outcomes including creative production, personality, attitudes, and environmental factors. Using multiple measures can address weaknesses while strengthening evaluations.

1. Teaching and Evaluating for
Creativity and Innovation
Jonathan A. Plucker
IEFE 2013
February 18-23, 2013

2. l Not discussing giftedness today, focusing
on creativity in schools
l Useful web site for conceptions of
giftedness:
l http://www.intelltheory.com

3. Overview
l What is Creativity?
l Major Types of Creativity Assessment
l Quick Activity
l Evaluating Creativity
Programs

4. What is Creativity?

5. Why are there so many myths
about creativity?
l Creativity is a construct that fascinates
people.
l Most cultures revere creative works and
the people who produce them.
l Both the creators and our societies like to
keep the creative process mysterious.
l Research has reinforced these myths
(unintentionally).

6. The #1 Myth …
l Creativity cannot be enhanced.

7. … Enhanceable?
l You would be surprised at the number of
people who believe that myth.
l In fact, a surprisingly large % of my
students don’t believe they are creative.
l EVEN IN CREATIVITY COURSES!

8. Should you...
Take Risks?
 Myth or Reality? Risk-taking is
commonly associated with
creativity.
Myth!!!
(sort of)

9. Take Risks ...
l Not all risk is created equal.
l Blind risk-taking leads to injury and
failure as often (if not more so) than
to success.

10. Take Risks ...
l “Quitters never win, winners never
quit, but those who never win and
never quit are idiots.”
l Risk management is associated with
long-term creative production.

11. Should you...
Avoid Evaluation and
 Myth or Reality? Creativity is
enhanced when evaluation and
external constraints are
minimized.
External Constraints?
Myth!!!

12. Avoid Constraints ...
l How often in life do you work
without constraints?
 Many of the most creative things
you’ve done have resisted the advice
of others!
l Draw a penny
l Amabile replication example

13. Constraints ...
l Learn how to work creatively within
the constraints imposed both by
yourself and others.
l When working with others, use
sensible constraints and make sure
that your expectations are clear.

14. An Alternative Model
l Our work is based on the belief that we
can make any person, any group, any
family, any company, any classroom
more creative.

15. A Definition of Creativity Should Be
Able to Explain …
l … that creativity may look different in
different contexts.
l … that creativity is the result of a
diverse set of influences.
l … that creativity is often in the eye of
the beholder.

16. A Definition of Creativity Should Also
Be Able to Explain …
l … how the behaviors of a 10-year-old
may be creative while the same
behaviors by a 40-year-old may not
be creative.
l … why the work of schizophrenics
may be original but not creative.

17. Our Definition
l Creativity is the interaction among
aptitude, process, and environment by
which an individual or group produces a
perceptible product that is both novel and
useful as defined within a social context.
l Plucker, Beghetto, & Dow (2004)

18. For example …

20. Creativity and Intelligence
Creativity and intelligence are
related….
But we are not sure how strong the
relationship is.

21. Creativity and Intelligence
Some new conceptions of intelligence include
creativity as an integral part.
For example,
Sternberg’s Triarchic Model of Successful
Intelligence:
Analytic
Practical
Creative

22. Major Types of Creativity
Assessment

24. How do we measure Creativity?
1. Ask people how creative they are
Pros: Easy, More reliable/valid than
you might think
Cons: People’s perceptions are often
wrong, Very easy to fake if it’s “high
stakes”

25. How do we measure Creativity?
2. Give people a creative personality
test
For example:
“I like to solve complex problems”
“I have a vivid imagination”
“I love to daydream”

26. How do we measure Creativity?
2. Give people a creative personality
test
Pros: Harder to fake, Fairly reliable
Cons: Still possible to fake, Overlaps
with general personality

27. How do we measure Creativity?
3. Past creative performance
Pros: Also easy, More reliable than
self-report
Cons: Relies on judgment of what to
put and honesty, Assumes student
has had chance to use their
creativity, not useful for young
students

28. How do we measure Creativity?
4. Observation
Pros: A lot more information than
relying on any type of self-report
Cons: People usually don’t like being
observed, People act differently
when observed, Lots of time

29. How do we measure Creativity?
5. A Divergent Thinking Test
Most commonly used:
Torrance Test of Creative Thinking.
Also used:
Guilford’s Alternate Uses Test

30. How do we measure Creativity?
5. A Divergent Thinking Test
For example:
Think of all of the different things
that would happen if people did not
need sleep. List as many as you can
in the next two minutes.

31. How do we measure Creativity?
5. A Divergent Thinking Test
These tests are scored according to
four different criteria:
Fluency:
How MANY did you list?

32. How do we measure Creativity?
5. A Divergent Thinking Test
Flexibility:
How many different CATEGORIES
did you list?

33. How do we measure Creativity?
5. A Divergent Thinking Test
Originality:
How UNIQUE or DIFFERENT were
your choices?

34. How do we measure Creativity?
5. A Divergent Thinking Test
Elaboration:
How DETAILED were your
responses?

35. How do we measure Creativity?
5. A Divergent Thinking Test
Pros: Good reliability/validity, Some
evidence that these tests are related
to positive job attitudes/performance
Cons: Time consuming, More
expensive, Possible to “cheat” if you
know how it is scored

36. How do we measure Creativity?
6. Ratings
People tend to agree on what is
creative and what is not creative,
even if their definitions of
“creativity” are different.

37. How do we measure Creativity?
6. Ratings
Any product can be rated for
creativity -- a poem, a painting, a
mathematical proof, a business plan,
a grant proposal…

38. How do we measure Creativity?
6. Ratings
Raters can be teachers and parents
(for students), or bosses and experts
(for workers)

39. How do we measure Creativity?
6. Ratings
One technique that is often used is
called the Consensual Assessment
Technique.

40. How do we measure Creativity?
6. Ratings
This technique involves analyzing all
of the different products to be rated,
and then assigning a score that
compares the products to each
other, NOT to a perfect ideal
product.

41. How do we measure Creativity?
6. Ratings
Some early research indicates that
even non-experts can produce
reliable and consistent ratings as
long as they have some experience
in the area.

42. How do we measure Creativity?
6. Ratings
Pros: Very reliable, Can evaluate
creativity in many different areas
Cons: Time consuming, Sometimes
hard to find “experts,” Validity open
to question

43. Creativity: Why Bother?
You can get more information about a
student or employee by not only giving
them a personality measure and/or an
ability measure but ALSO a creativity
measure.

44. Illuminating Activity
Instructions during assessment
and evaluation are important.

45. Evaluating Creativity
Programs

46. Recommendations
1. Use both quantitative and qualitative data
2. Focus on a range of outcomes
3. Use a range of measures
4. Use pre- and post-measure comparisons
5. Consider short-term and long-term
outcomes.

47. 1. Qualitative and
Quantitative Data
l We have good quantitative measures at
our disposal.
l But some important aspects of creativity
are hard to assess with an instrument
l So carefully constructed observations and
qualitative surveys should also be used.

48. 2. Range of Outcomes
l Creative production (i.e., “Being creative”)
is the important goal …
l … but we know that changes in creative
personality, creative attitude, and creative
process are necessary for creative
productivity.
l … as are changes to the creative
environment.

49. 3. Range of Measures
l We now have multiple good measures for
most aspects of creativity.
l Using 2 or more helps mediate each
instrument’s weaknesses …
l … and enhances their collective strengths.

50. 4. Pre/Post Comparisons
l Although “pure” experiments are difficult
in applied settings, at the very least these
evaluations should include “before and
after” comparisons.
l Including control or comparison groups is
always a good idea.

51. 5. Short/Long Term Outcomes
l We usually focus on short-term outcomes
when evaluating education interventions.
l Research increasingly suggests that many
short-term outcomes are hard to assess.
l Short-term outcomes may not be present
… but long-term outcomes may emerge
years later.