SlideShare una empresa de Scribd logo
1 de 12
Descargar para leer sin conexión
Feminist evaluation represents an important option in
evaluation, but its legitimacy and credibility depend on
the criteria used to define high-quality evaluation
practice. This commentary examines how feminist
evaluation principles and practices look from the
perspectives of five different frameworks, each offering
different criteria for judging evaluations.
NEW DIRECTIONS FOR EVALUATION, no. 96, Winter 2002 © Wiley Periodicals, Inc. 97
8
Feminist, Yes, but Is It Evaluation?
Michael Quinn Patton
The root problem reappears in different guises in all fields and throughout
the dominant tradition. It is, simply, that the majority of humankind was
excluded from education and the making of what has been called knowl-
edge (Minnich, 1990, p. 37).
Philosopher Elizabeth Minnich has examined in depth in her book
Transforming Knowledge the ways in which throughout history “the domi-
nant few not only defined themselves as the inclusive kind of human but
also as the norm and the ideal” (p. 37). Who are these dominant few? “A
few privileged men defined themselves as constituting mankind/humankind
and simultaneously saw themselves as akin to what mankind/humankind
ought to be in fundamental ways that distinguish them from all others” (p.
37). How did they do this? “[T]hey created root definitions of what it means
to be human that, with the concepts and theories that flowed from and rein-
forced those definitions, made it difficult to think well about, or in the mode
of, anyone other than themselves, just as they made it difficult to think hon-
estly about the defining few” (p. 38). The male pronoun as inclusive and
universal is but one obvious and prominent example.
Minnich, writing as a feminist scholar, has examined how the concepts
and conceptual frameworks we use, whether unconsciously as a matter of
tradition and training or intentionally as a matter of choice, carry embed-
ded messages about what and who is important. She asserts in the opening
quotation above that “this root problem reappears in different guises in all
fields.” Does it appear in evaluation? Let us substitute mankind/humankind
in Minnich’s analysis with evaluator/evaluation and see if we recognize our
own history as a field:
The majority of evaluators were excluded from determining what has been
called knowledge in evaluation. The dominant few not only defined themselves
98 FEMINIST EVALUATION
as the inclusive kind of evaluator but also as the norm and the ideal. A few priv-
ileged men defined themselves as constituting the field of evaluation and simul-
taneously saw themselves as akin to what evaluators ought to be in fundamental
ways that distinguish them from all others. They created root definitions of what
it means to be an evaluator that, with the concepts and theories that flowed from
and reinforced those definitions, made it difficult to think well about, or in the
mode of, anyone other than themselves, just as they made it difficult to think
honestly about themselves as the defining few.
A Feminist Analysis of Evaluation: One Perspective
Since I’m about to presume to offer one version of a feminist analysis of
evaluation, let me start with a little reflexivity about my voice and perspec-
tive, given that reflexivity and attention to voice are among the core contri-
butions of feminist analysis. I write as a white male (or, as I was called in
the Peace Corps by the Gourma people of eastern Burkina Faso, bompieno—
a white thing). I’ve been around since the formal beginning of the profes-
sion we call evaluation. Since 1975, I’ve attended every national conference
of the American Evaluation Association and all predecessor conferences of
the Evaluation Network. I’ve written a bit about evaluation, including some
books, some of which have had some success—“success” being defined as
being widely cited and broadly attacked, not necessarily in that order. As an
evaluation textbook author, I’ve made it a point of my participation in the
profession to observe and reflect on how the field has developed and is
developing, and to discuss those developments with colleagues. That’s a bit
of the background I bring to the question of whether Minnich’s analysis of
what she calls “the root problem” applies to evaluation.
It strikes me as accurate to say that in the early, formative days of the
profession, a dominant few defined evaluation and defined themselves as
the norm and the ideal, and those dominant few were primarily, though not
exclusively, privileged men. They defined themselves as constituting the
field of evaluation and simultaneously saw themselves as akin to what eval-
uators ought to be. They created root definitions of what it means to be an
evaluator that, with the concepts and theories that flowed from and rein-
forced those definitions, made it difficult to think well about, or in the mode
of, anyone other than themselves. Takes me right back to the seventies and
early eighties.
I say “they” rather than “we” because in those days, when utilization-
focused evaluation was an outlier idea and qualitative methods were regu-
larly disdained, the ideas I offered about evaluation were variously criticized
as lacking in rigor, selling out, immoral, soft, naive, romantic, idealistic, and
undermining the foundations of the emergent profession. I responded in
kind, the barbs flew back and forth, debates raged, and the adrenaline
flowed. “Hard” versus “soft” data. Accused of “getting in bed” with those I
was supposed to be evaluating. Ridiculed for not being “tough enough” to
IS IT EVALUATION? 99
tell the truth. Too relationship-oriented. Too formative. (A little formative
is okay, but only as foreplay for the real judgmental stuff: hard-hitting, tell-
it-like-it-is, sum-it-all-up, hard-on, summative evaluation.) Too kind. Too
gentle. I had no idea that aspiring to be an evaluator could cast such doubts
on a young man’s masculinity.
But I digress. I was recalling the litany of concerns I’ve encountered
over the years about various aspects of utilization-focused evaluation, espe-
cially those aspects that involve working closely with intended users to
build capacity for evaluative thinking, facilitate their making their own
judgments instead of simply genuflecting to mine, and helping them apply
evaluation findings to do their own reality-testing and bring about change
in accordance with their own values. The most common, damning, dismis-
sive, and persistent objection I’ve heard is that such activities are all well
and good if I choose to do them, but they aren’t evaluation.
What does this ancient history have to do with feminist evaluation?
Just this: The most common, damning, dismissive, and persistent objection I
hear to feminist evaluation is that it’s not really evaluation. It’s fine for those
so moved and motivated to engage in a feminist agenda of social change.
Just don’t call that engagement evaluation. It’s fine to work on raising con-
sciousness about women’s oppression and strive for women’s liberation, just
don’t do it under the guise of evaluation. Keep evaluation pure, by which is
meant rendering judgments of merit and worth from a dispassionate posi-
tion of independence and objectivity.
Defining Evaluation
It turns out that the various definitions of evaluation don’t just tell you what
you must do; they can be interpreted, and often are, as saying that’s all you
can do. If evaluation is defined as judgment of merit and worth, do that and
do only that. Everything else must be something other than evaluation. If
evaluation is defined as the application of social science methods to deter-
mine program effectiveness, do that and only that. Anything else is of lesser
value or even illegitimate. If evaluation is defined as determining the extent
to which a program achieves its goals, that becomes the priority task and all
else is at best secondary, if even worth the bother.
The power to define is the power to control, to include and exclude.
That is what Minnich was referring to as the root problem that reappears in
different guises in all fields. This root problem is the power of the dominant
few to define what constitutes legitimate activity and real knowledge, and
these dominant few not only define themselves as the inclusive kind of
whatever is being defined, in this case evaluator, but also as the norm and
the ideal. They create root definitions with concepts and theories that flow
from and reinforce those definitions, making it difficult to think well about,
or in the mode of, anyone other than themselves, just as they make it diffi-
cult to think honestly about the defining few.
100 FEMINIST EVALUATION
Definitions matter. Regularly and predictably on EvalTalk (the Ameri-
can Evaluation Association–sponsored listserv) questions emerge and dis-
cussions follow about whether this or that is evaluation. Is evaluation the
same as or different from research? monitoring? performance measurement?
accountability? auditing?
Feminist Evaluation Redefines Evaluation. Feminist evaluation can
include judging merit or worth, using social science methods to determine
effectiveness, and measuring goal attainment, but those activities are means
to a greater end—increased social justice for the oppressed, especially, but
not exclusively, women. And feminist evaluation offers different ways of
judging merit or worth, applying social science methods, or measuring goal
attainment—collaboratively, inclusively, embracing multiple perspectives,
and using evaluation processes and findings to foment change. And therein
lies the rub. If evaluation becomes a means rather than an end and is con-
ducted in ways that intentionally and actively bring about social justice, is
it still evaluation? And who gets to decide?
An Inclusive and Liberal Profession. Who gets to decide? For the
moment and for the purposes of this volume, you, the reader, do. While a
privileged few determined the early definitions, standards, and norms for
evaluation, drawing primarily from traditional social science research meth-
ods and auditing standards, the profession has been too amorphous and
unwieldy to remain so constrained. The 1980s pretty much put an end to
the quantitative-qualitative debate (both hardness and softness are now con-
sidered appropriate, even in mixed company—that is, among both evalua-
tors and nonevaluators). Methodological eclecticism, triangulated methods,
multiple methods, and mixed methods are all in vogue, and woe is s/he who
knows but one design or measurement approach. The 1990s blossomed
with diversity—participatory evaluation, collaborative evaluation, empow-
erment evaluation, inclusive evaluation, developmental evaluation, multi-
vocal evaluation, learning-oriented evaluation, democratic evaluation, and
lately, appreciative inquiry evaluation, to name but a few more prominent
examples. These built on the emergent diversity of the late 1980s, which
had seen the flowering of such approaches as fourth-generation evaluation,
responsive evaluation, constructivist evaluation, and even that 1970s
hanger-on, utilization-focused evaluation. In the context of this contempo-
rary, new millennium of evaluation diversity, feminist evaluation adds
another important option and a distinctly different way of conducting eval-
uations. Or does it? The nagging question planted by the original dominant
few remains: is it really evaluation? And will feminist evaluation be treated
as a legitimate and respected entry in the next edition of the unofficial eval-
uation thesaurus?
Certainly feminist evaluation has gained visibility, as witness this vol-
ume. Of course, this volume appears at a time when political pundits in the
larger society are questioning the contemporary relevance of feminism
IS IT EVALUATION? 101
(defined often by those self-same pundits as nothing more than a throwback
to their limited notion of the supposedly bra-burning sixties—and what
could be more irrelevant to anything at all than the overhyped, under-
achieved, oh-so-tiresome, and not incidentally, pre-evaluation profession
sixties). One is unlikely to hear overt criticisms of feminism among evalu-
ators, although some may have their private doubts. Evaluators are, on the
whole, a politically liberal lot. No, the objections to feminist evaluation do
not center on the feminist part of the terminological equation, at least not
explicitly. The objections flow from concern with the oxymoronic nature of
the combination of the terms “feminist” and “evaluation.” Feminist? Okay.
Fine. Important. Worthwhile. Raise consciousness. Liberate. You go, girl.
But is it evaluation?
Feminist Evaluation Criteria
Of course, questioning whether feminist evaluation is really evaluation may
simply be an underhanded (or more politically correct) way of resisting the
feminist critique and agenda. But for the purpose of these reflections, let’s
take the evaluation question at face value. The question of what constitutes
evaluation requires criteria, as does the even more devilish question of what
constitutes “good” or high-quality evaluation. Credibility flows from those
judgments. Quality and credibility are connected in that judgments of qual-
ity constitute the foundation for perceptions of credibility. Feminist evalu-
ation challenges traditional criteria for assessing evaluations, such as
objectivity, evaluator independence, neutrality, and minimal investigator
bias. Particular philosophical underpinnings or theoretical orientations and
special purposes for evaluation can generate different criteria for judging
quality and credibility, including whether an inquiry approach qualifies
as evaluation.
Alternative Criteria for Judging Evaluation Quality. I have identi-
fied five contrasting sets of criteria for judging the quality of qualitative
inquiry from different perspectives and within different philosophical frame-
works (Patton, 2002, pp. 542–552). For the purpose of this discussion, I
want to adapt and apply those criteria to examine the credibility of feminist
evaluation from different evaluation points of view. For heuristic purposes,
I’ll emphasize the differences rather than commonalities among these alter-
native ways of conducting and judging evaluations. The five contrasting,
and to some extent competing, sets of criteria flow from:
• Traditional scientific research criteria
• Social construction and constructivist criteria
• Artistic and evocative criteria
• Pragmatic and utilitarian criteria
• Critical change criteria
102 FEMINIST EVALUATION
Different perspectives about such things as truth and the nature of real-
ity constitute paradigms or worldviews based on alternative epistemologies
and ontologies. People viewing evaluations through different paradigmatic
lenses will react differently, just as we, as evaluators, vary in how we think
about what we do when we study and assess programs. After briefly expli-
cating the primary values and criteria of these five frameworks, we’ll look
at how each might judge feminist evaluation.
Traditional Scientific Research Criteria. Social science has traditionally
emphasized objectivity and independence, so evaluations judged within this
tradition will have to demonstrate procedures for minimizing investigator
bias. Traditional research criteria include attention to validity, reliability,
generalizability, strength of evidence supporting causal connections
between program interventions and measured outcomes, and consistency
with or contributions to social science theory with a marked preference for
quantitative measurement and experimental designs as the methodological
gold standard. The traditional scientific research criteria are the basis for
evaluation research as represented by Rossi, Freeman, and Lipsey (1999)
and Huey-Tsyh Chen and Peter H. Rossi (1987).
Social Construction and Constructivist Criteria. Constructivist criteria
contrast dramatically with and emerged in reaction to traditional research
criteria. Constructivist evaluators assert the inevitability of subjectivity,
commit to capturing and respecting multiple perspectives, favor qualita-
tive inquiry, and offer alternative criteria for judging methodological
quality—for example, trustworthiness, authenticity, reflexivity, praxis, par-
ticularity (doing justice to the integrity of unique cases), and the extent to
which the findings enhance and deepen understanding. Perhaps the most
prominent advocacy for constructivist evaluation is Fourth Generation
Evaluation (Guba and Lincoln, 1989). Sensitivity to and respect for multi-
ple stakeholder perspectives is a hallmark of constructivist evaluation
(Greene, 1998a, 1998b, 2000).
Artistic and Evocative Criteria. These criteria are not yet widely
employed in evaluation, although examples exist where role-playing and
dramatic techniques are used to gather and present data, or where poetry,
cartoons, and literary forms (short stories and narrative techniques) are
used to report findings. The performance art of The Vagina Monologues
(Ensler, 2001), based on interviews with women and presented as theater,
offers a prominent example of an artistic and evocative approach to com-
municating research findings. The audience feels as much as knows the
truth of the presentation because of the essence it reveals. The criteria for
judging artistic and evocative evaluations include creativity, aesthetic qual-
ity, and interpretive vitality. Evaluations communicated from this set of cri-
teria would aim to be stimulating, provocative, and connect with and move
intended audiences (clearly not your traditional research report). The find-
ings should open the program world to the evaluation audience by pro-
viding a deep sense of the lived experience of program participants using
IS IT EVALUATION? 103
literary and dramatic devices. The evaluator’s voice should be distinct, vis-
ible, and expressive. The findings should feel “true” or “authentic” or “real”;
that is, the evaluation acknowledges and taps into the feeling dimension of
human experience as much as the cognitive. These criteria have informed
“connoisseurship evaluation,” the label Eisner (1991, 1985) gave to his
artistically oriented approach.
Pragmatic and Utilitarian Criteria. The fourth set of criteria empha-
size addressing the specific, practical informative needs of specific in-
tended users of an evaluation. Criteria include responsiveness to concrete
intended users, situational adaptability, high-quality interactive engagement
between the evaluator and intended users, methodological flexibility, rele-
vance of the evaluation process and findings, and actual use of the findings
by intended users in intended ways. Pragmatic and utilitarian criteria under-
gird utilization-focused evaluation (Patton, 1997).
Critical Change Criteria. These final criteria apply to evaluators who see
themselves as change agents. In taking an activist stance as evaluators and
using evaluation to increase social justice, they eschew any pretense of tradi-
tional objectivity. Those working from a critical change orientation bring to
evaluation an explicit agenda of elucidating political, economic, and social
inequalities, and in so elucidating, they critique social inequities, raise con-
sciousness, and strive to change the balance of power in favor of those less
powerful, if by nothing else through increasing their capacity to represent
their own interests effectively through evaluation. Informed by the presump-
tion of the centrality of social injustice in understanding community and soci-
etal structures, and therefore program and policy interventions, an explicit
political praxis (connecting theory and action) and evaluator reflexivity are
expected to be part of the evaluation process and explicit in evaluation
reports. In short, critical change criteria include increasing consciousness
about injustices; identifying the nature and sources of inequalities and injus-
tices; representing the perspective of the less powerful; making visible the
ways in which those with more power exercise and benefit from power;
engaging those with less power respectfully and collaboratively; building the
capacity of those involved to take action; identifying potential change-making
strategies; praxis; and embedding the evaluation in a clear historical and val-
ues context. In that the critical change framework emphasizes using evalua-
tion to bring about social change and increase social justice, this category can
include collaborative and participatory approaches to evaluation that are used
explicitly and intentionally to build the capacity of those involved to better
understand their own situations, raise consciousness, and support future
action aimed at political change. Critical change criteria undergird empow-
erment evaluation (Fetterman, 2000), diversity-inclusive evaluation (Mertens,
1998), and those aspects of deliberative democratic evaluation that involve
values-based advocating for democracy (House and Howe, 2000).
Critiques of Feminist Evaluation Criteria. Throughout this volume,
principles of feminist evaluation have been elucidated. Different feminists
104 FEMINIST EVALUATION
and feminist evaluators emphasize somewhat different principles, so no uni-
versally agreed-on criteria can be applied in a monolithic way to all femi-
nist evaluation. That said, it seems to me that the following criteria are
commonly articulated in relation to feminist evaluation:
• Explicitly attending to gender inequities and, where possible and appro-
priate, understanding and connecting gender inequities to other forms of
social injustice (racism, social class inequities)
• Emphasizing the inevitably political nature of evaluation (knowledge is
power), therefore explicitly addressing power relationships and issues
throughout and in all aspects of the evaluation
• Engendering a sense of connectedness and equality between evaluator
and those in the program being evaluated (genuine power sharing)
• Using participatory processes that support consciousness-raising,
capacity-building, and evaluator reflexivity, and that make knowledge and
knowledge-creation resources available to those who participate in the
evaluation, especially disadvantaged women
• Explicitly acknowledging and valuing “women’s ways of knowing,”
including integrating reason, emotion, intuition, experience, and analytic
thought and supporting expression of women’s voices about their own
experiences
• Using evaluation processes and findings to bring about change, specifi-
cally increased social justice for women and other oppressed and disad-
vantaged peoples
Matching these principles against the preceding five sets of criteria for
judging evaluation quality, it’s not too hard to see where concerns about fem-
inist evaluation are located. Feminist evaluation is anathema to those adher-
ing to traditional research criteria, the same criteria that the privileged few
promulgated as the ideal in the early days of the profession. No surprise there.
Their objections derive not only from their criteria of objectivity, indepen-
dence, political neutrality, and generating truth, which feminist evaluation
eschews, but the vociferousness of their objections also derive from concern
that feminist evaluation will hurt the credibility of the entire evaluation pro-
fession. If the credibility of the evaluation profession hinges on its indepen-
dence and neutrality, and if feminist evaluators openly act as advocates and
activists, then the profession itself, for the sake of its credibility in the larger
world, must label feminist evaluation as something other than evaluation.
What about constructivist evaluation? Feminist evaluation, in honor-
ing and valuing multiple perspectives and eschewing singularity of truth,
has constructivist underpinnings, but many constructivist evaluators would
be uncomfortable with the explicit activism and advocacy of feminist eval-
uation. I would hypothesize that some sympathy toward constructivism is
a necessary but not sufficient condition for affirming the legitimacy of fem-
inist evaluation.
IS IT EVALUATION? 105
Artistic and evocative criteria strike me as welcoming feminist evalua-
tors to draw to a greater extent on the important literary traditions of fem-
inism and incorporate them into evaluation. The greatest influence of
feminist literature on feminist evaluation to date may be in the attention
given to voice. In addition to encouraging each woman to find her voice,
the critical and creative writing involved in feminist literature challenges
each woman (really each person) to own her voice and perspective.
Feminist theory has highlighted and deepened our understanding of the
intricate and implicate relationships between language, voice, and con-
sciousness (see Gilligan, 1982; Minnich, 1990). We are challenged by both
feminist and postmodern critiques of knowledge to be clear about and own
our authorship of whatever we propound, to be self-reflective, to acknowl-
edge biases and limitations, and to honor multiple perspectives. In an effort
to legitimize feminist evaluation by emphasizing its research and empirical
elements, usually within a constructivist framework, feminist evaluation
may have given too little attention to the literary forms of knowledge crea-
tion and reporting that might more fully enliven feminist evaluations.
Pragmatic and utilitarian criteria emphasize the importance of a match
between the values and perspectives of primary intended users of an evalua-
tion and the evaluator. Thus, by those criteria, feminist evaluation is a criti-
cally important option for programs operating according to feminist principles
and values, examples of which are presented in this volume. For programs
not explicitly feminist in orientation, but where gender differentiation is a
core issue (such as drug abuse treatment programs where patterns of use are
often heavily gender-differentiated, or early childhood education programs
where parenting roles are typically gender-based), feminist evaluation might
prove attractive as an option to primary intended users. Utilization-focused
evaluation advocates presenting primary intended users with a range of
options, facilitating their understanding of the strengths and weaknesses
of those options, and then helping them determine the best fit for those needs.
Feminist evaluation can be a viable and important option in utilization-
focused evaluations. None of the pragmatic and utilitarian criteria inherently
conflict with feminist evaluation principles as long as feminist evaluation isn’t
forced on unwilling primary intended users (which would typically violate
feminist principles of collaboration anyway). The more difficult situation is
where primary intended users are divided about the appropriateness and util-
ity of feminist evaluation. (Primary intended users are quite often divided
about one thing or another, which is why utilization-focused evaluation
emphasizes negotiation skills.) In those instances, feminist evaluation might
be combined with or conducted parallel to another evaluation approach.
Admittedly, for those evaluators who make feminist evaluation their primary
or even sole mode of engagement, or think of it as an all-encompassing
approach, such a compromise would be untenable. For those of us who are
of a more pragmatic bent, however, integrating different perspectives in an
evaluation opens up new possibilities for insight and is quite doable.
106 FEMINIST EVALUATION
Finally, the critical change criteria offer the most comfortable home for
feminist evaluation. The critical change criteria are not only entirely con-
sistent with feminist evaluation, but, indeed, feminist evaluations exemplify
those criteria. Feminist evaluation acknowledges the inevitably political and
moral nature of evaluative judgments and challenges evaluators to connect
voice and perspective to praxis—acting in the world with an appreciation
for and recognition of how those actions inherently express social, politi-
cal, and moral values (Schwandt, 1989, 2000). Feminist evaluation, in hon-
oring diverse perspectives and working collaboratively with program
participants, encourages each of us as evaluators to own our own perspec-
tive but also to take seriously the responsibility to authentically communi-
cate the perspectives of those we encounter during our inquiry.
Mixing Perspectives
The five criteria-based frameworks and feminist principles reviewed above
show the range of criteria that can be brought to bear in judging an evaluation.
They can also be viewed as alternative lenses for expanding the possibilities
available, not only for critiquing evaluations but also for undertaking them.
While each set of criteria manifests a certain coherence, it is possible to mix
and match approaches. Part of an evaluation might be highly participatory
while another part might apply quite traditional scientific criteria. Consider
an evaluation of a displaced-homemakers program where the formative eval-
uation with ongoing program participants followed feminist principles while
an independent study of program dropouts, conducted because of funder con-
cerns about dropouts, used traditional scientific research methods or a con-
structivist approach. As an evaluator, I have worked with and mixed criteria
from all five frameworks to create designs matched to the particular and mul-
tiple needs and interests of specific stakeholders, but any single evaluation has
tended to be dominated by one set of criteria with a second or third set as com-
plementary or conducted in parallel. This means, from my perspective, a fem-
inist evaluation does not have to stand alone, be all-encompassing, or be pure
but may be one part of a multiperspective/multi-element evaluation design
offering a form of evaluation triangulation. I also understand that this prag-
matic stance is disputed by those who focus on the need to implement a fem-
inist approach with holistic integrity.
Operating within any particular framework and using any specific set
of criteria will invite criticism from those who judge our work from a dif-
ferent framework and with different criteria. (For examples of the vehe-
mence of such criticisms between those using traditional social science
criteria and those using artistic narrative criteria, see Bochner, 2001.)
Understanding that criticisms (or praise) flow from criteria can help us
anticipate how to position our evaluations and make explicit what criteria
to apply to our own work as well as what criteria to offer others as they con-
sider evaluation options.
IS IT EVALUATION? 107
The Future
The future of feminist evaluation is linked to the future of gender relation-
ships and social justice. It seems unlikely that gender-based inequalities will
soon disappear. Award-winning and best-selling Canadian novelist Carol
Shields (The Stone Diaries) dealt with a diagnosis of terminal breast cancer
by writing one last novel, Unless, in which she determined to “address the
inequalities between men and women that I think are central to all our
problems in the world” (Russo, 2002, p. 35). While those inequalities
endure there will be feminists, both women and men, advocating for and
engaged in feminist evaluation.
References
Bochner, A. P. “Narrative’s Virtues.” Qualitative Inquiry, 2001, 7 (2), 131–157.
Chen, H., and Rossi, P. “The Theory-Driven Approach to Validity.” Evaluation and
Program Planning, 1987, 10, 95–103.
Eisner, E. The Art of Educational Evaluation. London: Falmer Press, 1985.
Eisner, E. The Enlightened Eye. New York: Macmillan, 1991.
Ensler, E. The Vagina Monologues: The V-Day Edition. New York: Villard, 2001.
Fetterman, D. M. Foundations of Empowerment Evaluation: Step by Step. Thousand Oaks,
Calif.: Sage, 2000.
Gilligan, C. In a Different Voice: Psychological Theory and Women’s Development.
Cambridge, Mass.: Harvard University Press, 1982.
Greene, J. C. “Balancing Philosophy and Practicality in Qualitative Evaluation.” In
R. Davis (ed.), Proceedings of the State Symposium on Educational Evaluation.
Champaign/Urbana: University of Illinois, 1998a.
Greene, J. C. “Qualitative Interpretive Interviewing.” In A. J. Reynolds and H. J. Walberg
(eds.), Educational Research for Educational Productivity. Vol. 7: Advances in
Educational Productivity. Greenwich, Conn.: JAI Press, 1998b.
Greene, J. C. “Understanding Social Programs Through Evaluation.” In N. K. Denzin
and Y. S. Lincoln (eds.), Handbook of Qualitative Research. (2nd. ed.) Thousand Oaks,
Calif.: Sage, 2000.
Guba, E., and Lincoln, Y. Fourth Generation Evaluation. Thousand Oaks, Calif.: Sage,
1989.
House, E. R., and Howe, K. R. “Deliberative Democratic Evaluation.” In K. E. Ryan, and
L. DeStafano (eds.), Evaluation as a Democratic Process: Promoting Inclusion, Dialogue,
and Deliberation. New Directions for Evaluation, no. 85. San Francisco: Jossey-Bass,
2000.
Mertens, D. M. Research Methods in Education and Psychology: Integrating Diversity with
Quantitative and Qualitative Approaches. Thousand Oaks, Calif.: Sage, 1998.
Minnich, E. Transforming Knowledge. Philadelphia: Temple University Press, 1990.
Patton, M. Q. Utilization-Focused Evaluation. (3rd ed.) Thousand Oaks, Calif.: Sage,
1997.
Patton, M. Q. Qualitative Research and Evaluation Methods. (3rd ed.) Thousand Oaks,
Calif.: Sage, 2002.
Rossi, P., Freeman, H., and Lipsey, M. Evaluation: A Systematic Approach. (6th ed.)
Thousand Oaks, Calif.: Sage, 1999.
Russo, M. “Final Chapter.” New York Times Magazine, April 14, 2002, pp. 32–35.
Schwandt, T. “Recapturing Moral Discourse in Evaluation.” Educational Researcher,
1989, 18 (8), 11–16, 34.
Schwandt, T. “Three Epistemological Stances for Qualitative Inquiry: Interpretivism,
Hermeneutics, and Social Constructivism.” In N. Denzin and Y. Lincoln (eds.),
Handbook of Qualitative Research. (2nd ed.) Thousand Oaks, Calif.: Sage, 2000.
MICHAEL QUINN PATTON is a faculty member at The Union Institute and
University.
108 FEMINIST EVALUATION

Más contenido relacionado

Similar a Patton 2002-new directions-for_evaluation

Class session 6 march 14, 2019
Class session 6 march 14, 2019Class session 6 march 14, 2019
Class session 6 march 14, 2019
tjcarter
 
Postpositivism and educational research final
Postpositivism and educational research finalPostpositivism and educational research final
Postpositivism and educational research final
clabbe2
 
Rorschach Measures Of Cognition And Social Functioning Essay
Rorschach Measures Of Cognition And Social Functioning EssayRorschach Measures Of Cognition And Social Functioning Essay
Rorschach Measures Of Cognition And Social Functioning Essay
Katherine Alexander
 
Sample exam questions bo resum-edocx
Sample exam questions bo resum-edocxSample exam questions bo resum-edocx
Sample exam questions bo resum-edocx
João Cabral
 

Similar a Patton 2002-new directions-for_evaluation (20)

Ch4
Ch4Ch4
Ch4
 
Presentation
PresentationPresentation
Presentation
 
VMI
VMI VMI
VMI
 
Chapter 5 presentation
Chapter 5 presentationChapter 5 presentation
Chapter 5 presentation
 
Personality and values chapter 5 ( organizational behavior)
Personality and values chapter 5 ( organizational behavior)Personality and values chapter 5 ( organizational behavior)
Personality and values chapter 5 ( organizational behavior)
 
Personality & values
Personality & valuesPersonality & values
Personality & values
 
Leadership principles
Leadership principlesLeadership principles
Leadership principles
 
Personality and Values
Personality and ValuesPersonality and Values
Personality and Values
 
VIA
VIAVIA
VIA
 
VIA
VIAVIA
VIA
 
VALUE EDUCATION_Unit_IV.pptx
VALUE EDUCATION_Unit_IV.pptxVALUE EDUCATION_Unit_IV.pptx
VALUE EDUCATION_Unit_IV.pptx
 
OB Ch.5
OB Ch.5OB Ch.5
OB Ch.5
 
Class session 6 march 14, 2019
Class session 6 march 14, 2019Class session 6 march 14, 2019
Class session 6 march 14, 2019
 
Ethics module1
Ethics module1Ethics module1
Ethics module1
 
Postpositivism and educational research final
Postpositivism and educational research finalPostpositivism and educational research final
Postpositivism and educational research final
 
MSPO individual behaviour, culture values, ethics,_MSPO.ppt
MSPO individual behaviour, culture values, ethics,_MSPO.pptMSPO individual behaviour, culture values, ethics,_MSPO.ppt
MSPO individual behaviour, culture values, ethics,_MSPO.ppt
 
Rorschach Measures Of Cognition And Social Functioning Essay
Rorschach Measures Of Cognition And Social Functioning EssayRorschach Measures Of Cognition And Social Functioning Essay
Rorschach Measures Of Cognition And Social Functioning Essay
 
Session 2 value generic
Session 2 value  genericSession 2 value  generic
Session 2 value generic
 
Sample exam questions bo resum-edocx
Sample exam questions bo resum-edocxSample exam questions bo resum-edocx
Sample exam questions bo resum-edocx
 
Organizational behaviour for personality
Organizational behaviour for personalityOrganizational behaviour for personality
Organizational behaviour for personality
 

Último

VIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 BookingVIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 Booking
dharasingh5698
 

Último (20)

World Press Freedom Day 2024; May 3rd - Poster
World Press Freedom Day 2024; May 3rd - PosterWorld Press Freedom Day 2024; May 3rd - Poster
World Press Freedom Day 2024; May 3rd - Poster
 
The Economic and Organised Crime Office (EOCO) has been advised by the Office...
The Economic and Organised Crime Office (EOCO) has been advised by the Office...The Economic and Organised Crime Office (EOCO) has been advised by the Office...
The Economic and Organised Crime Office (EOCO) has been advised by the Office...
 
(NEHA) Call Girls Nagpur Call Now 8250077686 Nagpur Escorts 24x7
(NEHA) Call Girls Nagpur Call Now 8250077686 Nagpur Escorts 24x7(NEHA) Call Girls Nagpur Call Now 8250077686 Nagpur Escorts 24x7
(NEHA) Call Girls Nagpur Call Now 8250077686 Nagpur Escorts 24x7
 
Junnar ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For S...
Junnar ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready For S...Junnar ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready For S...
Junnar ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For S...
 
Coastal Protection Measures in Hulhumale'
Coastal Protection Measures in Hulhumale'Coastal Protection Measures in Hulhumale'
Coastal Protection Measures in Hulhumale'
 
Election 2024 Presiding Duty Keypoints_01.pdf
Election 2024 Presiding Duty Keypoints_01.pdfElection 2024 Presiding Duty Keypoints_01.pdf
Election 2024 Presiding Duty Keypoints_01.pdf
 
VIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 BookingVIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Bhavnagar 7001035870 Whatsapp Number, 24/07 Booking
 
2024: The FAR, Federal Acquisition Regulations, Part 30
2024: The FAR, Federal Acquisition Regulations, Part 302024: The FAR, Federal Acquisition Regulations, Part 30
2024: The FAR, Federal Acquisition Regulations, Part 30
 
The U.S. Budget and Economic Outlook (Presentation)
The U.S. Budget and Economic Outlook (Presentation)The U.S. Budget and Economic Outlook (Presentation)
The U.S. Budget and Economic Outlook (Presentation)
 
Tuvalu Coastal Adaptation Project (TCAP)
Tuvalu Coastal Adaptation Project (TCAP)Tuvalu Coastal Adaptation Project (TCAP)
Tuvalu Coastal Adaptation Project (TCAP)
 
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance BookingCall Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
 
TEST BANK For Essentials of Negotiation, 7th Edition by Roy Lewicki, Bruce Ba...
TEST BANK For Essentials of Negotiation, 7th Edition by Roy Lewicki, Bruce Ba...TEST BANK For Essentials of Negotiation, 7th Edition by Roy Lewicki, Bruce Ba...
TEST BANK For Essentials of Negotiation, 7th Edition by Roy Lewicki, Bruce Ba...
 
Get Premium Budhwar Peth Call Girls (8005736733) 24x7 Rate 15999 with A/c Roo...
Get Premium Budhwar Peth Call Girls (8005736733) 24x7 Rate 15999 with A/c Roo...Get Premium Budhwar Peth Call Girls (8005736733) 24x7 Rate 15999 with A/c Roo...
Get Premium Budhwar Peth Call Girls (8005736733) 24x7 Rate 15999 with A/c Roo...
 
Call Girls Nanded City Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Nanded City Call Me 7737669865 Budget Friendly No Advance BookingCall Girls Nanded City Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Nanded City Call Me 7737669865 Budget Friendly No Advance Booking
 
WORLD DEVELOPMENT REPORT 2024 - Economic Growth in Middle-Income Countries.
WORLD DEVELOPMENT REPORT 2024 - Economic Growth in Middle-Income Countries.WORLD DEVELOPMENT REPORT 2024 - Economic Growth in Middle-Income Countries.
WORLD DEVELOPMENT REPORT 2024 - Economic Growth in Middle-Income Countries.
 
VIP Model Call Girls Shikrapur ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Shikrapur ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Shikrapur ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Shikrapur ( Pune ) Call ON 8005736733 Starting From 5K t...
 
PPT BIJNOR COUNTING Counting of Votes on ETPBs (FOR SERVICE ELECTORS
PPT BIJNOR COUNTING Counting of Votes on ETPBs (FOR SERVICE ELECTORSPPT BIJNOR COUNTING Counting of Votes on ETPBs (FOR SERVICE ELECTORS
PPT BIJNOR COUNTING Counting of Votes on ETPBs (FOR SERVICE ELECTORS
 
The NAP process & South-South peer learning
The NAP process & South-South peer learningThe NAP process & South-South peer learning
The NAP process & South-South peer learning
 
VIP Model Call Girls Narhe ( Pune ) Call ON 8005736733 Starting From 5K to 25...
VIP Model Call Girls Narhe ( Pune ) Call ON 8005736733 Starting From 5K to 25...VIP Model Call Girls Narhe ( Pune ) Call ON 8005736733 Starting From 5K to 25...
VIP Model Call Girls Narhe ( Pune ) Call ON 8005736733 Starting From 5K to 25...
 
2024: The FAR, Federal Acquisition Regulations, Part 31
2024: The FAR, Federal Acquisition Regulations, Part 312024: The FAR, Federal Acquisition Regulations, Part 31
2024: The FAR, Federal Acquisition Regulations, Part 31
 

Patton 2002-new directions-for_evaluation

  • 1. Feminist evaluation represents an important option in evaluation, but its legitimacy and credibility depend on the criteria used to define high-quality evaluation practice. This commentary examines how feminist evaluation principles and practices look from the perspectives of five different frameworks, each offering different criteria for judging evaluations. NEW DIRECTIONS FOR EVALUATION, no. 96, Winter 2002 © Wiley Periodicals, Inc. 97 8 Feminist, Yes, but Is It Evaluation? Michael Quinn Patton The root problem reappears in different guises in all fields and throughout the dominant tradition. It is, simply, that the majority of humankind was excluded from education and the making of what has been called knowl- edge (Minnich, 1990, p. 37). Philosopher Elizabeth Minnich has examined in depth in her book Transforming Knowledge the ways in which throughout history “the domi- nant few not only defined themselves as the inclusive kind of human but also as the norm and the ideal” (p. 37). Who are these dominant few? “A few privileged men defined themselves as constituting mankind/humankind and simultaneously saw themselves as akin to what mankind/humankind ought to be in fundamental ways that distinguish them from all others” (p. 37). How did they do this? “[T]hey created root definitions of what it means to be human that, with the concepts and theories that flowed from and rein- forced those definitions, made it difficult to think well about, or in the mode of, anyone other than themselves, just as they made it difficult to think hon- estly about the defining few” (p. 38). The male pronoun as inclusive and universal is but one obvious and prominent example. Minnich, writing as a feminist scholar, has examined how the concepts and conceptual frameworks we use, whether unconsciously as a matter of tradition and training or intentionally as a matter of choice, carry embed- ded messages about what and who is important. She asserts in the opening quotation above that “this root problem reappears in different guises in all fields.” Does it appear in evaluation? Let us substitute mankind/humankind in Minnich’s analysis with evaluator/evaluation and see if we recognize our own history as a field: The majority of evaluators were excluded from determining what has been called knowledge in evaluation. The dominant few not only defined themselves
  • 2. 98 FEMINIST EVALUATION as the inclusive kind of evaluator but also as the norm and the ideal. A few priv- ileged men defined themselves as constituting the field of evaluation and simul- taneously saw themselves as akin to what evaluators ought to be in fundamental ways that distinguish them from all others. They created root definitions of what it means to be an evaluator that, with the concepts and theories that flowed from and reinforced those definitions, made it difficult to think well about, or in the mode of, anyone other than themselves, just as they made it difficult to think honestly about themselves as the defining few. A Feminist Analysis of Evaluation: One Perspective Since I’m about to presume to offer one version of a feminist analysis of evaluation, let me start with a little reflexivity about my voice and perspec- tive, given that reflexivity and attention to voice are among the core contri- butions of feminist analysis. I write as a white male (or, as I was called in the Peace Corps by the Gourma people of eastern Burkina Faso, bompieno— a white thing). I’ve been around since the formal beginning of the profes- sion we call evaluation. Since 1975, I’ve attended every national conference of the American Evaluation Association and all predecessor conferences of the Evaluation Network. I’ve written a bit about evaluation, including some books, some of which have had some success—“success” being defined as being widely cited and broadly attacked, not necessarily in that order. As an evaluation textbook author, I’ve made it a point of my participation in the profession to observe and reflect on how the field has developed and is developing, and to discuss those developments with colleagues. That’s a bit of the background I bring to the question of whether Minnich’s analysis of what she calls “the root problem” applies to evaluation. It strikes me as accurate to say that in the early, formative days of the profession, a dominant few defined evaluation and defined themselves as the norm and the ideal, and those dominant few were primarily, though not exclusively, privileged men. They defined themselves as constituting the field of evaluation and simultaneously saw themselves as akin to what eval- uators ought to be. They created root definitions of what it means to be an evaluator that, with the concepts and theories that flowed from and rein- forced those definitions, made it difficult to think well about, or in the mode of, anyone other than themselves. Takes me right back to the seventies and early eighties. I say “they” rather than “we” because in those days, when utilization- focused evaluation was an outlier idea and qualitative methods were regu- larly disdained, the ideas I offered about evaluation were variously criticized as lacking in rigor, selling out, immoral, soft, naive, romantic, idealistic, and undermining the foundations of the emergent profession. I responded in kind, the barbs flew back and forth, debates raged, and the adrenaline flowed. “Hard” versus “soft” data. Accused of “getting in bed” with those I was supposed to be evaluating. Ridiculed for not being “tough enough” to
  • 3. IS IT EVALUATION? 99 tell the truth. Too relationship-oriented. Too formative. (A little formative is okay, but only as foreplay for the real judgmental stuff: hard-hitting, tell- it-like-it-is, sum-it-all-up, hard-on, summative evaluation.) Too kind. Too gentle. I had no idea that aspiring to be an evaluator could cast such doubts on a young man’s masculinity. But I digress. I was recalling the litany of concerns I’ve encountered over the years about various aspects of utilization-focused evaluation, espe- cially those aspects that involve working closely with intended users to build capacity for evaluative thinking, facilitate their making their own judgments instead of simply genuflecting to mine, and helping them apply evaluation findings to do their own reality-testing and bring about change in accordance with their own values. The most common, damning, dismis- sive, and persistent objection I’ve heard is that such activities are all well and good if I choose to do them, but they aren’t evaluation. What does this ancient history have to do with feminist evaluation? Just this: The most common, damning, dismissive, and persistent objection I hear to feminist evaluation is that it’s not really evaluation. It’s fine for those so moved and motivated to engage in a feminist agenda of social change. Just don’t call that engagement evaluation. It’s fine to work on raising con- sciousness about women’s oppression and strive for women’s liberation, just don’t do it under the guise of evaluation. Keep evaluation pure, by which is meant rendering judgments of merit and worth from a dispassionate posi- tion of independence and objectivity. Defining Evaluation It turns out that the various definitions of evaluation don’t just tell you what you must do; they can be interpreted, and often are, as saying that’s all you can do. If evaluation is defined as judgment of merit and worth, do that and do only that. Everything else must be something other than evaluation. If evaluation is defined as the application of social science methods to deter- mine program effectiveness, do that and only that. Anything else is of lesser value or even illegitimate. If evaluation is defined as determining the extent to which a program achieves its goals, that becomes the priority task and all else is at best secondary, if even worth the bother. The power to define is the power to control, to include and exclude. That is what Minnich was referring to as the root problem that reappears in different guises in all fields. This root problem is the power of the dominant few to define what constitutes legitimate activity and real knowledge, and these dominant few not only define themselves as the inclusive kind of whatever is being defined, in this case evaluator, but also as the norm and the ideal. They create root definitions with concepts and theories that flow from and reinforce those definitions, making it difficult to think well about, or in the mode of, anyone other than themselves, just as they make it diffi- cult to think honestly about the defining few.
  • 4. 100 FEMINIST EVALUATION Definitions matter. Regularly and predictably on EvalTalk (the Ameri- can Evaluation Association–sponsored listserv) questions emerge and dis- cussions follow about whether this or that is evaluation. Is evaluation the same as or different from research? monitoring? performance measurement? accountability? auditing? Feminist Evaluation Redefines Evaluation. Feminist evaluation can include judging merit or worth, using social science methods to determine effectiveness, and measuring goal attainment, but those activities are means to a greater end—increased social justice for the oppressed, especially, but not exclusively, women. And feminist evaluation offers different ways of judging merit or worth, applying social science methods, or measuring goal attainment—collaboratively, inclusively, embracing multiple perspectives, and using evaluation processes and findings to foment change. And therein lies the rub. If evaluation becomes a means rather than an end and is con- ducted in ways that intentionally and actively bring about social justice, is it still evaluation? And who gets to decide? An Inclusive and Liberal Profession. Who gets to decide? For the moment and for the purposes of this volume, you, the reader, do. While a privileged few determined the early definitions, standards, and norms for evaluation, drawing primarily from traditional social science research meth- ods and auditing standards, the profession has been too amorphous and unwieldy to remain so constrained. The 1980s pretty much put an end to the quantitative-qualitative debate (both hardness and softness are now con- sidered appropriate, even in mixed company—that is, among both evalua- tors and nonevaluators). Methodological eclecticism, triangulated methods, multiple methods, and mixed methods are all in vogue, and woe is s/he who knows but one design or measurement approach. The 1990s blossomed with diversity—participatory evaluation, collaborative evaluation, empow- erment evaluation, inclusive evaluation, developmental evaluation, multi- vocal evaluation, learning-oriented evaluation, democratic evaluation, and lately, appreciative inquiry evaluation, to name but a few more prominent examples. These built on the emergent diversity of the late 1980s, which had seen the flowering of such approaches as fourth-generation evaluation, responsive evaluation, constructivist evaluation, and even that 1970s hanger-on, utilization-focused evaluation. In the context of this contempo- rary, new millennium of evaluation diversity, feminist evaluation adds another important option and a distinctly different way of conducting eval- uations. Or does it? The nagging question planted by the original dominant few remains: is it really evaluation? And will feminist evaluation be treated as a legitimate and respected entry in the next edition of the unofficial eval- uation thesaurus? Certainly feminist evaluation has gained visibility, as witness this vol- ume. Of course, this volume appears at a time when political pundits in the larger society are questioning the contemporary relevance of feminism
  • 5. IS IT EVALUATION? 101 (defined often by those self-same pundits as nothing more than a throwback to their limited notion of the supposedly bra-burning sixties—and what could be more irrelevant to anything at all than the overhyped, under- achieved, oh-so-tiresome, and not incidentally, pre-evaluation profession sixties). One is unlikely to hear overt criticisms of feminism among evalu- ators, although some may have their private doubts. Evaluators are, on the whole, a politically liberal lot. No, the objections to feminist evaluation do not center on the feminist part of the terminological equation, at least not explicitly. The objections flow from concern with the oxymoronic nature of the combination of the terms “feminist” and “evaluation.” Feminist? Okay. Fine. Important. Worthwhile. Raise consciousness. Liberate. You go, girl. But is it evaluation? Feminist Evaluation Criteria Of course, questioning whether feminist evaluation is really evaluation may simply be an underhanded (or more politically correct) way of resisting the feminist critique and agenda. But for the purpose of these reflections, let’s take the evaluation question at face value. The question of what constitutes evaluation requires criteria, as does the even more devilish question of what constitutes “good” or high-quality evaluation. Credibility flows from those judgments. Quality and credibility are connected in that judgments of qual- ity constitute the foundation for perceptions of credibility. Feminist evalu- ation challenges traditional criteria for assessing evaluations, such as objectivity, evaluator independence, neutrality, and minimal investigator bias. Particular philosophical underpinnings or theoretical orientations and special purposes for evaluation can generate different criteria for judging quality and credibility, including whether an inquiry approach qualifies as evaluation. Alternative Criteria for Judging Evaluation Quality. I have identi- fied five contrasting sets of criteria for judging the quality of qualitative inquiry from different perspectives and within different philosophical frame- works (Patton, 2002, pp. 542–552). For the purpose of this discussion, I want to adapt and apply those criteria to examine the credibility of feminist evaluation from different evaluation points of view. For heuristic purposes, I’ll emphasize the differences rather than commonalities among these alter- native ways of conducting and judging evaluations. The five contrasting, and to some extent competing, sets of criteria flow from: • Traditional scientific research criteria • Social construction and constructivist criteria • Artistic and evocative criteria • Pragmatic and utilitarian criteria • Critical change criteria
  • 6. 102 FEMINIST EVALUATION Different perspectives about such things as truth and the nature of real- ity constitute paradigms or worldviews based on alternative epistemologies and ontologies. People viewing evaluations through different paradigmatic lenses will react differently, just as we, as evaluators, vary in how we think about what we do when we study and assess programs. After briefly expli- cating the primary values and criteria of these five frameworks, we’ll look at how each might judge feminist evaluation. Traditional Scientific Research Criteria. Social science has traditionally emphasized objectivity and independence, so evaluations judged within this tradition will have to demonstrate procedures for minimizing investigator bias. Traditional research criteria include attention to validity, reliability, generalizability, strength of evidence supporting causal connections between program interventions and measured outcomes, and consistency with or contributions to social science theory with a marked preference for quantitative measurement and experimental designs as the methodological gold standard. The traditional scientific research criteria are the basis for evaluation research as represented by Rossi, Freeman, and Lipsey (1999) and Huey-Tsyh Chen and Peter H. Rossi (1987). Social Construction and Constructivist Criteria. Constructivist criteria contrast dramatically with and emerged in reaction to traditional research criteria. Constructivist evaluators assert the inevitability of subjectivity, commit to capturing and respecting multiple perspectives, favor qualita- tive inquiry, and offer alternative criteria for judging methodological quality—for example, trustworthiness, authenticity, reflexivity, praxis, par- ticularity (doing justice to the integrity of unique cases), and the extent to which the findings enhance and deepen understanding. Perhaps the most prominent advocacy for constructivist evaluation is Fourth Generation Evaluation (Guba and Lincoln, 1989). Sensitivity to and respect for multi- ple stakeholder perspectives is a hallmark of constructivist evaluation (Greene, 1998a, 1998b, 2000). Artistic and Evocative Criteria. These criteria are not yet widely employed in evaluation, although examples exist where role-playing and dramatic techniques are used to gather and present data, or where poetry, cartoons, and literary forms (short stories and narrative techniques) are used to report findings. The performance art of The Vagina Monologues (Ensler, 2001), based on interviews with women and presented as theater, offers a prominent example of an artistic and evocative approach to com- municating research findings. The audience feels as much as knows the truth of the presentation because of the essence it reveals. The criteria for judging artistic and evocative evaluations include creativity, aesthetic qual- ity, and interpretive vitality. Evaluations communicated from this set of cri- teria would aim to be stimulating, provocative, and connect with and move intended audiences (clearly not your traditional research report). The find- ings should open the program world to the evaluation audience by pro- viding a deep sense of the lived experience of program participants using
  • 7. IS IT EVALUATION? 103 literary and dramatic devices. The evaluator’s voice should be distinct, vis- ible, and expressive. The findings should feel “true” or “authentic” or “real”; that is, the evaluation acknowledges and taps into the feeling dimension of human experience as much as the cognitive. These criteria have informed “connoisseurship evaluation,” the label Eisner (1991, 1985) gave to his artistically oriented approach. Pragmatic and Utilitarian Criteria. The fourth set of criteria empha- size addressing the specific, practical informative needs of specific in- tended users of an evaluation. Criteria include responsiveness to concrete intended users, situational adaptability, high-quality interactive engagement between the evaluator and intended users, methodological flexibility, rele- vance of the evaluation process and findings, and actual use of the findings by intended users in intended ways. Pragmatic and utilitarian criteria under- gird utilization-focused evaluation (Patton, 1997). Critical Change Criteria. These final criteria apply to evaluators who see themselves as change agents. In taking an activist stance as evaluators and using evaluation to increase social justice, they eschew any pretense of tradi- tional objectivity. Those working from a critical change orientation bring to evaluation an explicit agenda of elucidating political, economic, and social inequalities, and in so elucidating, they critique social inequities, raise con- sciousness, and strive to change the balance of power in favor of those less powerful, if by nothing else through increasing their capacity to represent their own interests effectively through evaluation. Informed by the presump- tion of the centrality of social injustice in understanding community and soci- etal structures, and therefore program and policy interventions, an explicit political praxis (connecting theory and action) and evaluator reflexivity are expected to be part of the evaluation process and explicit in evaluation reports. In short, critical change criteria include increasing consciousness about injustices; identifying the nature and sources of inequalities and injus- tices; representing the perspective of the less powerful; making visible the ways in which those with more power exercise and benefit from power; engaging those with less power respectfully and collaboratively; building the capacity of those involved to take action; identifying potential change-making strategies; praxis; and embedding the evaluation in a clear historical and val- ues context. In that the critical change framework emphasizes using evalua- tion to bring about social change and increase social justice, this category can include collaborative and participatory approaches to evaluation that are used explicitly and intentionally to build the capacity of those involved to better understand their own situations, raise consciousness, and support future action aimed at political change. Critical change criteria undergird empow- erment evaluation (Fetterman, 2000), diversity-inclusive evaluation (Mertens, 1998), and those aspects of deliberative democratic evaluation that involve values-based advocating for democracy (House and Howe, 2000). Critiques of Feminist Evaluation Criteria. Throughout this volume, principles of feminist evaluation have been elucidated. Different feminists
  • 8. 104 FEMINIST EVALUATION and feminist evaluators emphasize somewhat different principles, so no uni- versally agreed-on criteria can be applied in a monolithic way to all femi- nist evaluation. That said, it seems to me that the following criteria are commonly articulated in relation to feminist evaluation: • Explicitly attending to gender inequities and, where possible and appro- priate, understanding and connecting gender inequities to other forms of social injustice (racism, social class inequities) • Emphasizing the inevitably political nature of evaluation (knowledge is power), therefore explicitly addressing power relationships and issues throughout and in all aspects of the evaluation • Engendering a sense of connectedness and equality between evaluator and those in the program being evaluated (genuine power sharing) • Using participatory processes that support consciousness-raising, capacity-building, and evaluator reflexivity, and that make knowledge and knowledge-creation resources available to those who participate in the evaluation, especially disadvantaged women • Explicitly acknowledging and valuing “women’s ways of knowing,” including integrating reason, emotion, intuition, experience, and analytic thought and supporting expression of women’s voices about their own experiences • Using evaluation processes and findings to bring about change, specifi- cally increased social justice for women and other oppressed and disad- vantaged peoples Matching these principles against the preceding five sets of criteria for judging evaluation quality, it’s not too hard to see where concerns about fem- inist evaluation are located. Feminist evaluation is anathema to those adher- ing to traditional research criteria, the same criteria that the privileged few promulgated as the ideal in the early days of the profession. No surprise there. Their objections derive not only from their criteria of objectivity, indepen- dence, political neutrality, and generating truth, which feminist evaluation eschews, but the vociferousness of their objections also derive from concern that feminist evaluation will hurt the credibility of the entire evaluation pro- fession. If the credibility of the evaluation profession hinges on its indepen- dence and neutrality, and if feminist evaluators openly act as advocates and activists, then the profession itself, for the sake of its credibility in the larger world, must label feminist evaluation as something other than evaluation. What about constructivist evaluation? Feminist evaluation, in honor- ing and valuing multiple perspectives and eschewing singularity of truth, has constructivist underpinnings, but many constructivist evaluators would be uncomfortable with the explicit activism and advocacy of feminist eval- uation. I would hypothesize that some sympathy toward constructivism is a necessary but not sufficient condition for affirming the legitimacy of fem- inist evaluation.
  • 9. IS IT EVALUATION? 105 Artistic and evocative criteria strike me as welcoming feminist evalua- tors to draw to a greater extent on the important literary traditions of fem- inism and incorporate them into evaluation. The greatest influence of feminist literature on feminist evaluation to date may be in the attention given to voice. In addition to encouraging each woman to find her voice, the critical and creative writing involved in feminist literature challenges each woman (really each person) to own her voice and perspective. Feminist theory has highlighted and deepened our understanding of the intricate and implicate relationships between language, voice, and con- sciousness (see Gilligan, 1982; Minnich, 1990). We are challenged by both feminist and postmodern critiques of knowledge to be clear about and own our authorship of whatever we propound, to be self-reflective, to acknowl- edge biases and limitations, and to honor multiple perspectives. In an effort to legitimize feminist evaluation by emphasizing its research and empirical elements, usually within a constructivist framework, feminist evaluation may have given too little attention to the literary forms of knowledge crea- tion and reporting that might more fully enliven feminist evaluations. Pragmatic and utilitarian criteria emphasize the importance of a match between the values and perspectives of primary intended users of an evalua- tion and the evaluator. Thus, by those criteria, feminist evaluation is a criti- cally important option for programs operating according to feminist principles and values, examples of which are presented in this volume. For programs not explicitly feminist in orientation, but where gender differentiation is a core issue (such as drug abuse treatment programs where patterns of use are often heavily gender-differentiated, or early childhood education programs where parenting roles are typically gender-based), feminist evaluation might prove attractive as an option to primary intended users. Utilization-focused evaluation advocates presenting primary intended users with a range of options, facilitating their understanding of the strengths and weaknesses of those options, and then helping them determine the best fit for those needs. Feminist evaluation can be a viable and important option in utilization- focused evaluations. None of the pragmatic and utilitarian criteria inherently conflict with feminist evaluation principles as long as feminist evaluation isn’t forced on unwilling primary intended users (which would typically violate feminist principles of collaboration anyway). The more difficult situation is where primary intended users are divided about the appropriateness and util- ity of feminist evaluation. (Primary intended users are quite often divided about one thing or another, which is why utilization-focused evaluation emphasizes negotiation skills.) In those instances, feminist evaluation might be combined with or conducted parallel to another evaluation approach. Admittedly, for those evaluators who make feminist evaluation their primary or even sole mode of engagement, or think of it as an all-encompassing approach, such a compromise would be untenable. For those of us who are of a more pragmatic bent, however, integrating different perspectives in an evaluation opens up new possibilities for insight and is quite doable.
  • 10. 106 FEMINIST EVALUATION Finally, the critical change criteria offer the most comfortable home for feminist evaluation. The critical change criteria are not only entirely con- sistent with feminist evaluation, but, indeed, feminist evaluations exemplify those criteria. Feminist evaluation acknowledges the inevitably political and moral nature of evaluative judgments and challenges evaluators to connect voice and perspective to praxis—acting in the world with an appreciation for and recognition of how those actions inherently express social, politi- cal, and moral values (Schwandt, 1989, 2000). Feminist evaluation, in hon- oring diverse perspectives and working collaboratively with program participants, encourages each of us as evaluators to own our own perspec- tive but also to take seriously the responsibility to authentically communi- cate the perspectives of those we encounter during our inquiry. Mixing Perspectives The five criteria-based frameworks and feminist principles reviewed above show the range of criteria that can be brought to bear in judging an evaluation. They can also be viewed as alternative lenses for expanding the possibilities available, not only for critiquing evaluations but also for undertaking them. While each set of criteria manifests a certain coherence, it is possible to mix and match approaches. Part of an evaluation might be highly participatory while another part might apply quite traditional scientific criteria. Consider an evaluation of a displaced-homemakers program where the formative eval- uation with ongoing program participants followed feminist principles while an independent study of program dropouts, conducted because of funder con- cerns about dropouts, used traditional scientific research methods or a con- structivist approach. As an evaluator, I have worked with and mixed criteria from all five frameworks to create designs matched to the particular and mul- tiple needs and interests of specific stakeholders, but any single evaluation has tended to be dominated by one set of criteria with a second or third set as com- plementary or conducted in parallel. This means, from my perspective, a fem- inist evaluation does not have to stand alone, be all-encompassing, or be pure but may be one part of a multiperspective/multi-element evaluation design offering a form of evaluation triangulation. I also understand that this prag- matic stance is disputed by those who focus on the need to implement a fem- inist approach with holistic integrity. Operating within any particular framework and using any specific set of criteria will invite criticism from those who judge our work from a dif- ferent framework and with different criteria. (For examples of the vehe- mence of such criticisms between those using traditional social science criteria and those using artistic narrative criteria, see Bochner, 2001.) Understanding that criticisms (or praise) flow from criteria can help us anticipate how to position our evaluations and make explicit what criteria to apply to our own work as well as what criteria to offer others as they con- sider evaluation options.
  • 11. IS IT EVALUATION? 107 The Future The future of feminist evaluation is linked to the future of gender relation- ships and social justice. It seems unlikely that gender-based inequalities will soon disappear. Award-winning and best-selling Canadian novelist Carol Shields (The Stone Diaries) dealt with a diagnosis of terminal breast cancer by writing one last novel, Unless, in which she determined to “address the inequalities between men and women that I think are central to all our problems in the world” (Russo, 2002, p. 35). While those inequalities endure there will be feminists, both women and men, advocating for and engaged in feminist evaluation. References Bochner, A. P. “Narrative’s Virtues.” Qualitative Inquiry, 2001, 7 (2), 131–157. Chen, H., and Rossi, P. “The Theory-Driven Approach to Validity.” Evaluation and Program Planning, 1987, 10, 95–103. Eisner, E. The Art of Educational Evaluation. London: Falmer Press, 1985. Eisner, E. The Enlightened Eye. New York: Macmillan, 1991. Ensler, E. The Vagina Monologues: The V-Day Edition. New York: Villard, 2001. Fetterman, D. M. Foundations of Empowerment Evaluation: Step by Step. Thousand Oaks, Calif.: Sage, 2000. Gilligan, C. In a Different Voice: Psychological Theory and Women’s Development. Cambridge, Mass.: Harvard University Press, 1982. Greene, J. C. “Balancing Philosophy and Practicality in Qualitative Evaluation.” In R. Davis (ed.), Proceedings of the State Symposium on Educational Evaluation. Champaign/Urbana: University of Illinois, 1998a. Greene, J. C. “Qualitative Interpretive Interviewing.” In A. J. Reynolds and H. J. Walberg (eds.), Educational Research for Educational Productivity. Vol. 7: Advances in Educational Productivity. Greenwich, Conn.: JAI Press, 1998b. Greene, J. C. “Understanding Social Programs Through Evaluation.” In N. K. Denzin and Y. S. Lincoln (eds.), Handbook of Qualitative Research. (2nd. ed.) Thousand Oaks, Calif.: Sage, 2000. Guba, E., and Lincoln, Y. Fourth Generation Evaluation. Thousand Oaks, Calif.: Sage, 1989. House, E. R., and Howe, K. R. “Deliberative Democratic Evaluation.” In K. E. Ryan, and L. DeStafano (eds.), Evaluation as a Democratic Process: Promoting Inclusion, Dialogue, and Deliberation. New Directions for Evaluation, no. 85. San Francisco: Jossey-Bass, 2000. Mertens, D. M. Research Methods in Education and Psychology: Integrating Diversity with Quantitative and Qualitative Approaches. Thousand Oaks, Calif.: Sage, 1998. Minnich, E. Transforming Knowledge. Philadelphia: Temple University Press, 1990. Patton, M. Q. Utilization-Focused Evaluation. (3rd ed.) Thousand Oaks, Calif.: Sage, 1997. Patton, M. Q. Qualitative Research and Evaluation Methods. (3rd ed.) Thousand Oaks, Calif.: Sage, 2002. Rossi, P., Freeman, H., and Lipsey, M. Evaluation: A Systematic Approach. (6th ed.) Thousand Oaks, Calif.: Sage, 1999. Russo, M. “Final Chapter.” New York Times Magazine, April 14, 2002, pp. 32–35. Schwandt, T. “Recapturing Moral Discourse in Evaluation.” Educational Researcher, 1989, 18 (8), 11–16, 34.
  • 12. Schwandt, T. “Three Epistemological Stances for Qualitative Inquiry: Interpretivism, Hermeneutics, and Social Constructivism.” In N. Denzin and Y. Lincoln (eds.), Handbook of Qualitative Research. (2nd ed.) Thousand Oaks, Calif.: Sage, 2000. MICHAEL QUINN PATTON is a faculty member at The Union Institute and University. 108 FEMINIST EVALUATION