ReeseTolandSloneNorsworthy2010

Psychotherapy Theory, Research, Practice, Training © 2010 American Psychological Association
2010, Vol. 47, No. 4, 616 – 630 0033-3204/10/$12.00 DOI: 10.1037/a0021182

EFFECT OF CLIENT FEEDBACK ON COUPLE
PSYCHOTHERAPY OUTCOMES

ROBERT J. REESE, LARRY A. NORSWORTHY
MICHAEL D. TOLAND, AND Abilene Christian University
NORAH C. SLONE
University of Kentucky
Using outcome data to monitor the Keywords: couple therapy, patient-
progress of treatment and the therapeu- focused research, feedback, psychother-
tic alliance, also known as “client feed- apeutic outcomes
back” or “patient-focused research,”
has yielded impressive results in indi- Continuous assessment, or client feedback, is a
vidual psychotherapy. Client feedback method of tracking client progress across psycho-
has demonstrated reductions in prema- therapy that allows for clinicians to monitor
ture terminations and improved psycho- whether progress is being made in treatment. If
clients are not making progress as expected, ther-
therapy outcomes. However, little re- apists have the opportunity to modify or adjust
search has been conducted using this treatment as necessary. The American Psycholog-
paradigm with couples receiving ther- ical Association (APA) Division 29 Task Force for
apy. The purpose of this study was to Empirically Supported Relationships has made sev-
investigate whether the effectiveness of eral recommendations to help clients achieve posi-
tive outcomes in psychotherapy. In a conclusion
client feedback would extend to couple based on these recommendations, Ackerman et al.
therapy. Results from a randomized (2001) stated, “Practitioners are encouraged to rou-
couple clinical trial conducted in a nat- tinely monitor patients’ responses to the therapy
uralistic setting indicated that couples relationship and ongoing treatment. Such monitor-
in a client feedback condition demon- ing leads to increasing opportunities to repair alli-
ance ruptures, to improve the relationship, to mod-
strated statistically significantly more ify technical strategies, and to avoid premature
improvement compared with couples termination” (p. 496).
receiving treatment as usual and that A growing body of research investigating the
improvement occurred more rapidly. effects of client feedback in psychotherapy has
yielded encouraging results (e.g., Lambert et al.,
Also, 4 times as many couples in the 2001, 2002; Reese, Norsworthy, & Rowlands,
feedback condition reported clinically 2009; Whipple et al., 2003). A meta-analysis of
significant change by the end of treat- three previous client feedback studies conducted by
ment. Lambert and colleagues (2003) found an overall
effect size (Cohen’s d) of 0.39 for clients in a
feedback condition identified as deteriorating (de-
clined in treatment by about half of a standard
deviation) compared with clients deteriorating in a
Robert J. Reese, Michael D. Toland, and Norah C. Slone, no-feedback condition. Client feedback has consis-
Department of Educational, School, & Counseling Psychol-
tently been found to benefit clients identified as at
ogy, University of Kentucky; and Larry A. Norsworthy, De-
partment of Psychology, Abilene Christian University.
risk for terminating prematurely (i.e., clients who do
Correspondence regarding this article should be addressed not improve or deteriorate early in treatment).
to Robert J. Reese, PhD, Department of Educational, School, The research, however, has found conflicting
& Counseling Psychology, University of Kentucky, Lexing- results on whether client feedback works for all
ton, KY 40506-0017. E-mail: jeff.reese@uky.edu clients, not just those who are not progressing as

616

Client Feedback in Couple Therapy

expected (Harmon et al., 2007; Lambert, Har- comes, therapists can more readily intervene and
mon, Slade, Whipple, & Hawkins, 2005; Reese et attend to disconnects in the therapeutic relation-
al., 2009). Conclusions are difficult because stud- ship. Harmon et al. (2007) and Whipple et al.
ies that have found feedback to benefit all clients (2003) have found that adding clinical support
(Harmon et al., 2007; Hawkins, Lambert, Ver- tools, including a measure of the therapeutic al-
meersch, Slade, & Tuttle, 2004; Reese et al., liance, yielded incremental effectiveness for at-
2009) have implemented client feedback differ- risk clients when compared with just tracking
ently from studies that have found improvement outcome.
only for clients identified as at risk for terminat- The research on client feedback is impressive
ing prematurely (Lambert et al., 2002; Whipple et but has focused almost exclusively on individual
al., 2003). For example, Whipple et al. (2003) therapy. Couples and individuals experience sim-
found that only clients not on track benefited ilar barriers to positive psychotherapy treatment
from feedback; however, the feedback data were outcome, including deterioration, premature ter-
shared only with the therapist. In contrast, mination, and ruptured therapeutic alliances
Hawkins et al. (2004) found that providing feed- (Snyder, Castellani, & Whisman, 2006). Meta-
back was beneficial for all clients when it was analytic studies reported an overall effect size
provided to both the therapist and client. In turn, (Cohen’s d) for couple therapy ranging from 0.61
Harmon et al. (2007) showed that feedback ben- (Shadish et al., 1993) to 0.84 (Shadish & Bald-
efited all clients, but providing feedback to both win, 2003). According to recent findings (Snyder
therapist and client did not lead to increased et al., 2006), couples that receive therapy are
effectiveness. More recent research (Reese et al., approximately 80% better off than couples that
2009) has found that client feedback was benefi- do not receive treatment, which is comparable to
cial for all clients when compared with a treat- effect sizes seen in the individual psychotherapy
ment as usual (TAU) condition. In this study, literature (Lambert & Ogles, 2004; Wampold,
feedback was provided for both therapist and 2001). Shadish et al. (1993) compared the effect
client and included a measure to monitor the sizes for couple therapy studies with those for
therapeutic alliance every session. Research is individual therapy studies and found a nonsignif-
needed to further address the processes by which icant difference between the effect sizes (d
feedback is most effective, but the continuous 0.05, SE 0.12, n 6). The outcome literature
assessment literature has consistently established provides substantial evidence that both individual
that feedback is beneficial for improving psycho- and couple psychotherapy are effective forms of
therapy outcomes, especially for clients at risk for treatment.
dropping out of treatment. The psychotherapy outcome literature for cou-
The rationale for continuous assessment is ples, like the individual literature, has also dem-
based in part on research that has demonstrated onstrated that several approaches are effective.
that, in the aggregate, clients who benefit from For example, Snyder et al. (2006) reported that
therapy demonstrate improvement sooner rather both emotion-focused couple therapy and behav-
than later in treatment (e.g., Howard, Kopta, ioral couple therapy have yielded impressive re-
Krause, & Orlinsky, 1986; Lutz, Martinovich, & sults in multiple clinical trials. Shadish and Bald-
Howard, 1999). Monitoring outcome early in win’s (2005) meta-analytic findings suggest that
treatment increases the likelihood of identifying couples in treatment with behavioral couple ther-
clients who are not progressing as expected. An- apy were 72% better off than couples in a control
other predictor of effective psychotherapy is hav- condition. Gollan and Jacobson (2002) demon-
ing a strong therapeutic alliance (Horvath & strated effective couple therapy using emotion-
Symonds, 1991). As was stated earlier, the APA focused couple therapy, demonstrating recovery
Division 29 Task Force recommends the ongoing rates of 70 –73% with a weighted mean effect
monitoring of both outcome and the therapeutic size of 1.31 in comparison to a waitlist control.
alliance. Monitoring the therapeutic alliance has Although these therapies have evidenced greater
been found to be a statistically significant predic- effectiveness when compared with no-treatment
tor of positive outcomes (Harmon et al., 2007). conditions, the literature suggests that when these
Therapists can quickly and directly respond to approaches are directly compared with one an-
problems with the alliance when alerted (Lambert other, evidence has yet to demonstrate one ap-
et al., 2002). As with monitoring treatment out- proach being superior to another (Shadish &

617

Reese, Toland, Slone, and Norsworthy

Baldwin, 2003; Shadish et al., 1993; Snyder et back condition also reported higher levels of mar-
al., 2006). ital satisfaction at posttreatment, and a greater
One factor that makes studying couple therapy percentage of marriages were intact at follow-up
outcome difficult is that therapy is both an indi- when compared with marriages in the TAU con-
vidual and shared experience for each partner. dition. These findings for PCOMS are consistent
Outcome is affected by the influence of each with previous studies that focused on individual
partner’s readiness to change and level of distress therapy (Miller, Duncan, Brown, Sorrell, &
(Isakson et al., 2006; Tambling & Johnson, Chalk, 2006; Reese et al., 2009).
2008). An individual within the partnership may Replication of the Anker et al. (2009) findings
have significantly different views of the partner- is necessary for further evidence that continuous
ship itself, the therapy experience, and the ther- assessment, and specifically PCOMS, works with
apist. A continuous feedback system used in cou- couples. The purpose of our study was to repli-
ple therapy may allow both researchers and cate the results of Anker et al. with a sample from
clinicians to better understand how individuals the United States. The current study focused on
respond in couple psychotherapy at both the in- the effectiveness of using PCOMS with couples
dividual and couple level. in psychotherapy as compared with a TAU con-
Only one study was identified that has used trol condition. We had three major hypotheses.
client feedback with couples and therapists while First, we hypothesized that couples in the feedback
in therapy. Anker, Duncan, and Sparks (2009) condition would demonstrate better outcomes than
used the Partners for Change Outcome Manage- those in the TAU condition as measured by the
ment System (PCOMS; Duncan, Miller, & ORS after controlling for pre-ORS scores. Second,
Sparks, 2004) with a sample of 205 White Euro- we hypothesized that couples in the feedback con-
Scandinavian heterosexual couples. PCOMS condition would improve more quickly (i.e., in fewer
sists of two brief measures that are used to track sessions) than couples in the TAU condition. Third,
client progress in therapy during each session. we hypothesized that more couples in the feedback
The Outcome Rating Scale (ORS; Miller, Dun- condition would meet the criteria for clinical signif-
can, Brown, Sparks, & Claud, 2003) consists of icance at posttreatment than would couples in the
four items and measures client outcome, and the TAU condition.
Session Rating Scale (SRS; Duncan et al., 2003)
also consists of four items and measures the ther-
apeutic alliance. The ORS is administered at the Method
beginning of each session and the SRS is admin- Participants
istered and scored at the end of each session.
The results of each scale were administered, Clients. Clients were 46 heterosexual cou-
scored, and discussed every session. Participants ples (N 92) that received couple therapy during
in the study presented with a broad range of the course of an academic year at a graduate
relationship issues, including communication, training clinic for a marriage and family therapy
jealousy/conflict, and coping with partner’s phys- master’s program. There were 55 possible cou-
ical or psychological problems (Anker et al., ples, but nine couples (3 feedback condition,
2009). Couples were assigned to one of 10 ther- 6 TAU) did not return for a second session for
apists and randomly assigned to a treatment con- reasons unknown. The mean pretreatment ORS
dition: feedback or TAU. Couples in the feedback score for those that did not return for a second
condition reported statistically significantly session (23.36) was almost identical to those in-
higher residual ORS scores than couples in the cluded in the study (23.62). Seventy-four percent
TAU condition, yielding an effect size of d of the sample was Caucasian (n 68), 4.3%
0.50, which is considered large when comparing African American (n 4), 16.3% Hispanic/
the differences between treatments (Wampold, Latino (n 15), 3.2% multiracial (n 3), and
2001). Four times as many couples in the feed- 2.2% (n 2) did not indicate ethnicity. The mean
back condition experienced clinically significant age was 30.18 years (SD 9.71), with ages
change (i.e., change beyond the standard error of ranging from 19 to 56 years. The primary reason
measure that includes starting treatment below couples sought counseling included relationship
the clinical cut score and finishing treatment distress (n 36 couples; marital discord, com-
above the clinical cut score). Couples in the feed- munication, parenting, divorce, separation, extra-

618


marital affairs, sexual difficulties), individual dis- The internal consistency estimated with the
tress affecting the relationship (n 4 couples; ORS (first session) for the current sample was
pornography addiction, depression, anxiety, sex- .88, 95% CI [.84, .92]. Anker et al. (2009) re-
ual abuse), and relationship enhancement (n 6 ported an internal consistency coefficient alpha
couples; premarital, relationship enhancement). estimate of .93 with 410 individuals participating
Therapists. All of the 261 sessions at the in couple therapy. Reese et al. (2009) have found
training clinic for marriage and family therapy similar reliability estimates. Evidence of concur-
were provided by 13 second-year practicum stu- rent validity for scores derived from the ORS is
dents (7 women and 6 men; 10 Caucasian, 2 based on Pearson correlations with scores on
African American, 1 Hispanic) enrolled in an other established outcome measures, including
American Association for Marriage and Family the Symptom Checklist–90 —Revised (Deroga-
Therapy–approved program. Practicum students tis, 1992; r .57; Reese, Norsworthy, & Row-
received weekly individual and group supervi- lands, 2006), the Clinical Outcomes in Routine
sion. All of the couple sessions were video- Evaluation (Barkham et al., 2001; r .67; Miller
recorded for supervision purposes. One therapist & Duncan, 2004), and the OQ45 (r .59; Miller
met with the couples for a 50-min session typi- et al., 2003).
cally on a weekly basis. There were no session PCOMS. All therapists and supervisors in
limits, and the sessions did not follow a particular the feedback condition attended a 1-hr training
treatment format or protocol. Theoretical orienta- session that covered the rationale for using
tions of the student therapists across both treatment PCOMS and how to administer, score, and inter-
conditions were all grounded in a general family pret the ORS and SRS. The feedback condition
systems framework, using a variety of approaches used the protocol as outlined in the scoring and
including solution-focused, narrative/postmodern, administration manual for PCOMS (Miller &
and strategic therapy. The median number of cou- Duncan, 2004). Each client was administered the
ples seen by each therapist was three, ranging from ORS at the beginning of every couple session
one to eight couples. Therapists met with couples with the therapist present. After completing the
on average for 5.91 sessions (Mdn 5), ranging ORS (approximately 1 min), the therapist scored
from two to 17 sessions. the items in the session. The total score was
charted on a graph that indicated each client’s
progress across treatment. Because this study was
Measures conducted with couples, one chart was used that
ORS. The ORS is a four-item, self-report showed the individual progress of each partner.
measure that is designed to evaluate session-to- Therapists used the data within the session as
session progress made in therapy. Using a visual they saw fit, but the manual provides guidelines
analog scale, clients rate their level of psycholog- for how to intervene with clients who fall into the
ical distress on items adapted from the three areas following four categories:
of the Outcomes Questionnaire 45 (OQ45; Lam-
bert et al., 1996). Specifically, clients respond to ● No change. For a client who has not shown
how they are doing individually (personal well- reliable change (a gain of 5 points) after three
being), socially (work, school, friendships), inter- sessions, therapists are directed to address the
personally (family, close relationships), and therapeutic alliance and the course of treatment.
overall (general sense of well-being). Clients If the client has not demonstrated reliable im-
make a mark on each of the four analog scales provement after six sessions, the manual suggests
that are 10 cm in length, with marks near the left consultation, supervision, or staffing.
end of the scale indicating lower distress and ● Deteriorating. Clients in this category (a de-
marks near the right end of the scale indicating crease of 5 points since entering treatment) are
higher distress. A ruler or template is then used to considered to be at risk for terminating prema-
measure the distance from the left end of the turely or having a poor outcome. Therapists are
scale to the client’s mark. The score is recorded directed to discuss possible reasons for the
for each item to the nearest millimeter and then drop in score, review the SRS items with the
all are summed, for a total score ranging from 0 client to assess the therapeutic alliance, or con-
to 40. Lower scores reflect more distress. sider changing the treatment approach, fre-

619


quency, mode, or even therapist if no improve- nested wherein each client is nested within a
ment is noted after three sessions. couple, which is then nested within a therapist.
● Reliable change. Treatment is going accord- This means that the ORS scores of partners
ingly (evidenced by a gain of at least 5 points within the same couple are likely to be more
since beginning therapy). Therapists are ad- correlated than ORS scores for partners in differ-
vised to reinforce changes and to continue ent couples. In the language of MLM, each client
treatment until progress begins to plateau, is perceived as a Level-1 unit and couples are
whereupon a therapist should consider reduc- seen as a Level-2 unit. Similarly, ORS scores of
ing the frequency of sessions. couples within the same therapist are likely to be
● Clinically significant change. The client may more correlated than ORS scores for couples
no longer be struggling with issues that led to working with different therapists. As a result, the
seeking therapy. Clinically significant change language of MLM would consider therapist to be
is defined by a client beginning treatment be- a Level-3 unit.
low the clinical cut score of 25, improving at For the first research hypothesis, we predicted
least 5 points since starting therapy, and having that couples in the feedback condition would
a total score in the nonclinical range (25 or demonstrate better outcomes than those in the
above). Therapists are advised to consolidate TAU condition as measured by the ORS after
changes, anticipate potential setbacks, and con- controlling for pre-ORS scores. This means a
sider reducing the frequency of sessions. two-level cross-sectional multilevel model was
needed to address this hypothesis. The Level-2
The SRS was administered to each client and predictor is feedback condition (FEEDBACK;
again scored by the therapist (approximately 1 1 feedback condition; 0 TAU condition) and
min) toward the end of the session. If the total the Level-1 predictor or covariate is pre-ORS
score was below 36 or any one of the items was scores. Because the primary interest is in the
below 9, the therapist followed up and asked Level-2 predictor, FEEDBACK, pre-ORS scores
about the reason for the lower scores. The total were grand mean centered by subtracting each
score was then charted on a graph for the corre- client’s score from the overall mean pre-ORS
sponding session. Again, scores on the SRS for score (Mpre-ORS; for details on centering predic-
each partner in the couple were recorded on one tors in MLM, see Enders & Tofighi, 2007).
graph. The SRS was used as part of the feedback The multilevel model used to address the first
process for PCOMS, but the data were not in- research hypothesis or explain variation in ORS
cluded in the analyses for the current study. scores is
Y ij 00 01 FEEDBACKj
Data Analysis
10 pre-ORSij Mpre-ORS 0j rij, (1)
We applied multilevel modeling (MLM; Hox,
2002), also referred to as hierarchical linear mod- where Yij is the post-ORS score for client i in
eling (Raudenbush & Bryk, 2002), to answer the couple j; 00 is a fixed effect reflecting the overall
first two primary hypotheses (for a gentle intro- mean post-ORS for couples in the TAU condition
duction to MLM, see Peugh, 2010). In general, after controlling for pre-ORS scores; 01 is a
multilevel data tend to result when data are nat- fixed effect reflecting mean difference between
urally nested data structures (e.g., clients nested couples in the TAU and feedback conditions after
within therapists, therapists nested within a coun- controlling for pre-ORS scores (i.e., a positive dif-
seling center, repeated observations nested within ference would mean that couples in the feedback
clients, who are then nested within therapists). condition had a higher mean post-ORS than couples
The issue with nested data structures is that the in the TAU condition after controlling for pre-ORS
traditional assumption of independence of obser- scores); 01 is a fixed effect or covariate reflecting
vations is violated, which is necessary for tradi- the slope between pre- and post-ORS scores after
tional techniques such as analysis of variance controlling for FEEDBACK; 0j is a Level-2 ran-
(Peugh, 2010). Ignoring this issue will result in dom couple effect or the deviation of couple j from
biased parameter estimates (i.e., means, vari- the overall mean post-ORS for couples after con-
ances, and covariances) and increase Type I error trolling for pre-ORS scores; and rij is a Level-1
rates. In this study, the data structure is naturally random client effect or client ij’s difference in

620


post-ORS score from the overall mean post-ORS in ORS for a one-session increment in time); 200
for couples after controlling for pre-ORS scores. is the quadratic or curvature growth rate between
In MLM, the random effects are estimated as adjacent sessions for those in the TAU condition
2 2
variances such that Couple and Client capture the (i.e., the expected nonlinear change in ORS for a
intercept variances in ORS scores at the couple one-session increment in time); 001 is the mean
and client levels, respectively. Conceptually, difference between couples in the TAU and feed-
2
Couple measures the variation in mean post-ORS back conditions at the start of therapy; 101 is the
scores across couples that is not due to feed- average linear slope difference between couples
back condition and is similar to MSBetween in in the TAU and feedback conditions (i.e., a pos-
analysis of covariance (ANCOVA). Similarly, itive value would mean that couples in the feed-
2
Client is the average variance in individual back condition improved faster in ORS scores
clients’ scores within couples after accounting than those in the TAU condition); 201 is the
for pre-ORS scores and feedback condition and difference in curvature growth rates between cou-
is like MSWithin in ANCOVA. ples in the TAU and feedback conditions (i.e., a
For the second research hypothesis, we predicted positive value would mean that couples in the
that couples in the feedback condition would im- feedback condition have more positive curvature
prove more quickly (i.e., in fewer sessions) than growth rates than couples in the TAU condition);
couples in the TAU condition. This means that a 00j is a Level-3 random couple effect or the
three-level multilevel growth model was needed to deviation of each couple mean from the overall
address this hypothesis. In this model, repeated initial couple ORS mean; r0ij is a Level-2 random
observations or time represented Level 1, which client effect or the deviation of client ij’s ORS
are nested within each client (Level 2), which are score from the overall initial couple ORS mean;
then nested within each couple (Level 3). The and etij is a Level-1 random client effect or client
Level-3 predictor in this model is feedback con- ij’s residual error at session t (this error term
dition (FEEDBACK; as previously defined) and reflects the difference between each client’s pre-
Level-1 predictors are the time measure as a dicted and observed ORS score).
linear function (SESSION) and nonlinear func- Similar to the random effects estimated in the
tion (SESSION2). The nonlinear function of time multilevel model for the first hypothesis, the ran-
allows the model to capture the curvature in the dom effects in growth models are estimated as
2 2 2
ORS growth patterns, which is more realistic than variances such that Couple, Client, and Error
assuming all couples’ ORS scores grow in a estimate the intercept variances in ORS scores at
linear manner. the couple-level, client-level, and repeated obser-
2
The multilevel model used to address the sec- vations level, respectively. Conceptually, Couple
ond research hypothesis or explain variation in measures the variance in mean ORS scores across
ORS scores over sessions is couples that is not due to feedback condition,
2
Client is the average variance in individual cli-
Y tij 000 100 SESSIONtij ents’ scores within couples that is not due to
2 feedback condition (i.e., individual differences
200 SESSIONtij 001 FEEDBACKj 2
between clients), and Error captures within-
101 FEEDBACKj SESSIONtij person variation in ORS scores (i.e., the variabil-
2
ity of an individual client’s score around her or
201 FEEDBACKj SESSIONtij 00j his mean ORS score).
r0ij etij, (2) For all MLM analyses, predictors or covariates
were added to each of these basic models at the
where Ytij is the ORS score at session t for client appropriate level (client level or couple level). To
i in couple j; 000 is a fixed effect reflecting the compare the statistical fit of competing models,
overall average couple mean ORS at the start of we used the 2 log-likelihood (–2LL) value or
therapy for those in the TAU condition (centered deviance statistic from two nested models and
at Session 1 such that substituting 0 for SESSION found the difference in deviance estimates. Mod-
reflects the effect of the treatment at the first els are nested when one model is a subset of the
session); 100 is the overall average linear growth larger statistical model. Also, the deviance is a
rate between adjacent sessions for those in the measure of fit, and the higher the deviance, the
TAU condition (i.e., the expected linear change poorer the fit of the model to the sample data. The

621


difference in the deviances tests the null hypoth- in the TAU condition (see Table 1 for pre- and
esis that two models do not have statistically post-ORS mean treatment scores, standard devi-
significantly different model fits to the sample ations, and effect sizes within each condition). To
data. A rejection of this null hypothesis indicates evaluate the first hypothesis, we first estimated a
that the model with more estimated parameters model like that shown in Equation 1 except we
fits the sample data better than a model with included only the covariate pre-ORS scores
fewer estimated parameters. The difference in the (grand mean centered; Mpre-ORS 23.62) in the
deviance statistics is 2 distributed with degrees model (covariate-only model in Table 2). The
of freedom equal to the difference in parameters covariate-only model was estimated as a baseline
estimated by two nested models. The test of two model as is typically done in traditional hierar-
models’ deviances is often referred to in the sta- chal regression analyses to determine the incre-
tistical literature as a likelihood ratio test. All mental improvement of one model to the next.
MLM analyses were conducted with Proc Mixed The covariate-only model (see column 1 of Table
in SAS Version 9.2 using maximum likelihood 2) suggests a statistically significant positive
estimation and the Satterthwaite degrees of free- slope ( 10 0.26, p .001) between pre-ORS
dom method. scores and post-ORS scores across clients. This
means that scores improved from pre-ORS to
post-ORS, while the average post-ORS for a cli-
Results
ent with an average pre-ORS score was 30.17
Preliminary Analyses ( 00). The standardized mean effect size from
pre- to post-ORS was 0.71 ([30.17 – 23.62]/
Although our data are inherently nested within 9.21]), indicating that clients improved by almost
therapists at the highest level, initial MLM anal- three fourths of a standard deviation from pre- to
yses indicated that therapist fixed effects (i.e., post-ORS.
feedback condition vs. TAU) could not be di- One way to understand the overall utility of the
rectly estimated at the therapist level because of covariate-only model is to compute a global
the limited number of therapists used in this study pseudo-R2 effect size statistic, like multiple R2 in
(n 13), but therapist-level variance at the in- regression. This is done by correlating and squar-
2
tercept ( Therapist) could be estimated (i.e., the ing the predicted ORS scores for each participant,
variation in mean ORS scores across therapists). using the fixed effects parameters for the
It is important to note that ignoring the variability covariate-only model with the observed ORS
at the therapist level results in this variability scores for each client. The global pseudo-R2
being pushed into the variance estimates at other .13, which means that 13% of the variation in
levels of the multilevel model, which are then ORS scores can be explained by knowing the
ultimately biased. Therefore, we estimated the pre-ORS scores (see Peugh, 2010). To under-
2
Therapist for each of our models. As a result, we stand the specific amount of variability explained
used a three-level multilevel model to analyze the at a level, we computed a local pseudo-r2 statis-
nested structure of our data, clients nested within tic, which is similar to a semipartial r2 statistic in
couples, to address the first research hypothesis. traditional regression. As such, the estimated pro-
Similarly, we used a four-level multilevel growth portion of variance between couples explained by
model (session within client within couple within
therapist). However, each of these models is es- TABLE 1. Pretest and Posttest Mean Outcome Rating Scale
timating a single variance component at the ther- (ORS) Scores and Effect Sizes for the Client Feedback and
apist level and described as models with one less Treatment as Usual Conditions
level (i.e., ignoring therapist fixed effects because
they could not be estimated). Client Treatment
feedback as usual
(n 54) (n 38)
Did PCOMS Produce Differences in Outcomes Measure M SD M SD
for Couples?
Pre-ORS score 23.34 9.15 24.03 9.47
Descriptive statistics show that clients in the Post-ORS score 31.92 7.15 27.67 9.53
Standardized effect size 0.94 0.38
feedback condition improved 8.58 points com-
pared with the 3.64-point improvement by clients Note. Standardized effect size (Mpost – Mpre)/SDpre.

622


TABLE 2. Fixed and Random Effect Estimates for Multilevel Models Predicting
Postoutcome Rating Scale (ORS) Scores

Parameter Covariate-only model ANCOVA

Fixed effects (regression coefficients)
Intercept: Mean post-ORS ( 00 ) 30.17 (0.99) 27.56 (1.46)
Client pre-ORS ( 10 ) 0.26 (0.09) 0.27 (0.09)
Feedback ( 01 ) 4.44 (1.9)
Random effects (regression variances)
2
Client intercept variance ( Client) 33.27 (7.04) 33.48 (7.1)
2
Couple intercept variance ( Couple) 27.51 (10.31) 22.52 (9.35)
2
Therapist intercept variance ( Therapist) 1.14 0.99
Standardized effect size 0.71a 0.48b
Note. Standard errors are in parentheses. Client pre-ORS client’s initial ORS score
grand mean centered; Feedback type of feedback condition (0 treatment as usual;
1 feedback).
a
Standardized effect size (M post Mpre)/SDpre.

01
b
Standardized effect size
2 2
nTAU 1 sTAU post ORS nFeedback 1 sFeedback post ORS
.
N 2
p .05. p .01. p .001.

the covariate-only model with pre-ORS is 0.27 points higher than couples in the TAU condition
(i.e., [37.63 – 27.51]/37.63). This means that 27% after controlling for pre-ORS scores (see last
of the between-couples variance in post-ORS column of Table 2). The standardized mean effect
scores is accounted for by knowing the pre-ORS size between couples’ ORS scores after control-
scores. Moreover, the intraclass correlation for ling for pre-ORS scores was 0.54 (see formula at
2 2 2
couple (ICCCouple Couple/[ Client Couple bottom of Table 2; U.S. Department of Educa-
2
Therapist]) was .44 (which had been .53). This tion: Institute of Education Sciences, 2008, p.
means that 44% of the variance in post-ORS 43). This means that the feedback condition
scores is due to pre-ORS scores. The ICCTherapist scored just over half a standard deviation higher
was .02, meaning that 2% of the variability in on post-ORS scores than the TAU condition after
post-ORS scores (after controlling for pre-ORS controlling for pre-ORS scores, which is within
scores) was attributed to therapists, which is the range of other naturalistic therapist effects
within the range of other naturalistic therapist (see Anker et al., 2009).
effects (see Anker et al., 2009; Baldwin, Berkel- When we examine the ANCOVA model ran-
jon, Atkins, Olsen, & Nielsen, 2009). dom effects, we see that 39.5% of the variance in
For the second model we added treatment con- post-ORS scores remains between couples.
dition (FEEDBACK) as a predictor to the former Moreover, comparing the ICCCouple from both
model (see Equation 1). The second model can be models (covariate only vs. ANCOVA), the pro-
conceptually thought of as a multilevel ANCOVA portion of variance between couples explained by
model (see column 2 of Table 2). The –2LLs for the the ANCOVA model is (27.51 – 22.52)/27.51
covariate-only and ANCOVA models were 629.6 .18 or 18%. That is, 18% more between-couples
and 624.4, respectively. The difference in fit be- variance in post-ORS scores is explained by
tween these two models was statistically signifi- knowing the type of feedback condition. The
cant, 2(1) 5.2, p .02. Results from this global pseudo-R2 effect size statistic for the
approach suggest that including a model with a ANCOVA model was .19.
treatment effect for feedback while controlling
for pre-ORS scores improves the overall fit of the Preliminary Growth Curve Analyses
model to the data versus including only pre-ORS
scores. This means that couples in the feedback In the MLM literature, it is recommended that
condition scored on average 4.44 ( 00) ORS a minimum of four time points be used to specify

623


a quadratic (or nonlinear) group model (Willett, p .001) and quadratic ( 100 0.08, p
Singer, & Martin, 1998). By adding a nonlinear .001) growth rates (see Table 3). The global
component to the model, researchers can increase pseudo-R2 effect size statistic for the uncondi-
precision in estimates of change (Muthen, 1999;
´ tional model was .09, which suggests that 9% of
Muthen & Curran, 1997). However, couples var-
´ the variation in ORS scores can be explained by
ied in the number of sessions attended. Results knowing linear change and quadratic change.
from preliminary analyses identified an outlier To evaluate whether growth rates vary be-
couple (17 sessions) as an influential case and tween couples receiving TAU versus feedback
was subsequently removed from all subsequent during couple therapy (Hypothesis 2; Equation
multilevel growth curve analyses. Although the 2), we added treatment condition (FEEDBACK)
minimum recommended number of sessions for a to the growth model (conditional growth model;
quadratic growth model is four, we chose to see Table 3). The –2LLs for the unconditional
include all couples with at least two sessions and conditional growth models were 2,684.8 and
because models including all couples did not 2,675.9, respectively. The difference in fit between
differ from models including couples who at- these two models was statistically significant,
tended a minimum of four sessions. Moreover, 2
(3) 8.9, p .053, indicating that couples in the
including couples with fewer than four sessions feedback condition improved more quickly than
increases the generality of the results to natural- couples in the TAU condition. The global
istic settings and helps maintain adequate statis- pseudo-R2 effect size statistic for the conditional
tical power.
model was .10. Inspection of the conditional growth
model results specifically shows that clients receiv-
Growth Curves for Feedback and TAU ing feedback during couple therapy have a statisti-
Conditions cally significant different linear growth rate com-
To evaluate the second hypothesis, we first pared with those in the TAU condition ( 101 1.5,
estimated a model like that shown in Equation 2 p .02, d 0.81, i.e., d [effect(time)]/SDpre,
except that we did not include feedback condition where time 5 and SDpre 9.31; for more details,
(FEEDBACK) in the growth model (see uncon- see Feingold, 2009, p. 7). Because the session num-
ditional growth model in Table 2). The uncondi- bers varied across couples, the value of time was set
tional growth model estimates an intercept or to 5 to reflect the median number of sessions at-
average starting mean ORS score, linear growth tended., However, there was not a statistically sig-
rate, and nonlinear (quadratic) growth rate across nificant difference in the conditions’ quadratic
couples. These results indicate that all couples growth rates ( 201 0.11, p .14). A depiction
start with an average ORS score of 24.46 and of the difference in these growth rates up to five
have statistically significant linear ( 100 1.87, sessions is presented in Figure 1. We chose to stop

TABLE 3. Fixed and Random Effect Estimates for Multilevel Growth Models for Outcome Rating Scale (ORS) Scores

Parameter Unconditional growth model Conditional growth model

Fixed effects (regression coefficients)
Intercept: Mean ORS ( 000 ) 24.46 (1.07) 24.53 (1.65)
Session ( 100 ) 1.87 (0.28) 0.91 (0.56)
Session2 ( 200 ) 0.08 (0.03) 0.01 (0.07)
Feedback ( 001 ) 0.001 (2.16)
Feedback Session ( 101 ) 1.5 (0.65)
Feedback Session2 ( 201 ) 0.11 (0.07)
Random effects (regression variances)
2
Error variance ( Error) 32.53 (2.62) 31.68 (2.55)
2
Client intercept variance ( Client) 10.01 (4.14) 10.3 (4.16)
2
Couple intercept variance ( Couple) 24.48 (9.82) 23.66 (9.68)
2
Therapist intercept variance ( Therapist) 10.7 10.78
Note. Standard errors are in parentheses. Session session number centered at Session 1; Feedback type of
feedback condition (0 treatment as usual; 1 feedback).
p .05. p .01. p .001.

624


40 the client finishes therapy above an established
35 cut score that separates a clinical from nonclini-
Predicted ORS

30 cal population. The cut score for the ORS is 25.
25 The reliable change index and cut score for the
20 ORS were based on two samples from a commu-
15 nity mental health center (Miller et al., 2003) and
10 a residential alcohol and drug treatment center
1 2 3 4 5 (Miller, Mee-Lee, Plum, & Hubble, 2005).
Session
More clients in the feedback condition, both at
Feedback TAU the individual and couple level, completed treat-
FIGURE 1. Average growth curves across five sessions for ment having obtained reliable (gain of 5 points)
the feedback (dashed line) and treatment as usual (TAU; solid and clinically significant (gain of 5 points and
line) conditions.
crossing the clinical threshold) change when
compared with clients in the TAU condition (see
at five sessions given that the sample median num- Table 4). Approximately 65% of clients at the
ber of sessions per couple was five. Figure 1 depicts individual level reported reliable or clinical
the quicker improvement in ORS scores for the change in the feedback condition compared with
feedback condition over the TAU condition after approximately 31.6% in the TAU condition. At
five sessions. the couple level (only couples where both part-
ners met the criteria were included), 44.4% of the
Clinical Significance couples in the feedback condition compared with
15.8% of the couples in the TAU condition re-
Observing the number of clients who incur ported reliable or clinically significant change.
clinically significant change across treatment has
Four times as many couples in the feedback con-
become a common way to assess psychotherapy
dition were categorized as obtaining clinically
outcome (Lambert, Hansen, & Bauer, 2008).
Jacobson and Truax (1991) developed formulas significant change. It is important to note that
to evaluate change in therapy using the terms only 15 couples in the feedback condition and 11
reliable change and clinically significant change couples in the TAU condition were eligible to
to denote meaningful change in therapy. Reliable achieve clinical significance (both partners had
change is simply the increase or decrease in a pre-ORS 25); if this is considered, then 53.3%
client’s score on an outcome measure that ex- of the eligible couples in the feedback condition
ceeds the measurement error for the instrument. achieved clinical significance and 18.2%
For the ORS, the amount of change needed to achieved clinical significance in the TAU condi-
incur reliable change is 5 or more points. A tion. Chi-square analyses indicated that the dif-
decrease of 5 or more points is termed deterio- ferences in the outcome classifications across
ration. Clinically significant change occurs when treatment conditions were statistically significant
a client has reliable change (gain of 5 points) and at both the individual, 2(3, N 92) 10.42,

TABLE 4. Individuals and Couples That Achieved Clinical Significance or Reliable Change in the Client Feedback and
Treatment as Usual Conditions

Individuals Couples
Client feedback Treatment as Client feedback Treatment as
(n 54) usual (n 38) (n 27) usual (n 19)
Classification n % n % n % n %

1. Deteriorated 4 7.4 4 10.4 1 3.7 1 5.3
2. No change 15 27.8 22 57.9 3 11.1 9 47.4
3. Reliable change 9 16.7 2 5.3 4 14.8 1 5.3
4. Clinically significant change 26 48.1 10 26.3 8 29.6 2 10.5
5. Not classified 11 40.7 6 31.6
Note. Couples were classified only if both partners completed treatment in the same category.

625


2
p .02, and couple, (3, N 46) 8.18, p therapy achieved clinical significance. At the
.04, levels. couple level, approximately 30% of the couples
in the feedback condition that completed treat-
Discussion ment were classified as clinically significant com-
pared with only 10.5% in the TAU condition.
An impressive amount of research has accu- These rates are much lower than those reported
mulated that supports the efficacy of using con- by Christensen et al. (2004), who found that 52%
tinuous outcome assessment (i.e., client feed- of couples that received a form of behavior ther-
back) in individual psychotherapy. Little research apy reported clinically significant change. This
has been conducted to evaluate whether using comparison is problematic, however, on the sur-
client feedback in couple therapy would yield face. The couples in the Christensen et al. study
similar results. Our study investigated whether were chronic and more distressed and also at-
the benefits of using a client feedback system, tended more sessions (22.9 sessions vs. 5.9 ses-
PCOMS (Duncan et al., 2004), would extend to sions) than the couples in our sample. When
couples in therapy. Results indicated that couples couples that were ineligible for clinical signifi-
randomly assigned to a feedback condition expe- cance (pre-ORS scores 25) are removed from
rienced statistically significant more improve- our sample, the rate of achieving clinical signif-
ment than those in the TAU condition and also icance is 53.3% for couples in the feedback con-
improved more quickly as evidenced by a steeper dition, which is comparable to the Christensen et
growth curve. Couples in the feedback condition al. study.
were also more likely to incur reliable and clin- The differences between pre- and post-ORS
ically significant change. The results of this study scores for couples are similar to client feedback
are comparable with the studies that have used studies that used PCOMS compared with TAU
PCOMS with individuals to address outcome for individual psychotherapy. Although the cur-
(Miller et al., 2005; Reese et al., 2009) and with rent differences in outcome are slightly smaller
the only other couple therapy study using than the 10.8-points gain found in Miller et al.
PCOMS (Anker et al., 2009). (2005) and the 12.69- and 10.83-point gains
In our study, couples in the feedback condition found in two samples by Reese et al. (2009), the
experienced treatment gains more than double of difference in scores between treatment conditions
those in the TAU condition on the ORS (8.58 (at least double) is similar.
points vs. 3.64 points). Findings from Anker et al. Although the results of our study closely re-
(2009) found almost identical treatment gains semble the Anker et al. (2009) couple study, there
when comparing feedback and TAU conditions are two differences of note. First, the therapists in
(8.3 points vs. 3.11 points). Effect sizes for the our sample were all graduate trainees and the
feedback condition in both studies were also therapists in the Anker et al. study were licensed
found to be large (d 0.8; Cohen, 1992). In professionals. Second, the trainees in our study
addition, the effect size for the difference be- received much less training (1 hr vs. 8 hr). The
tween the feedback condition and TAU condition trainees in our study, however, did have supervi-
(d 0.48) was also similar to the Anker et al. sors who were able to provide continued instruc-
(2009) effect size (d 0.50). tion and discuss couple progress and the ORS and
When observing treatment effectiveness from SRS measures. Our study provides evidence that
a clinical significance perspective, clients in the client feedback is useful for therapy trainees who
feedback condition were more likely to experi- provide couple therapy.
ence clinically significant change (48.1%) com-
pared with those in the TAU condition (26.3%). Limitations of Our Study
Similar to the Anker et al. (2009) study, we found
that 4 times as many couples in the feedback There are multiple limitations of our study that
condition were classified as having incurred clin- warrant mentioning. First, many clients in the
ically significant change when compared with feedback condition had missing session data. We
those in the TAU condition. The results for the were not able to discern a pattern for the missing
feedback condition are comparable to Shadish data other than from anecdotal evidence from
and Baldwin’s (2003) summary of meta-analytic therapists that indicated both logistical issues (“I
studies that reported 40 –50% of clients in couple forgot” or “I did not bring copies of the measures

626


to the session”) and clinical reasons (“It did not were aware of PCOMS, and some expressed frus-
feel necessary every week” or “The couple had a tration with not being able to use it with their
crisis and it did not feel appropriate to use”). All clients. It is possible that they may have been
clients had ORS and SRS data for at least half of applying components of the system verbally with
their sessions, but the consequences of these their clients. This possibility, however, was not
missing data may have led to underestimating the monitored or evaluated.
effects of the feedback intervention. Concerns for A fourth concern is that only therapist trainees
this limitation are tempered by the similar results were used, thereby limiting the generalizability of
in our study compared with those in the Anker et the results to therapists with more experience.
al. (2009) study. Future research should investi- More experienced therapists may have used
gate the influence of administering a continuous PCOMS more effectively and demonstrated
assessment system every session compared with larger treatment gains, or conversely, the lack of
every second or third session. Such a study would experience may have heighted the demand char-
address the potential differential effects for some acteristics and led to an overestimate of treatment
clinics that administer continuous assessment effects. For example, couples were aware that
systems every few sessions rather than every their therapist was a trainee being evaluated and
session. perhaps did not want to negatively influence the
A second limitation is that we did not use student’s grade. We do not believe, however, that
multiple outcome measures, such as marital sat- this is a large concern. A previous study we
isfaction or couple distress. We are also unable to conducted (Reese et al., 2009) did not show treat-
extrapolate the results of our study to assume that ment outcome differences between licensed, pro-
larger treatment gains in the feedback condition fessional staff and trainees. In addition, the Anker
resulted in more couples remaining together et al. (2009) couple study found similar treatment
when compared with the couples in the TAU outcomes with experienced therapists.
condition. The findings of our study, however,
are very similar to those in the Anker et al. (2009) Future Research and Conclusions
study, which provide evidence of the validity for
the ORS in couple therapy. Treatment gains as The use of PCOMS with both individuals and
measured by the ORS in their study also showed couples appears to have much promise, but more
treatment gains on an established measure of research is needed to clarify the variables and
marital adjustment, the Locke–Wallace Marital mechanisms of change associated with the posi-
Adjustment Test (Locke & Wallace, 1959). Cou- tive outcomes found in studies using PCOMS.
ples who had better outcomes as measured by the There is little understanding of why PCOMS
ORS were also more likely to remain together at leads to better outcomes, and until these pro-
a 6-month follow-up. cesses are better understood, the confidence one
A third limitation is the lack of consistently can attribute to the specific effects of PCOMS is
monitored treatment integrity. The appealing limited. Continuous assessment originally was
qualities of PCOMS are that it is easy to imple- designed to identify clients not progressing as
ment and provides the therapist with latitude as expected. It would logically follow that being
how to best integrate the measures into treatment. able to identity clients early in treatment who are
Although conducting a study in a naturalistic not improving would afford the therapist the op-
setting is a strength, a weakness is being uncer- portunity to alter treatment. However, PCOMS
tain of the differences in how PCOMS was im- has been found to work with all clients, including
plemented. The effects of feedback may have those progressing as expected. Most of the re-
been underestimated. Anecdotally, therapists research using the OQ45 has found that continuous
ported differing levels of allegiance to using assessment is more beneficial for clients not pro-
PCOMS. This concern is tempered by the fact gressing as expected (e.g., Whipple et al., 2003).
that therapists received weekly supervision and PCOMS differs from Lambert and colleagues’
supervisors were encouraged to ensure protocol (1996) signal system because PCOMS uses a
compliance by noting the use of PCOMS in measure of the therapeutic relationship every ses-
video-recorded sessions (all sessions were re- sion, whereas the Lambert et al. system uses an
corded) and through verbal reminders in supervi- alliance measure when a client is not improving
sion. Also, the therapists in the TAU condition as expected. Does this difference matter? The

627


authors of PCOMS (Duncan et al., 2004) have With the increased need to demonstrate psy-
opined that having access to weekly feedback chotherapy’s utility due to forces such as man-
regarding the relationship may serve to heighten aged care and third-party reimbursement, mea-
attention and focus on the therapeutic alliance suring the progress of treatment as it occurs has
and promote active collaboration. Future research become an important area of study with exciting
should attempt to isolate the contribution of the results. Ongoing client feedback has been found
SRS to the effectiveness of PCOMS. to help avoid premature termination and meet the
PCOMS may also be effective because seeing needs of clients in a more effective, efficient
the measures weekly creates expectancy effects manner. Overall, the results of this study indicate
regarding improvement. A second possibility is that using client feedback is a useful approach
that cognitive dissonance plays a role in reporting with couples that received treatment at a graduate
improved outcome, and a good therapeutic rela- training clinic and are consistent with the findings
tionship may increase the likelihood that clients from previous client feedback studies focused on
feel better and have a good therapeutic relation- individual therapy. More research needs to be
ship. Another possibility is that seeing one’s conducted, but PCOMS appears to hold much
graphed progress promotes improvement; there is promise for use with couples given its ease of use
an established body of literature in psychology and encouraging results.
that points to the importance of receiving feed-
back for promoting behavioral change (e.g., References
Alvero, Bucklin, & Austin, 2001). Using
PCOMS may stimulate such behavioral changes ACKERMAN, S., SMITH, B., BEUTLER, L. E., GELSO, C. J.,
for both clients and therapists. Studies that ma- GOLDFRIED, M. R., HILL, C., & RAINER, J. (2001).
nipulate the manner in which therapist and client Empirically supported therapy relationships: Conclu-
sions and recommendations of the Division 29 Task
observe the data would help address these issues. Force. Psychotherapy Theory, Research, Practice,
Similar studies have been conducted with the Training, 38, 495– 497. doi:10.1037/0033–3204.38.4.495
OQ45 (e.g., Harmon et al., 2007; Hawkins et al., ALVERO, A. M., BUCKLIN, B. R., & AUSTIN, J. (2001). An
2004), but as mentioned earlier, the findings have objective review of the effectiveness and essential char-
acteristics of performance feedback in organizational
been mixed. settings. Journal of Organizational Behavior Manage-
Future research should also investigate ment, 21, 3–29. doi:10.1300/J075v21n01_02
whether demand characteristics influence scores ANKER, M. G., DUNCAN, B. L., & SPARKS, J. A. (2009).
on the ORS and SRS given that the measures are Using client feedback to improve couple therapy out-
completed and then directly discussed with the comes: A randomized clinical trial in a naturalistic set-
ting. Journal of Consulting and Clinical Psychology, 77,
therapist. The SRS, in particular, appears suscep- 693–704. doi:10.1037/a0016062
tible to social desirability because the client is BALDWIN, S. A., BERKELJON, A., ATKINS, D. C., OLSEN,
evaluating the quality of the session in the pres- J. A., & NIELSEN, S. L. (2009). Rates of change in
ence of the therapist. It is possible that the de- naturalistic psychotherapy: Contrasting dose-effect and
good-enough level models of change. Journal of Con-
mand characteristics are heighted in couple ther- sulting and Clinical Psychology, 77, 203–211. doi:
apy given the added presence of a partner, which 10.1037/a0015235
makes the results even more “public.” Perhaps BARKHAM, M., MARGISON, F., LEACH, C., LUCOCK, M.,
simply the process of asking the client to evaluate MELLOR-CLARK, J., EVANS, C., & MCGRATH, G.
the session is pivotal because it overtly commu- (2001). Service profiling and outcome benchmarking
using the CORE-OM: Toward practice-based evidence
nicates that the therapist values the client’s input in the psychological therapies. Journal of Consulting
and the scores from the measure are less impor- and Clinical Psychology, 69, 184 –196. doi:10.1037/0022-
tant. To evaluate the role of social desirability, a 006X.69.2.184
study could have three randomized treatment CHRISTENSEN, A., ATKINS, D. C., BERNS, S., WHEELER, J.,
BAUCOM, D., & SIMPSON, L. E. (2004). Traditional ver-
conditions use PCOMS. One condition would use sus integrative behavioral couple therapy for signifi-
PCOMS as typically prescribed, the second con- cantly and chronically distressed married couples. Jour-
dition would complete the SRS without the ther- nal of Consulting and Clinical Psychology, 72, 176 –191.
apist present and see the results after the client doi:10.1037/0022-006X.72.2.176
has left, and the third group would complete the COHEN, J. (1992). A power primer. Psychological Bulle-
tin, 112, 155–159. doi: 10.1037/0033-2909.112.1.155
SRS without the therapist having access to the DEROGATIS, L. R. (1992). Administration, scoring and
results. Differences in the scores and outcome procedures manual II. Baltimore: Clinical Psychometric
could be assessed. Research.

628


DUNCAN, B. L., MILLER, S. D., REYNOLDS, L., SPARKS, J., 45.2. Stevenson, MD: American Professional Creden-
CLAUD, D., BROWN, J., & JOHNSON, L. D. (2003). The tialing Services.
Session Rating Scale: Psychometric properties of a “work- LAMBERT, M. J., HARMON, C., SLADE, K., WHIPPLE, J. L.,
ing” alliance scale. Journal of Brief Therapy, 3, 3–12. Re- & HAWKINS, E. J. (2005). Providing feedback to psy-
trieved from http://www.journalbrieftherapy.com/ chotherapists on their patients’ progress: Clinical re-
DUNCAN, B. L., MILLER, S. D., & SPARKS, J. A. (2004). sults and practice suggestions. Journal of Clinical Psy-
The heroic client: A revolutionary way to improve effec- chology, 61, 165–174. doi:10.1002/jclp.20113
tiveness through client-directed, outcome-informed ther- LAMBERT, M. J., & OGLES, B. (2004). The efficacy and
apy. San Francisco: Jossey-Bass. effectiveness of psychotherapy. In M. J. Lambert (Ed.),
ENDERS, C. K., & TOFIGHI, D. (2007). Centering predic- Bergin and Garfield’s handbook of psychotherapy and
tor variables in cross-sectional multilevel models: A behavior change (5th ed., pp. 139 –193). New York: Wiley.
new look at an old issue. Psychological Methods, 12, LAMBERT, M. J., WHIPPLE, J. L., HAWKINS, E. J., VER-
121–138. doi:10.1037/1082-989X.12.2.121 MEERSCH, D. A., NIELSEN, S. L., & SMART, D. W.
FEINGOLD, A. (2009). Effect sizes for growth-modeling (2003). Is it time for clinicians to routinely track patient
analysis for controlled clinical trials in the same metric outcome? A meta-analysis. Clinical Psychology, 10,
as for classical analysis. Psychological Methods, 14, 43– 288 –301. doi:10.1093/clipsy/bpg025
53. doi:10.1037/a0014699 LAMBERT, M. J., WHIPPLE, J. L., SMART, D. W., VER-
GOLLAN, K. K., & JACOBSON, N. S. (2002). Developments MEERSCH, D. A., NIELSEN, S. L., & HAWKINS, E. J.
in couple therapy research. In H. A. Liddle, D. A. (2001). The effects of providing therapists with feed-
Santisteban, R. F. Levant, & J. H. Bray (Eds.), Family back on patient progress during psychotherapy: Are
psychology: Science-based interventions (pp. 105–122). outcomes enhanced? Psychotherapy Research, 11, 49 –
Washington, DC: American Psychological Association. 68. doi:10.1093/ptr/11.1.49
H ARMON , C. S., L AMBERT , M. J., S MART , D. M., LAMBERT, M. J., WHIPPLE, J. L., VERMEERSCH, D. A.,
HAWKINS, E., NIELSEN, S. L., SLADE, K., & LUTZ, W. SMART, D. W., HAWKINS, E. J., NIELSON, S. L., &
(2007). Enhancing outcome for potential treatment fail- GOATES, M. (2002). Enhancing psychotherapy out-
ures: Therapist– client feedback and clinical support comes via providing feedback on client progress: A
tools. Psychotherapy Research, 17, 379 –392. doi: replication. Clinical Psychology and Psychotherapy, 9,
10.1080/10503300600702331 91–103. doi:10.1002/cpp.324
LOCKE, H. J., & WALLACE, K. M. (1959). Short marital-
HAWKINS, E. J., LAMBERT, M. J., VERMEERSCH, D. A.,
adjustment and prediction tests: Their reliability and
SLADE, K. L., & TUTTLE, K. C. (2004). The therapeutic
validity. Marriage and Family Living, 21, 251–255. Re-
effects of providing patient progress information. Psy-
trieved from http://www.jstor.org
chotherapy Research, 14, 308 –327. doi:10.1093/ptr/
LUTZ, W., MARTINOVICH, Z., & HOWARD, K. (1999).
kph027
Patient profiling: An application of random coefficient
HORVATH, A. O., & SYMONDS, B. D. (1991). Relation
regression models to depicting the response of a patient
between working alliance and outcome in psychother-
to outpatient psychotherapy. Journal of Consulting and
apy: A meta-analysis. Journal of Counseling Psychol- Clinical Psychology, 67, 571–577. doi:10.1037/0022-
ogy, 38, 139 –149. doi:10.1037/0022– 0167.38.2.139 006X.67.4.571
HOWARD, K. L., KOPTA, S. M., KRAUSE, M. S., & MILLER, S. D., & DUNCAN, B. L. (2004). The Outcome
ORLINKSY, D. E. (1986). The dose-effect relationship in and Session Rating Scales: Administration and scoring
psychotherapy. American Psychologist, 41, 159 –164. manual. Ft. Lauderdale, FL: Author.
doi:10.1037/0003-066X.41.2.159 MILLER, S. D., DUNCAN, B. L., BROWN, J., SORRELL, R., &
HOX, J. (2002). Multilevel analysis techniques and appli- CHALK, B. (2006). Using outcome to inform and improve
cations. Mahwah, NJ: Erlbaum. treatment outcomes. Journal of Brief Therapy, 5, 5–22.
ISAKSON, R. L., HAWKINS, E. J., HARMON, S. C., SLADE, Retrieved from http://www.journalbrieftherapy.com/
K., MARTINEZ, J. S., & LAMBERT, M. J. (2006). Assess- MILLER, S. D., DUNCAN, B. L., BROWN, K., SPARKS, J., &
ing couple therapy as a treatment for individual dis- CLAUD, D. (2003). The Outcome Rating Scale: A prelim-
tress: When is referral to couple therapy contraindi- inary study of the reliability, validity, and feasibility of a
cated? Contemporary Family Therapy, 28, 313–322. doi: brief visual analog measure. Journal of Brief Therapy, 2,
10.1007/s10591-006 –9008-9 91–100. Retrieved from http://www.journalbrieftherapy
JACOBSON, N. S., & TRUAX, P. (1991). Clinical signifi- .com/
cance: A statistical approach to defining meaningful MILLER, S. D., MEE-LEE, D., PLUM, W., & HUBBLE,
change in psychotherapy research. Journal of Consult- M. A. (2005). Making treatment count: Client-directed,
ing and Clinical Psychology, 59, 12–19. doi:10.1037/ outcome informed clinical work with problem drinkers.
0022-006X.59.1.12 In J. L. Lebow (Ed.), Handbook of clinical family ther-
LAMBERT, M. J., HANSEN, N. B., & BAUER, S. (2008). apy (pp. 281–308). Hoboken, NJ: Wiley.
Assessing the clinical significance of outcome results. In MUTHEN, B. O. (1999, October 29). Re: Timepoints [Online
´
A. M. Nezu & C. M. Nezu (Eds.), Evidence-based discussion]. Retrieved from http://www.statmodel
outcome research: A practical guide to conducting ran- .com/discussion/messages/14/20.html#POST16727
domized controlled trials for psychosocial interventions MUTHEN, B. O., & CURRAN, P. J. (1997). General longi-
´
(pp. 359 –378). New York: Oxford University Press. tudinal modeling of individual differences in experi-
LAMBERT, M. J., HANSEN, N. B., UMPHRESS, V., LUNNEN, mental designs: A latent variable framework for anal-
K., OKIISHI, J., BURLINGAME, G., & REISINGER, C. ysis and power estimation. Psychological Methods, 2,
(1996). Administration and scoring manual for the OQ 371– 402. doi:10.1037/1082-989X.2.4.371

629


PEUGH, J. L. (2010). A practical guide to multilevel mod- SNYDER, D. K., CASTELLANI, A. M., & WHISMAN, M. A.
eling. Journal of School Psychology, 48, 85–112. doi: (2006). Current status and future directions in couple
10.1016/j.jsp.2009.09.002 therapy. Annual Review of Psychology, 57, 317–344.
RAUDENBUSH, S. W., & BRYK, A. S. (2002). Hierarchical doi:10.1146/annurev.psych.56.091103.070154
linear models: Applications and data analysis methods TAMBLING, R. B., & JOHNSON, L. N. (2008). The relation-
(6th ed.). Newbury Park, CA: Sage. ship between stages of change and outcome in couple
REESE, R. J., NORSWORTHY, L. A., & ROWLANDS, S. therapy. American Journal of Family Therapy, 36, 229 –
(2006, August). Does a popular client feedback model 241. doi:10.1080/01926180701290941
improve psychotherapy outcome? Poster session pre- U.S. Department of Education: Institute of Education
sented at the annual meeting of the American Psycho- Sciences. (2008). What works clearing house: Procedures
logical Association, New Orleans, LA. and standards handbook (version 2.0). Retrieved from
REESE, R. J., NORSWORTHY, L., & ROWLANDS, S. (2009). http://ies.Ed.gov/ncee/wwc/pdf/wwc_procedures_
Does a continuous feedback model improve psycho- v2_standards_handbook.pdf
therapy outcomes? Psychotherapy Theory, Research, WAMPOLD, B. E. (2001). The great psychotherapy debate:
Practice, Training, 46, 418 – 431. doi:10.1037/a0017901 Models, methods, and ﬁndings. Hillsdale, NJ: Erlbaum.
SHADISH, W. R., & BALDWIN, S. A. (2003). Meta-analysis WHIPPLE, J. L., LAMBERT, M. J., VERMEERSCH, D. A.,
of MFT interventions. Journal of Marital and Family SMART, D. W., NIELSEN, S. L., & HAWKINS, E. J. (2003).
Therapy, 29, 547–570. doi:10.1111/j.1752– 0606 Improving the effects of psychotherapy: The use of
.2003.tb01694.x early identiﬁcation of treatment failure and problem-
SHADISH, W. R., & BALDWIN, S. A. (2005). Effects of solving strategies in routine practice. Journal of Coun-
behavioral marital therapy: A meta-analysis of random- seling Psychology, 50, 59 – 68. doi:10.1037/0022–
ized controlled trials. Journal of Consulting and Clinical 0167.50.1.59
Psychology, 73, 6 –14. doi:10.1037/0022-006X.73.1.6 WILLETT, J. B., SINGER, J. D., & MARTIN, N. C. (1998).
SHADISH, W. R., MONTGOMERY, L., WILSON, P., WILSON, The design and analysis of longitudinal studies of de-
M., BRIGHT, I., & OKWUMABUA, T. (1993). The effects velopment and psychopathology in context: Statistical
of family and marital psychotherapies: A meta-analysis. models and methodological recommendations. Devel-
Journal of Consulting and Clinical Psychology, 61, 992– opment and Psychopathology, 10, 395– 426. doi:
1002. doi:10.1037/0022-006X.61.6.992 10.1017/S0954579498001667

630

ReeseTolandSloneNorsworthy2010

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a ReeseTolandSloneNorsworthy2010

Similar a ReeseTolandSloneNorsworthy2010 (20)

Último

Último (20)

ReeseTolandSloneNorsworthy2010