Study abroad provides students with a unique opportunity to develop foreign language skills alongside native speakers. Or does it? This panel will look at different assessment methods and tools used to measure student linguistic gains after a semester studying abroad in France, Spain, and Argentina. Special emphasis will be placed on “high-impact” practices that contribute to positive gains in language learning during study abroad experiences including homestays, direct enrollment, and community engagement.

1. Measuring Linguistic Gains in Immersion
Settings: Empirical Studies from French and
Spanish as a Second Language
Dennis M. Wiseman
Council for International Education Exchange
Annual Conference
Breaking Barriers: Cost, Curriculum, Culture
Baltimore, Md
November 19-22, 2014

2. Wofford College
• Four-year, private, liberals arts college
• Founded in 1854
• Methodist affiliated
• Phi Beta Kappa
• # 4 in the Open Doors survey
• 1600 students

3. National Survey of Student Engagement

4. National Survey of Student Engagement

5. National Survey of Student Engagement

6. National Survey of Student Engagement

7. Linguistic Gains: High Impact
Practices
All French or Spanish students in these data
have engaged in at least one of the following
activities in study abroad.
– Home stay
– Language course work
– Internship
– Volunteer or service learning

8. Methodology
• OPIc
• Immediately upon return from study abroad,
set / proctored
• French students
– Paris, Nantes, Rennes, Bordeaux, Aix-en-
Provence, Dijon, Arles
• Spanish students
– Spain, Argentina, Panama, Chile, Dominican
Republic, Peru, Ecuador, Bolivia, Honduras,
Nicaragua

9. SPAIN
LAT. AMER.
SPANISH_STUDY_ABROAD_2007_'13
0 20 40 60 80 100 120 140
LAT. AMER. SPAIN
Series1 118 77

10. ACTFL_OPI_DID_NOT_PARTICIPATE_#
0 20 40 60 80 100 120 140
SPAIN DNT
SPAIN
LAT. AMER DNT
LAT. AMER.
LAT. AMER. LAT. AMER DNT SPAIN SPAIN DNT
Series1 118 16 77 29

11. SPANISH_LAT._AMER._ACTFL_2007_2013_#
0 5 10 15 20 25 30 35 40 45 50
DNT
IL
IM
IH
AL
AM
AH
AH AM AL IH IM IL DNT
Series1 1 6 31 44 18 2 16

12. SPANISH_SPAIN_ACTFL_2007_2013_#
0 5 10 15 20 25 30 35
DNT
IL
IM
IH
AL
AM
AH
AH AM AL IH IM IL DNT
Series1 1 7 25 15 29

13. SPANISH_LAT._AMER._ACTFL_2007_2013_%
0 5 10 15 20 25 30 35 40
DNT
IL
IM
IH
AL
AM
AH
AH AM AL IH IM IL DNT
Series1 0.8 5.17 26.27 37.28 15.25 1.6 13.56

14. SPANISH_SPAIN_ACTFL_2007_2013_%
0 5 10 15 20 25 30 35 40
DNT
IL
IM
IH
AL
AM
AM AL IH IM IL DNT
Series1 1.3 9.1 32.47 19.48 37.66

15. French_ACTFL_2007_2013
AH, 3
AM, 6
AL, 21
IH, 16
IM, 12
IL
IM
IH
AL
AM
AH
0 5 10 15 20 25

16. French_ACTFL_’09_’13
AH, 3
AM, 4
AL, 19
IH, 13
IM, 7
IL
IM
IH
AL
AM
AH
0 2 4 6 8 10 12 14 16 18 20

17. French_ACTFL_’11_’13
AH, 3
AM, 2
AL, 11
IH, 10
IM, 4
IL
IM
IH
AL
AM
AH
0 2 4 6 8 10 12

18. French_ACTFL_’12_’13
AH, 3
AM, 1
AL, 7
IH, 6
IL
IM
IH
AL
AM
AH
0 1 2 3 4 5 6 7 8

19. French_ACTFL_2013
AH, 2
AM, 1
AL, 6
IH, 1
IL
IM
IH
AL
AM
AH
0 1 2 3 4 5 6 7

20. French_ACTFL_Advanced_2007_
2013
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
2013
2012_2013
2011_2013
2009_2013
2007_2013

21. Spanish_ACTFL_Advanced_2007
_2013
0% 5% 10% 15% 20% 25% 30% 35%
2013
2012_2013
2011_2013
2009_2013
2007_2013

22. What is COPI telling us about Spanish
improvement in the study abroad
context?
CIEE Conference
Baltimore, November 2014
Vera Cerqueiras, BA Spanish Coordinator

23. CONTENTS
• Describe the process of training,
implementation and correction of the exam;
• Analyze the progress made by students between
their entrance COPI and their exit COPI;
• Show some of the challenges and the
subsequent effects (washback) resulting from
this evaluation.
2

24. What is COPI (Computer based Oral Proficiency Interview)?
3
• The COPI exam is a computerized
semi-adaptive exam aimed at
determining the ability for using
oral spanish language at a
specific moment taking into
account the ACTFL Guidelines.
• It is intended for high school
students, college students, and
professionals whose native
language is English or who have
an advanced English level
• Standarized exam
• Measuring linguistic production
holistically
• Providing feedback to both the
test takers and their professors.
• Exam results can be used to show
the progress of a student after a
semester of study.
• It cannot be used to make
decisions to pass or fail Spanish
courses
Characteristics

25. Evaluation Levels
4
1. Superior: The candidate gives opinions, hypothesizes, and uses abstract language in unfamiliar
language situations
2. Advanced: The candidate narrates and describes situations with some level of complication
3. Intermediate: The candidate demonstrates creative use of the language in simple and everyday
situations
4. Novice: The candidate uses fixed expressions, short phrases and lists of memorized words
The three latter levels are subdivided into 3 sublevels:
High: The candidate does not have the ability to reach and maintain the next level
Medium: Represents the prototypical level
Low: The candidate has difficulty in meeting the level requirements

26. Administration of the Exam
The exam has the following components:
• A CD for the administration of the COPI exam
• An application for the certification of the exam
• Lab with PC`s ,Computers, headset with microphone and Speaker,
keyboards
• The candidates must select their Spanish level taking in account the
summary description of each level given by the program
• The program selects from 7 to 11 tasks
• The candidates must complete the exam by recording their answers
• Teachers / Raters should be in the computer
lab to assist.
• The exam lasts between 45/60 minutes
5

27. Interview Evaluation and Grading and Final Report of the Evaluation
of the level reached by the candidate.
6
• The program saves the
candidate’s activities.
• A double assessment is
performed by two certified
professors who correct all the
activities completed by each
candidate.
• The evaluator assigns a
corresponding level for each
answer he/she listens to, taking
into account the level assigned
by the student or the program
• The evaluator has the option to
write comments about the
activity, which the program then
saves as part of the evaluation.
• The program establishes an
overall average after the
evaluator assigns a level for each
of the activities performed by
the candidates.
• A final report with the
certification of each candidate’s
obtained level is generated by
the program including the
overall average and a description
of the proficiency

28. Exam and Certification of COPI Evaluators
• Evaluators are trained by using a kit provided by CAL
(Center of applied linguistics)
• They have to pass an exam according to ACTFL
Guidelines
• This exam consists of 15 comprehensive COPI exams
that should be rated
• Evaluators approve the exam if at least 60% (9/15)
scores coincide with those indicated by the CAL
experts.
7

29. 8

30. >197 %

31. 10

32. 11

33. Why COPI implementation represents a challenge?
12
• BECAUSE it implies working with
isolated language samples that
must be analyzed without
consideration of neither its
production context nor its
acquisition process.
• BECAUSE the evaluator's work
needs permanent training and
peer feedback in order to adjust
and perfect the administration,
correction and considerations of
the exam.
• The internalization of the practices
incorporated by giving the COPI
exam allow us to focus on the
elaboration of evaluation
instructions and on the importance
that the "knowledge of the world"
has towards the successful
achievement of that instruction.
• The progress data between entry
exams and exit exams tests the
results obtained by the very same
students in the evaluations of the
Spanish courses they take as part of
their immersion program studies.
Until now they are consistent. In
other words, those students who
advance the most are the same
students who achieve the highest
grades in their Spanish courses.
Washback Effects

34. Effects of International
Service-Learning on the
Development of Spanish Oral
Proficiency
CIEE Conference: Breaking Barriers
November 19-22, 2014, Baltimore
Francisco Salgado-Robles, Ph.D.
The City University of New York

35. Overview of presentation
① Goal
② Community Service-Learning
a) General notions
b) SL in higher education
c) SL in the Spanish classroom
d) SL at-home: Linguistic gains
e) SL abroad: Linguistic gains
③ Research Questions
④ Instrument
⑤ Participants
⑥ Blogging & Service-Learning
⑦ Results
⑧ Discussion
⑨ Conclusions
⑩ Challenges & Recommendations

36. GOAL
¡ Gain further insight into the effects of context
(i.e., service-learning in an international
setting) on the L2 acquisition process (i.e., the
development of Spanish oral proficiency)
through a case-control study analysis of
participants’ experiences enrolled in a
Language for Specific Purposes (LSP) course
(i.e., Spanish for the working environment)
during a summer semester abroad.
3

37. SERVICE-LEARNING (SL):
GENERAL NOTIONS
¡ SL? “Experiential education that engages students in
activities that address human and community needs
together with structured opportunities intentionally
designed to promote student learning and
development” (Jacoby, 1996)
¡ Students’ service experiences are tied to the academic
content of an on-campus course or curriculum (Giles,
Honnet, & Migliore, 1991)
¡ From the tenets of experiential education and
constructivist theories that advocate for learning through
first-hand discovery (Furco, 2001)
¡ “One of the most effective approaches for engaging
students and boosting learning” (Learn and Serve
America, 2011)
¡ Shape learners’ knowledge and cultural beliefs,
professional aspirations, and community engagement.
4

38. SL IN HIGHER EDUCATION
¡ Based on statistics from 2009-2010, ½ + of
community colleges and approximately ¼ of all
universities integrate SL in different academic
curricula (Learn and Serve America, 2011)
¡ SL pedagogy has been adopted not only due to
the overwhelming amount of research in support
of this type of educational experience, but also
because it has been witnessed how life-changing
these opportunities can be for their students
(Jeandron & Robinson, 2010)
5

39. SL IN THE SPANISH CLASSROOM
¡ The incorporation of SL in Spanish curricula follows
similar trends observed in other disciplines in higher
education
¡ Community engagement has been a “growing
presence in Spanish classes” and that there is
presently “significant activity and interest within our
ranks (Hellebrandt & Jorge, 2013)
¡ Rapidly expanding Hispanic population in the United
States (Rabin, 2009; Zlotkowski, 1999)
¡ Convincing evidence of the positive impact SL can
have on students taking Spanish as a second or
heritage language (Petrov, 2013)
6

40. SL AT-HOME: L2 GAINS
¡ Offer students authentic communicative contexts in
the target language
¡ Communicative competence improved from regular
interaction with NSs after participating in SL
(Barreneche, 2011; Navarro, 2012; Olazagasti-Segovia,
2004; Salgado-Robles, 2014)
¡ Facilitate linguistic improvements in the L2 (Hale, 1999;
Mullaney, 1999; Overfield, 1997)
¡ Spanish SL students scored higher on L2 reading,
speaking, and writing tasks (Malkin, 2010)
¡ Students increased their use of Spanish outside the
classroom and felt more confident speaking in Spanish
(Pellettieri, 2011)
7

41. ¡ SL as an effective way enhance student’ language
skills, knowledge of the host country, intercultural
sensitivity, and global competence (Kiely, 2011)
¡ Students gained confidence in their Spanish skills,
developed intercultural skills (Kopischke Smith &
Moreno-López, 2012)
¡ Facilitate linguistic improvements in the L2 (Cubillos,
2013)
¡ Students perceived a positive relationship between
their participation in SL projects and the
development of their linguistic and cultural
proficiency (Brown & Purmensky, 2014)
8
SL ABROAD: L2 GAINS

42. RESEARCH QUESTIONS
① What is the effect of a community-enhanced
Spanish for Specific Purpose course on the oral
proficiency at the Intermediate-level?
② What aspects of oral proficiency are most and least
influenced by the community engagement
experience?
9

43. INSTRUMENT
10

44. PARTICIPANTS (CONT’D)
Experimental group
22 students (14 f / 8 m)
19-21 yrs old
3rd year Spanish
Spanish minors/majors
Spanish for the Professions
Control group
18 students (11 f / 7 m)*
19-23 yrs old
3rd year Spanish
Spanish minors/majors
Spanish for the Professions
* 1 heritage speaker removed
11

45. PARTICIPANTS
Experimental group Control group
12

46. 13
BLOGGING & SPANISH FOR THE
WORKING ENVIRONMENT

47. BLOGGING & SPANISH FOR THE
WORKING ENVIRONMENT (CONT’D)
14

48. BLOGGING & SPANISH FOR THE
WORKING ENVIRONMENT
15

49. RESULTS (CONT’D)
ü Pre & Post Score Comparison (Experimental Group, N=22)
16

50. RESULTS (CONT’D)
ü Pre & Post Score Comparison (Control Group, N=18)
17

51. RESULTS
Post
Mean
p-value
Mastery sentence 43.10 50.00 0.0003***
Experimental Vocabulary 36.77 51.22 <0.0001***
Group Fluency 45.52 54.19 <0.0001***
(+SL) Pronunciation 46.18 57.30 0.0003***
Mastery sentence 43.94 46.04 0.0002***
Control Vocabulary 39.75 52.76 <0.0001***
Group Fluency 47.00 49.15 0.1320
(-SL) Pronunciation 44.60 48.63 0.0302
*** High Statistical Significance
Pre
Mean
ü Paired t-tests run to determine significance of trends over time
18

52. DISCUSSION
① What is the effect of a community-enhanced Spanish
for Specific Purpose course on the oral proficiency at
the Intermediate-level?
ü A statistically significant impact on the proficiency level of
Intermediate-level learners.
ü Just one participant experienced L2 gains towards the
advanced-low proficiency level in this experiment.
② What aspects of oral proficiency are most and least
influenced by the community engagement
experience?
ü SL had an observable and statistically significant impact on all
sub-skills of oral proficiency measured by the Versant Test.
19

53. CONCLUSIONS
① The incorporation of the SL component into a Spanish
for Specific Purposes course in an international setting
had a significant effect on oral proficiency;
② The learning conditions provided by the community
engagement facilitated oral skill development across
all sub-skills;
③ Overall, the SL pedagogy worked successfully for this
LSP course and, thus, should be further promoted,
since:
ü Provide more opportunities for L2 use in meaningful ways;
ü Practice different L2 skills: speaking, reading, writing, listening, cultural understanding, etc.;
ü Debrief and reflect on SL experience with others;
ü Introduction to the professional field in a hands-on way;
ü Give more confidence to students when they are in their services.
20

54. CHALLENGES &
RECOMMENDATIONS
q Challenges:
§ Time and effort: More so than the ‘normal’ course;
§ Technological problems and conquering the learning curve;
§ Lots of smaller tasks (visit report via Google Docs; blog entry
and comments; in-class led discussions); some students had a
hard time keeping up with due dates.
q Recommendations:
§ Cooperative efforts among researchers, students, instructors,
administrators, and community supervisors are plainly needed;
§ The development and dissemination of SL in LSP pedagogic
materials are warranted;
§ More large-scale studies and longitudinal assessment of L2
acquisition (both qualitative and quantitative) within the SL
(and LSP) contexts would help to further understand a new
strand of experiential learning (Kolb, 1984).
21

55. ACKNOWLEDGEMENTS
² Department of Hispanic Studies, U of Kentucky
² Service-Learning Council, U of Kentucky
² Education Abroad Office, U of Kentucky
² Spanish Studies Abroad, Seville, Spain
² Non-profit organizations
² 40 students

56. THANK YOU!

57. QUESTIONS? COMMENTS?