Assessing intercultural language learning, handout from Joe van Dalen's presentation at AFMLTA conference in Sydney 2009

1. ASSESSMENT OF INTERCULTURAL LEARNING
Introduction to the project:
In 2007 and 2008, I was part of a project conducted by the Research Centre for
Languages and Cultural Education at the University of South Australia, under the
leadership of Angela Scarino and Tony Liddicoat. The project was on Assessing
Intercultural Learning, funded by the Australian Research Council. I was one of
three practising teachers from the School of Languages in Adelaide involved in this
project along with about 6 other teachers from a number of primary and secondary
schools across Adelaide, representing the government, Catholic and Independent
sectors.
I wish to stress that any views and conclusions related by me today are not
necessarily those of the researchers. They will no doubt have drawn their own
conclusions. These comments are merely personal observations as a participant in
that ARC project although I will draw on some thinking and considerations from the
research team as we discussed the challenges and the concepts of assessment as a
group throughout the two phases of the investigation. It became evident very
quickly to the team and participants involved how complex this aspect of teaching
and learning in the classroom actually is as far as intercultural learning is
concerned.
The Intercultural Language Teaching and Learning in Practice (ILTLP) Project
mobilised many teachers of languages into considering the development of
intercultural programs in their classrooms, to view language learning and teaching
as a relationship between language and culture. It has required teachers and
students to consider or re-evaluate their own cultural positions with respect to the
linguistic and cultural material with which they work.
Adopting an intercultural orientation, however, has also required us to consider the
assessment of an intercultural capability on the basis that assessment and learning
are inextricably linked. Specifically, we have needed to consider what indicators of
intercultural learning might look like and how we might assess such learning. We
needed to consider what evidence to collect, and in a practical sense, what can be
1

2. collected in the classroom, in order to make any valid judgements of intercultural
learning.
Since the teachers involved in this project represented such varied levels of
teaching in the classroom, challenging questions were raised about the kinds of
intercultural learning that might take place in a year 2 or 3 class as opposed to a
middle school classroom or yet again, in a senior school classroom. Are there
different assessment procedures that would be appropriate for different cohorts?
The Process
As participants in the ILTLP project will know, everyone had first to reflect on his
or her own cultural position, to clarify cultural identity and then to conceptualise
what the intercultural meant. Broadly speaking, and to state the now fairly
obvious, participants understood intercultural teaching and learning to be
developing an awareness and understanding of linguistic and cultural diversity, of
seeing themselves and their students as both language users and as cultural beings.
The intercultural works from the premise that both language and culture affects
communication and relationships, and that an awareness of behaviour and
attitudes towards language and culture, of the multiple interpretations in
interaction that are possible, and of the nature of the resource materials we select
in the classroom, are very important considerations.
The first challenge I and others in the team experienced in this project was
something of the obvious - to decide exactly what we wanted students to learn
conceptually, and whether the planned interactions were going to elicit the sought
after evidence of intercultural learning. In other words, what kinds of formative
and summative assessment tasks were going to be useful in determining whether
intercultural learning had taken place?
It is worthwhile highlighting the diversity of views held by participants in this
project. During initial discussions in the first round, some believed that it was
simply not possible to assess intercultural learning in conventional ways, arguing
that a student’s development of the intercultural is very much tied up with the
2

3. growth of the individual, the socio-cultural development of the student, if you will.
This growth is organic, continuous, and unpredictable, manifesting at different
stages for different students. It is a development linked to the whole notion of
learning, and even learning for learning, which could only be arbitrarily “assessed”
for something like intercultural understanding.
On the other hand, given that all learning is along a developmental continuum,
even though not always predictably linear, it seems not out of the question to be
able to take a snapshot along this continuum, to examine tasks or performances
which can conceivably provide evidence of intercultural learning.
Increasingly, we now find that writers of curricula at systems levels are now
including the notion of the intercultural and, rightly or wrongly, believe that it is a
capacity that can be ticked off as a performance standard within the context of
conventional approaches to assessment tasks, much in the same way as higher
order thinking or treatment of a topic in depth might be able to be ticked off. The
tension between these two polarities, that is, the views that the intercultural is
assessable or not assessable, certainly makes this area of investigation challenging,
not the least from a practical point of view of the daily teaching routines within
the classroom.
It is not possible to give you a detailed description of the kinds of units of learning
planned by the teachers of Italian, Spanish, Chinese and French involved. However,
in the first phase of the research, there were “topics” for want of a better
description that ranged from asking year 2 students of Italian to deal with the
formalities of greetings, and what it is like to live in Rome for Year 10s. There were
excursions into the culture of the indigenous of Mexico in a Beginners Spanish class
which included adult learners, contrasting these with the treatment of the
indigenous in this country. Year 9 students of Chinese looked at schooling while the
role of women in China was examined by a class of year 11s. All the while, teachers
aimed to get students to be clear on their cultural position while examining the
culture of others. This was asking students to demonstrate a sophisticated skill.
3

4. The French teachers organised activities which included looking at the Parisian
Banlieue in a year 12 class, the Pari-roller event in the context of Youth Culture in
a year 10/11 class, and the nature of lunch in Australia contrasted with La Cantine
in a French école primaire for a group of primary year 5 and 6 group.
If I can highlight Kathy Moore’s endeavour, the primary school French colleague
with a group of year 6 primary school students who only have 50 or so minutes per
week of French, who focused on La Cantine. The purpose of the assessment was for
students to recognize:
 that there is not one way to have lunch and
 that ‘lunchtime’ means different things to different people
There was an initial discussion in English about the students’ understanding of what
a canteen is, comparing experiences from their primary school and other schools
within Australia and overseas.
Interactive tasks included:
 An initial discussion in English about their understanding of what is a
canteen, comparing experiences from Glen Osmond Primary and other
schools within Australia and overseas
 A written task in English responding to: Describe how you imagine la
cantine in a French school to be.
 Spoken responses in French of meal preferences in a French cantine.
 Written examples in French of meal preferences taken from a French school
menu
 Written self-reflections about lunchtime in France and at Glen Osmond
Primary School, after working with several French texts.
Students then had to write reflections to the following questions:
1. Imagine you are going to spend a week or so with a family in France. You
will be going to school with them. What will it be like at lunchtime?
4

5. 2. A French student is visiting Glen Osmond for a short time and will be with
your class. What do you need to tell them about lunchtime in your school in
order for them to feel comfortable?
3. Where would you prefer to spend your lunchtime? Explain your reasons.
The latter tasks clearly required students to apply the knowledge gained of the
other culture and put themselves in a position in which they had to be aware of
their own cultural practices and values. Kathy reported several instances where
some students had an “ah ha” moment, as they came to realise such marked
differences between French and Australian lunch routines, as well as pondering the
origins of word used.
I started the first round of trialling taking the perhaps naïve viewpoint that the
intercultural can be gathered principally in the traditional form of a summative
task, one I had to consider undertaking, given the time constraints at my disposal
in the course. My aim was to:
• To assess the students’ writing ability in French as a response to a written
text (a SACE required task).
• To assess the students’ understanding of the cultural context of the text to
which they must respond.
• To assess the students’ appreciation of their own cultural perspective in
writing the response.
After a series of interactive and comprehension activities, and reflecting on what
students had gained from these, the final task was for students to write a response
in French to a letter by a citizen complaining about roller-bladers riding on the
footpaths, endangering others and preventing the complainant, in particular, from
walking his dog in peace. (This complaint was authentic as it was also listed on the
Pari-roller website.) Students had to examine this complaint as an Australian
outsider, being mindful of their own cultural position and that of French
participants. This was an explicit criterion. Not all students were able to manage
the task in the way I had hoped.
5

6. The re-assessment
Now, you can see in these activities that they are practicable enough in the
classroom, of reasonable interest and authenticity to students, and forming the
basis of plenty of language learning regardless of whether or not anything
specifically of intercultural value could emerge from them. And it was precisely
this fact that was the basis of a whole raft of questions that were raised in the
evaluation of the first round of the trial.
It is a somewhat sobering experience to be grilled on why we did the things we did?
Did we elicit the kinds of evidence we were expecting? If not, why not? What did
we actually want students to learn from an intercultural perspective?
While some of my students were able to respond in articulate and interesting ways,
there were others who could not interpret the task as required, even after one-on-
one discussions. I had several students from Africa in the group who were finding
adjusting to Australian ways of learning challenging enough! A Chinese teacher who
looked at the role of women in Chinese society with a group of year 11 girls also
had some sophisticated responses from the students. However, she, like many of
us, found that she had to revisit the notion of the intercultural and ask herself
whether students could actually go beyond the obvious comparison between
cultures and sometimes sophisticated statement of opinion from a linguistic point
of view. Where was the evidence that students were able to interpret the
differences and similarities and take an intercultural position?
Many participants also found that there was a greater amount of time spent
discussing ideas in English with the students rather than in the target language in
order to get across what the students should aim to achieve. In other words, we
found that classroom time readily focused on the culture and rather than the
language, or conversely, focused on the language at the expense of the culture.
Here entered a tricky balancing act.
6

7. In evaluating what had occurred during the trial and after the first round, the
researchers were formulating the view that the traditional ways of measuring
achievement, as I had certainly adopted, were insufficient for assessing the
intercultural. The implication for us was to shift from the tasks created for the
class to emphasising what have been described as “data points”, that is, from an
emphasis from stand alone procedures, or structured activities (tests or tasks) to
the totality of formal and informal judgements made in interaction.
As participants, we were asked to submit a range of these data points, along with a
description of the interaction, what we hoped would be achieved in each
interaction, and the evidence that arose out of it. Interactions could be written
tasks in [language] or English, recordings of conversations and interactions,
formative assessment snapshots etc.
A common observation made by several participants was the following:
 The trouble is that sometimes you get an insight from a student through
spontaneous and unplanned conversation. It is difficult to formally call that
a data point for collecting evidence, at least, at a practical level in the
classroom, even though, in a sense, it is actually a data point at which some
“evidence” emerges. How to practically capture them is the challenge.
The second round
In other words, going into the second round of trials, we were looking at
assessment from a different perspective. Assessment was viewed more as
something socio-cultural, interactive and as activity rather than as a single event,
procedure, task or episode. Assessment became the developmental path and the
inquiry for both teachers and learners. And as assessment was reconceptualized,
the process of eliciting evidence, validating and judging it changed.
In the literature, assessment is described as being of three kinds, broadly speaking.
There are the inherent assessments, that is, those that happen informally and non-
verbally in all situations. I have already mentioned that many commented students
7

8. made off the cuff remarks that actually demonstrated an intercultural
understanding but which were not captured in any formal way although teachers
wished they had been able to do so. There are also the discursive assessments, that
is, those that occur when members of a social group talk about what they are doing
in an evaluative way. Lastly there are the documentary assessments that occur
when activities are evaluated according to a scheme that produces numbers and
symbols (in other words the formative and summative assessments with which we
are familiar).
To reorientate our thinking, Angela Scarino cited a quotation from Delandshere,
2002, that read:
“We are moving from an educational practice of assessment where we have
defined a priori what we are looking for, to an educational practice where we are
participating in activities in which we formulate representations to better
understand and transform the world around us. If our purpose is to understand
and support learning and knowing and to make inferences about these phenomena,
then it seems that the ideas of inquiry – open, critical, dialogic – rather than
assessment (as is currently understood) would be more helpful.”
Our second round of trialling assessment, therefore, was mindful of the idea that
one experience, whether a text, a DVD, an image and so on, is just one experience
and will never provide a complete picture. For students to decentre, that is to see
culture and language from the inside and the outside, is the more critical factor.
The data we intended to collect would need to be recordings of interactions that
happened from moment to moment (an extremely tricky procedure), surveys and
interviews and other forms of self-reporting, and also our own recordings of the
things we had observed happening in the classroom. Of course, samples of written
or spoken tasks would also be a feature.
In the assessment process, we were continuing to move forward with key questions
that were posed from the outset of this research project:
• How do we move from the cultural to the intercultural?
8

9. • What exactly do we want students to learn? What language is needed to do
so?
• Why is it important?
• What data or evidence captures the learning?
• What are the questions we need to ask students to progress to the
intercultural?
• What processes will we use to gather this? How can we connect the “data”
points?
To capture the data, the focus was more on the assessment than the content, on
how students were developing. In my own class, I certainly found it necessary to
focus more on the progress of a small group of individuals rather than the whole
class of 26 students. I have had to interrogate more vigorously the relationship
between language and culture and to interrogate the students about this
connection. For example, given the text, why is the structure before you the way it
is, why is this or that vocabulary or language used, what is not said?
Rather than focus on the end product, the focus had fallen more on the process of
thinking in getting there. We were looking more for cultural positioning from
students and asking students to respond as themselves rather than as an imaginary
person or in an imaginary situation. (Exams are full of questions that ask students
to imagine themselves to be this or that person or to be in this or that situation
rather than responding to their own feelings, values and knowledge directly.)
In the second phase of research, I had a class composed of a different mix of
students: 4 year 10 students (I boy and 3 girls), 23 year 11 students (10 males and
13 females). Of the 10 males, there were 6 Africans boys, 1 German, 1 Laotian and
2 Anglo-Australians. Of 13 females, there were 3 German girls, 2 Africans, 4 Anglo-
Australian girls and 4 Anglo-Australian adults. Given the size of the class I decided
to focus 10 of these students for collecting data and representing a cross section of
male and female students and their backgrounds.
9

10. The topic we studied was multiculturalism, broadly speaking, with some emphasis
of the prohibition of the wearing of the hijab in French schools. There were 9
major activities which included students keeping a journal which they could write
in English or French (some of the Africans found French easier), recordings made of
some interactive tasks, oral reports in French on multiculturalism, comprehension
exercises on various texts including writing personal opinions, and a final written
task symbolising a cumulative end-point. All these activities served as data points
for evidence. There were also the required moments of explicit grammar teaching
and the introduction and revision of vocabulary.
Just to highlight two activities: the first and the most revealing was one in which
students were asked to answer a series of questions about the culture they most
identify with, and then to talk to a person in the class they had not talked to share
their answers. You can imagine the intensity and amazement of Aussie students
who learnt about Kenyan or Congolese wedding ceremonies or attitudes, or adults
talking to young Burundi boys about Australian insights. And as often happens in
the classroom, I was slow enough not to realise that I had not adequately catered
for recording some of the interactions!! Nevertheless, this was a genuine
intercultural moment, so to speak, since students suddenly had insights into the
other and their own cultural headsets.
For the culminating activity, students were presented with two sets of photos and
had to choose one set, to write a response in French by interpreting these pictures
and their feelings, and try to express their cultural point of view. Before this task,
and in fact during the series of activities, I had to explicitly remind students what
we were aiming to achieve, that is, to explain again what the intercultural actually
entailed. What students were asked to do was quite challenging, abstract in some
ways, and requiring them have an adult headset. One student only actually
analysed the language written across the photo of a Muslim girl wearing the hijab.
The girls wearing the hijab in the tricolor was better understood in terms of the
cultural context.
10

11. As a general observation of interaction in the class, I found that the African
students participated less publicly in whole class discussions while the Anglo-
Australian girls were the most vocal. The class as a whole was fully engaged,
however, and many expressed an appreciation of the opportunity to deal with the
topic in the manner that we did.
On the question of multiculturalism the African and German students thought
Australia was a multicultural society while the Anglo-Australian students thought
Australia was on the way of being one but that there was still quite a bit of
intolerance.
Being involved in this project has certainly changed my view of how I teach and it
is difficult not to see every resource used now in terms of a cultural context and
the potential for some intercultural understanding. However, I do see on-going
problems with assessing intercultural learning; while curriculum designers
increasingly frame the intercultural component into formulaic assessment, I feel
uncomfortable about it being something easily accounted for in performance. The
reality is that we are dealing with a student’s development and internal state of
cognition that is not easily accounted for at a given moment. Only the most
articulate, perhaps, will be able to express their insight within a one off
assessment piece.
I find it difficult to evaluate how much my explicit teaching of the teaching of the
intercultural actually determined the responses rather than there being an
unexpected and valuable “intercultural” revelation from the students. Maybe it
does not matter. I was certainly taken aback when one of the students in a piece of
writing, which was in the form of a dialogue, had one of the characters remark:
“Oh you are an interesting intercultural person!” And so here we have a case of the
concept already having become what I call a commodity, something that is
packaged, and in that sense, potentially removed from authentic interaction.
Certainly, intercultural teaching and learning is not an easy recipe or methodology
that is a panacea for learning language and culture, and for going beyond cultural
11

12. comparisons. It requires a different kind of alertness and different kind of
questioning from the teachers to progress the intercultural learning in the
students. It will take some experience to capture the right bits of evidence to show
that education, or self-education in a broad sense, has sparked young minds into
looking at themselves and others in an exciting way.
Of course, good teachers have always had elements of the intercultural in the
classroom. Good teachers have always managed to ask the right questions to
progress their students’ learning. Through the experience of this research project, I
hope to join them.
Joe van Dalen
Acting Deputy Principal
School of Languages
255 Torrens Road
West Croydon SA 5008
Joe.vandalen@schooloflanguages.sa.edu.au
Mob: 0421 954 216
July 10, 2009
12