This document provides an overview of the scientific method and key concepts for designing experimental research studies. It discusses topics like developing research questions and hypotheses, choosing appropriate variables and experimental designs, collecting and analyzing data, and interpreting results. The document is intended to serve as an introduction to experimental methods for language researchers.

1. Scientific Method 6
1 MA English Philology
Barbara Konat
Department of Epistemology and Cognitive Sciences
Faculty of Social Sciences
2013

2. Experiment
Observation
Representativeness
Sample
Generalizations
Intuition
Knowledge
Intersubjective
Verification
Replicability

3. Article reporting original research.
Review article.
Theoretical article.

4. I - introduction
M - method
R - results
and
D - discussion

6. Abstract
– What is the paper about?
– How did the researchers set up their study?
i.e., who were the subjects, what was the
experiment?
– What did the experiment measure?
– What were the main results of the study?
(Gonzales et al. 2006)

7. Introduction
– What were the theoretical considerations underlying the
research?
– Why was the particular topic chosen for study?
– Does the chosen topic have implications beyond itself?
– What are the authors hypotheses?
– What questions do the researchers hope to answer with the
results of their study?
(Note that this is a different question than what their
hypotheses were.)
– How did the authors decide on their research strategy, i.e.
did they develop an experiment or chose to do a
correlational study?
(Gonzales et al. 2006)

8. Method
– How were the hypotheses turned into testable
questions?
– How were the variables manipulated, i.e. how
was the experiment done?
– Were appropriate controls used?
– Were the measures used appropriate to the
question being asked, i.e. is income an
acceptable measure of socio-economic status?
(Gonzales et al. 2006)

9. Results
What are the main results of the study?
– Can the results be used to answer the research
question?
– Can the results be generalized beyond the
context of the study?
(Gonzales et al. 2006)

10. Discussion
– What conclusions do the researchers draw
from their results?
– What questions were left unanswered by the
study?
(Gonzales et al. 2006)

11. Preparing your own research plan
(Gonzales et al. 2006)

12. 1. Narrowing down the topic of interest.
2. Conducting an exhaustive literature review.
3. Deciding on a question.
4. Formulating a hypothesis.
5. Developing an experiment.
6. Analyzing data.
7. Interpreting results.
Introduction
Methods
Results
Discussion
(Gonzales et al. 2006)

13. Narrowing down the topic of interest
(Gonzales et al. 2006)

14. Conducting an exhaustive literature
review
Classics + current methods
(Gonzales et al. 2006)

15. Developing a research question
What do you want to know?
The question must be:
– Precise
– Interesting
– Relevant – timely and important
– Novel
– Testable.
(Gonzales et al. 2006)

16. Types of research questions
1. Descriptive questions.
2. Causal questions.
3. Consequence questions.
4. Nondirectional relational.
(Gonzales et al. 2006)

17. (Gonzales et al. 2006)

18. Scientific methodology
„Scientific methodology encompasses
standardized methods of testing whether
an idea, translated into a hypothesis,
has explanatory value over a phenomenon
in a setting that allows falsifiability.”
(Gonzales et al. 2006: 66)
(Gonzales et al. 2006)

19. OPERATIONALIZATION

20. Methods:
Discourse analysis, Corpus Study,
Psycholinguistics experiment, Survey,
Recordings, Gesture analysis, Longitudinal
studies, Response times, Brain imaging and
many others…

21. Exercise
Work in pairs.
Prepare:
1. Research question.
2. Research hypotheses.
3. Research method.
4. Expected results.
Substantiate your choice

22. Part Two
The practical steps
(Gonzales et al. 2006)

23. 1. Narrowing down the topic of interest.
2. Conducting an exhaustive literature review.
3. Deciding on a question.
4. Formulating a hypothesis.
5. Developing an experiment.
6. Analyzing data.
7. Interpreting results.
Introduction
Methods
Results
Discussion
(Gonzales et al. 2006)

24. (Gonzales et al. 2006)

25. Variables
What is a variable?
OED on-line:
(Gonzales et al. 2006)

26. „A variable can best be qualified as a set of
events that can take on different values.
Typical ones include sex, age, scores on an
exam, number of milliseconds required to
respond to a stimulus etc. „
(Gonzales et al. 2006:67)

27. Variable classification
By their nature
• Behavioral
• Stimulus
• Subject
By their use
• Independent
• Dependent
• Extraneous
• Constant
(Gonzales et al. 2006)

28. (Gonzales et al. 2006)
Birthday party Wedding
Office meeting Funeral
Independent: Social setting (levels: Birthday party, Wedding, Office
meeting, Funeral)
Dependent: Amount of dancing
Extraneous: Alcohol consumption
Constant: Number of subjects

29. (Gonzales et al. 2006)
Birthday party Wedding
Birthday party Wedding
Independent: Social setting +Alcohol
Dependent: Amount of dancing

30. Constants
• the same number of subjects in each social
situation
• the same number of males and females
Constants are kept constant to prevent
unwanted variation.
(Gonzales et al. 2006)

31. Discrete or continous
• Discrete: the number of children in the family
(1,5 child?)
• Continous (decomposable): the number of
km you’ve ran this morning (1,7 km).
(Gonzales et al. 2006)

32. Qualitative or quantitative
• Qualitative:
– Female
– Left-handed
– Educated
• Quantitative (can occur in different amounts):
– Amount of money on your bank account
– Number of foot tappings in dance experiment
(Gonzales et al. 2006)

33. Operationalization
• Transform any concept you are interested in
into observable phenomenon that can be
measured (intersubjectivity!)
How do you measure the amount of dancing?
(Gonzales et al. 2006)
attempted rhythmic movement performed by a
person not including swaying while sitting or
leaning against a wall, or foot-tapping while sitting
or standing

34. Experimental design
• Control group
• Random assignment
– Between subject design
– Within subject design
(Gonzales et al. 2006)

35. (Gonzales et al. 2006)
Birthday party +alc Birthday party
Alone +alc Alone
Setting + amount of alcohol-> Amount of dancing
Experimental design

36. Research hypotheses
and experimental hypotheses
• Experimental hypotheses: predictions about
the type of information that can be used to
support research hypothesis
• Experimental hypotheses: hypotheses
involving the experiments themselves.
(Gonzales et al. 2006)

37. If then
If then
(Gonzales et al. 2006)
Research hypotheses
Experimental hypotheses
Operationalization

38. The Data
• Observable
• Measurable
• Can help to support or negate the research
hypothesis
(Gonzales et al. 2006)

39. The experimental and the null
hypotheses
1) that the chosen task will produce the
predicted effect
2) that it will not
(Gonzales et al. 2006)

40. Reliability and validity
Reliability:
• Test-retest
• Inter-observer.
Validity:
• Construct
• Ecological
• Internal
• External
(Gonzales et al. 2006)

41. Few last steps
• Data collection
• Obtaining results
• Results interpretation
(Gonzales et al. 2006)

42. Additional factors
• Language
• Culture
• Nationality
• Linguistic ability
(Gonzales et al. 2006)

43. Based on:
Gonzalez-Marquez, Monica, Raymond Becker,
and James Cutting. „An introduction to
experimental method for language
researchers”. In: Methods in Cognitive
Linguistics, Eds.: Monica Gonzalez-Marquez,
Seana Coulson, Michael J. Spivey, Irene
Mittelberg. John Benjamins Pub Co, 2007.