The document provides an overview of research methods for a final exam review. It discusses the exam format, guidelines for constructing surveys, different question types, ordering questions, self-administered questionnaires, and analyzing data. It also covers field research, agency records, secondary data analysis, content analysis, evaluation research, experimental designs, and interpreting findings. The exam will include multiple choice and short answer questions covering these topics on conducting research and analyzing data.
2. The Exam
Is worth 15% of your grade
Will include:
- 40 multiple choice questions
- 3 short answer questions
3. How to ask Questions
Open-ended
Respondent is asked to provide his or her own answer
Produces narrative data
Closed-ended
Respondent selects an answer from a list
Choices should be exhaustive and mutually exclusive
Produces data that can be readily processed
4. Guidelines for Asking Questions
Make items clear – avoid ambiguous questions; do
not ask “double-barreled” questions
Short items are best – respondents like to read and
answer a question quickly
Avoid negative items – leads to misinterpretation
Avoid biased items and terms – do not ask questions
that encourage a certain answer
Avoid terms that have bad associations
5. Questionnaire Construction
General questionnaire format – critical, must be laid
out properly
Matrix questions – same set of answer categories
used by multiple questions
Contingency questions – relevant only to some
respondents – answered only based on their previous
response
6. Ordering Questions
Ordering may affect the answers given
Estimate the effect of question order
Perhaps devise more than one version
Depending on whether it is in-person, or via mail,
internet etc.
Begin with most interesting questions
End with less interesting, demographic data
7. Self-Administered Questionnaires
Can be home-delivered
Researcher delivers questionnaire to home of respondent,
explains the study, and then comes back later
Mailed (sent and returned) survey is most common
Researchers must reduce the trouble it takes to return a
questionnaire
8. Warning Mailings and Cover Letters
Used to increase response rates
Warning mailings – “address correction requested”
card sent out to determine incorrect addresses and to
“warn” residents to expect questionnaire in mail
Cover letters – detail why survey is being conducted,
why respondent was selected, why is it important to
complete questionnaire
Follow-ups to mail surveys increase response rates
9. Survey Research
Surveys are best suited for studies that have
individuals as their units of analysis
Strengths:
Surveys tend to be high on reliability and generalizability, but
validity can often be a weak point
Surveys useful in describing characteristics of large population
10. Survey Research
Weaknesses
Standardized questionnaire items often represent the
least common denominator in assessing people‟s
attitudes, orientations, circumstances, and experiences
Surveys often appear superficial in their coverage of
complex topics
Difficult populations are hard to contact through
customary sampling methods
Generally weaker on validity and stronger on reliability
Subject to recall error
Social desirability may be a problem
11. What is Field Research?
Field research is usually associated with qualitative
data
It encompasses two different methods of obtaining
data:
Direct observation
Asking questions
Often no precisely defined hypotheses to be tested
Used to make sense out of an ongoing process
Questions related to field research tend to be „How?‟
and „Why?‟
12. Characteristics of Qualitative Research
Context is critical
In-depth and detailed
Researcher immerses self in data
Bias is recognized
Inductive rather than deductive (bottom up)
Data are interpreted rather than analyzed
13. Roles of the Observer in Field Research
Full participant (e.g., a participant in a
demonstration against stop and frisk)
Participant as observer (e.g., Perrone‟s research on
drug use in New York dance clubs)
Observer as participant (e.g., observational studies of
the police)
Complete observer (e.g., research in a courtroom
setting, or setting that is open to the public)
14. Recording Observations
Note-taking, tape recording when interviewing and
when making observations (dictation device)
Videotaping or photographs can make records of
“before” and “after” some physical design change
Field notes – observations are recorded as written
notes, often in a field journal; first take sketchy notes
and then rewrite your notes in detail
Structured observations – observers mark closed-
ended forms, which produce numeric measures
15. Strengths and Weaknesses of Field Research
Strengths:
Provides great depth of understanding
Flexibility (no need to prepare much in advance)
More appropriate to measure behavior than surveys
High on validity
Weaknesses:
Low on reliability – often very personal
Generalizability – personal nature may produce
findings that may not be replicated by another
Precise probability samples can‟t normally be drawn
16. Agency Records
Published Statistics – government organizations
routinely collect and publish compilations of data
(e.g., NCVS, Census Bureau, BJS; often available in
libraries and online
Nonpublic Agency Records – agencies produce data
not routinely released (e.g., police departments,
courthouses, correctional facilities)
New Data Collected by Agency Staff – collected for
specific research purposes; less costly than collecting
the data yourself and more control
Agency records are often used for descriptive studies
17. Problems with Reliability and Validity
Virtually all criminal justice record keeping is a social
process – “social production of data”
Records reflect decisions made by CJ personnel as well as actual
behavior by juveniles and adults
Discretion factors in keeping records
Criminal justice organizations are often more interested in
keeping track of individual cases than in examining
patterns
Potential for clerical errors due to volume of data
Users of data series collected over time must be especially
careful
Imperative that you understand how the data were collected
18. Secondary Data Analysis
Sources – websites (BJS, NCVS, ICPSR, NACJD),
libraries
Advantages
Cheaper
Faster
Benefit from work of skilled researchers
Disadvantages
Data may not be appropriate to your research question
Not useful for evaluation studies (which are designed to answer
specific questions about specific programs)
Threats to validity
19. Content Analysis
Systematic study of messages – can be applied to
virtually any form of communication
Decide on operational definitions of key variables
Decide what to watch, read, listen to & time frame of
documents
Analyze collected data
Well-suited to answer “Who says what, to whom,
why, how, and with what effect?”
20. Content Analysis
Systematic study of messages – can be applied to
virtually any form of communication
Decide on operational definitions of key variables
Decide what to watch, read, listen to & time frame of
documents
Analyze collected data
Well-suited to answer “Who says what, to whom,
why, how, and with what effect?”
Examines content and meaning
21. Content Analysis
Essentially a coding operation
Communications need to be coded according to some
conceptual framework
Choice between depth & specificity of understanding:
Manifest content – visible, surface content – similar to
using closed-ended survey questions
Latent content – underlying meaning
22. Evaluation Research
It is gaining in popularity among researchers that
really want to make a difference
Federal requirements for program evaluations often
accompany the implementation of new programs
Funding is available for program evaluations
Can utilize a number of different research designs
Links the intended actions and goals of criminal
justice policy to empirical evidence that supports
them having the desired effects
23. Conditions Requisite for Randomized Experiments
Staff must accept random assignment and agree to
minimize exceptions to randomization
Case flow must produce enough subjects in
experimental and control groups for statistical tests
Experimental interventions must be consistently
applied to experimental and withheld from control
group
Need equivalence prior to intervention, and ability to
detect differences in outcome measures after
intervention
24. The Policy Process
Begins with a demand for a new course of action or
opposition to an existing policy
Policy makers consider their ultimate goals and
means of achieving those goals
Resources are allocated (considerations include;
personnel, equipment, supplies etc.)
What are the policy outputs (i.e. what is actually
produced?)
What is the impact of the policy output?
25. Interpreting Data
Empirical research is a logical rather than a
mathematical operation
Statistics – branch of math appropriate to research
Descriptive statistics – used to summarize and
describe data in manageable forms
Inferential statistics – assist in forming conclusions
from our observations; usually about a population
based on studying a sample
26. Types of Analysis
Univariate analysis – describing single variables (e.g.
# of males; average age; place of birth etc.)
Bivariate analysis – describes the associations that
connect one variable with another
Multivariate analysis – examines relationships
among three or more variables
27. Data Analysis
Central tendency - mean, median, mode
Means are susceptible to extreme values. A few very large, or a
few very small numbers can change the mean dramatically
Because of this, it is important to examine measures of
dispersion
Simplest measure of dispersion is the range – the distance
from the highest to the lowest value (e.g. 13 to 19 years)
Standard deviation – the average amount of variability in a set
of scores (i.e. the average amount each individual observation
varies from the mean)
The larger the standard deviation, the larger the average
distance each data point is from the mean of the distribution
28. Rates
Fundamental descriptive statistics in criminal justice
research
Used to standardize some measure for comparative purposes
Total Murders in Four States, 2004
Total Murders Total Population
California 2,407 35,894,000
Florida 946 17,397,000
Louisiana 574 4,516,000
Pennsylvania 650 12,406,000
29. Short Answer Questions
1. Compare and contrast the strengths and
weaknesses of field research and survey
research. Give specific attention to the topics of
validity, reliability and generalizability
2. Discuss the advantages and disadvantages of
secondary data and give an example of each
3. Identify the difference between the various roles
of the observer in field research. Give an
example of a research scenario that would be
appropriate for each role