research proposal

1. How are research proposals to
SSHRC evaluated?
A look inside the black box
of peer-adjudicated social science
Charles Davis
RTA/FCAD
Ryerson University
15 September 2005
Member, SSHRC committee 21, 2001-2004
Chair, 2002-2004

2. Basic program features
• A Standard Research Grant (SRG) is intended
to fund a 3-year research program
• Up to $250K over 3 years to individual or team
– Maximum $100k/year
• 2447 SRG proposal adjudicated
– 40.1% funded
– 28.9% of requested funds approved (~ $80M)
• The success rate of new scholars is about 10%
lower than that of established scholars

3. 21 adjudication committees (2004-5)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Classics, ancient and mediaeval studies, religious studies, classical archaeology01
History: history of science, technology and medicine02
Fine arts: history and philosophy of art, architecture, theatre, music, film, dance03
Linguistics, applied linguistics and translation05
Economics07
Sociology and demography08
Geography, urban planning and environmental studies09
Psychology10
Education 1: Arts education, bilingual education, civic education, computer assisted instruction,
counselling and career guidance, early childhood, educational psychology, environmental
education, geography, health sciences education, history, mathematics, moral, values and
religious education, pedagogy, physical education, reading and writing, science, second language,
special education and vocational education (For additional disciplines, see Committee 17) 12
Interdisciplinary and multidisciplinary studies15
Anthropology and archaeology16
Education 2: library and information science and archival science: adult, continuing and
community education; comparative education, curriculum, distance education; educational
administration, planning, and governance; history, philosophy & theory of education; higher
education, measurement and evaluation, sociology of education, teacher education (For
additional disciplines, see Committee 12)17
Literature 1: English (from the Mediaeval to the Victorian period), French; German; Slavic18
Literature 2: American, modern and contemporary literatures in English, English Canadian, First
Nations, French Canadian & Québec, romance, other languages & literatures19
Health studies and social work 20
Human resources management, information systems, international business, management;
marketing, organizational studies; business policy, industrial relations21
Accounting, finance, management science, productions and operations management22
Law, socio-legal studies and criminology23
Political science and public administration24
Philosophy25
Communication, cultural studies and women's studies26

4. golden rules of peeradjudicated grantmaking
•The process is objective. It does not matter
whether you have friends or colleagues on
selection committees.
•The process is not random. It is not a form of
lottery.
•Winning proposals are not selected.
– weaker proposals are eliminated from the
competition – the winners are those that remain.

5. The ‘rules of the game’
perpetuate the Matthew
Effect
“Unto he that hath shall be given.
From he that hath not shall be
taken away”
i.e. the funding mechanism obeys
a law of accumulated advantage

6. Scoring formula
• Regular scholar
– Record of achievement 60%
– Research Program 40%
• New scholar
– Whichever is higher:
• Record of achievement 60%, research program 40%
• Record of achievement 40%, research program 60%

7. Research achievement
• evaluation of the record of research
achievement is based primarily on
contributions to research the applicant has
made within the last six years
• if the applicant's research career has been
interrupted, research achievement is
evaluated based on his or her most recent
period of research activity.
• For regular scholars, applicant's five most
significant contributions are taken into
account in order to accurately situate the
most recent six years in the context of the
applicant's overall career.
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

8. Research achievement
Research contributions include:
• refereed publications, including books, chapters of books and
articles;
• book reviews by the applicant/co-applicant or published
reviews of his/her work;
• research reports, papers presented at scholarly meetings or
conferences, and other forms of written scholarly expression
or participation in public discourse and debate which
constitute a contribution to research;
• where appropriate, contributions to the training of future
researchers, including the supervision of graduate theses
and/or the involvement of students in research activities;
• research results from previous research grants, other awards
from SSHRC or other sources;
• academic awards and distinctions-new scholars may include
scholarships and fellowships
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

9. Research achievement
Evaluation criteria:
• quality and significance of published work (taking
into consideration the quality of the chosen
publication venues);
• originality of previous research and its impact on
the discipline or field;
• quantity of research activity relative to the stage
of the applicant's career;
• demonstrated importance of other scholarly
activities and contributions;
• recentness of output (taking into account the
nature of the applicant's career pattern and
previous non-research responsibilities);
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

10. Research achievement
Evaluation criteria:
• importance and relevance of dissemination of
research results to non-academic audiences (as
appropriate);
• significance of any previous research supported
by SSHRC or any other agency;
• where applicable, contribution to the training of
future researchers. (The committee must make
allowances for applicants who have not supervised
graduate students simply because their university
does not offer graduate programs.)
• efforts made, where appropriate, to develop
research partnerships with civil society
organizations and government departments.
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

11. Record of research achievement

12. Research program: one or
more projects over 3 years
• explicit objectives, situated within the context of
current scholarly literature;
• relationship of the proposed research to the
individual's ongoing research or to insights gained
from earlier achievements-,
• importance, originality and anticipated contribution
of the proposed research;
• theoretical approach or framework;
• research strategies or methodologies (detailed
methodology not necessary);
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

13. Research program
• plans for the communication of
research results within and beyond the
academic community
• specific roles and responsibilities of
students and research assistants,
including how their duties will
complement their academic training;
• relationship of requested budget to
proposed program of research.
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

14. Research program: evaluation
criteria
• degree of originality and nature of expected
contribution to the advancement of knowledge
• scholarly and intellectual as well as social and
cultural significance of the research
• appropriateness of the theoretical approach or
framework;
• appropriateness and expected effectiveness of the
research strategies or methodologies
• suitability and expected effectiveness of plans to
communicate research results both within and, as
appropriate, beyond the academic community
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

15. Program of research

16. Score needed for funding
Meritorious
but not
funded
cutoff zone:
currently
about 7.3 for
SRGs
funded
rejected
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

17. Achievement vs. research program
total score
9
8.5
8
7.5
achievement 6
funded
achievement 7
7
6.5
achievement 8
Meritorious
but not funded
6
5.5
5
achievement 9
rejected
5
6
7
8
9
research program
Source: SSHRC SRG Program Manual for Adjudication Committee Members, Dec. 2001

18. Some common errors
• Theoretical framework weak or insufficiently
explained
• Methodology weak or insufficiently explained
• A project is extended over 3 years to make it look
like a program
• Budget is padded or poorly formulated
• Padding of CV
• “me too” proposals
– SSHRC funded research like this last year
– Another research project in already worked-over area
• Implausible teams

19. Some common errors
• Failure to respect page limits (6 pages means
5.75-6 pages – 6.1 pages is no good)
• Include literature review or information
compilation as research
• Grad students’ roles not consistent with
research program flow and objectives
• In a resubmission, failure to take into
account the views of the committee and the
external assessors
• Ultra cartesian or ultra baconian research
designs

20. Risky storylines
• “I’m Too Important to Submit a Fully
Worked-out Research Proposal – my
record speaks for itself”
– Variant: “We’re a team of Very High
Profile Researchers. Our collective
Research Achievement is off your scale”
• “The fate of the world hangs on the
outcome of my project”

21. Risky storylines
• “My colleague got a grant last year to work on hamsters,
so I will work on hamsters also”
• “It would please God if this proposal were funded”
• “My research results will overturn all established
theories”
• “The Minister mentioned that this would make a great
research project”
• “Because of the proliferation of incommensurable
discourses in late postmodernity, you cannot understand
what I am saying and I cannot understand my
respondents, but I will study them anyway if you pay for
it”

22. A typical 3-year program
• Year 0: literature review completed;
methods and instruments selected;
preliminary hypotheses formulated
• Year 1: refine instruments and hypotheses
through qualitative research (focus groups,
grounded theory, etc.). Test instruments
• Year 2: apply instruments for data gathering
• Year 3: analysis, interpretation, modeling,
dissemination of results

23. Common winning formulas for
new scholars
• New scholar with good track record
extends doctoral research via 3-year
program
– Watch out. If the proposed research is too
close to the doctoral research, it will be
regarded as derivative. If it is too far
away, it will be regarded as too bold.
– The most successful ones have published
several articles (often with their PhD
supervisor) before applying for a first
grant

24. Common winning formulas for
new scholars
• New scholar as PI with established
scholar as co-investigator with
specified roles
– The co-investigator brings up the
“research achievement” score in
proportion to his/her role in the project

25. Budgeting tricks and traps
• the committee may reduce your
requested budget.
• It is good to ask for money for grad
student stipends – build grad students
into your program
– Note: It is best to use doctoral students.
In regional universities it is OK to use
master’s students. If you use undergrads,
make sure you have a good reason.

26. Budgeting tricks and traps
• Do NOT request conference travel money in
Year 1.
– Hint: OK to request modest funds for grad
student travel to conferences, if they present.
• Do NOT inflate travel cost estimates.
– it is permissible to include travel costs of work
with research collaborators, but not
collaborators’ research costs
• Note that research travel costs include
dissemination costs, which are also
calculated separately

27. Budgeting tricks and traps
• Do NOT request funds for computers
unless computers are clearly necessary
for the research and they are
unavailable through the university
– OK to ask for laptops for field research
• NEVER ask for funding for less than
three years

28. Budgeting tricks and traps
• Research Time Stipends are only available if the
home university contributes one-to-one matched
funding
• Do NOT request funds for books. SSHRC does not
like to pay for books.
• Be CAREFUL if you request funds for
consultancies – this is thin ice
• NOTE THAT once the budget is approved, SSHRC
says that you can spend your grant however you
like – but your University controllers do not
necessarily know this.
– At any rate, you cannot pay yourself an honorarium.

29. Budgeting tricks and traps
• See SSHRC’s list of ineligible items. It
includes: training, purchase or rental
of standard office equipment,
preparation of teaching materials,
entertainment and hospitality costs,
research leading to a degree, fees and
honoraria to colleagues, indirect or
overhead costs, etc.
• ALWAYS include a clear explanatory
budgetary note

30. Budget for hypothetical three-year, one-person small project
at a small or medium (non-doctoral) university
year 1
Master's
year 2
year 3
12000
12000
12000
3800
3800
3800
travel-applicant
Canadian
foreign
1000
2000
2000
3000
2000
travel-student
Canadian
foreign
2000
RTS
other expenses
professional
supplies
2000
equipment
4000
other equipment 1000
other
1000
total
26800
1500
10,000
1000
1000
1000
1000
31800
24300

31. Thank you!
Questions or comments?