1. Can we use effectively use
predictors of success to allocate
funding?
A study on the correlation between GRE scores and other factors of
success in engineering
Presented by:
Melissa Lee
Department of Civil Engineering

2. Presentation outline
• Introduction
• Literature review
• Methodology, results and discussion
• Conclusions

3. Introduction
• Limited funding opportunities for students
• Funding in the form of assistantships and
fellowships helps attract the best potential
students
• How do we recruit and identify students?
• How do we identify those most likely to succeed
through their graduate school applications?

4. Introduction
• Helps us better effectively manage enrolment
and offers of funding
• How do we know what we know about graduate
school admission?
• What do we know about how we admit students,
our admission profile, and funding profile?

5. Existing Literature
Little to No
correlation
Very accurate
predictor
Valid but with
other factors
• Cooksey and
Stenning (1981)
• Morrison and
Morrison (1995)
• Powers (2004) found
that there is some
validity, though it is
clearly accompanied
by other
corresponding factors
• Holt, Bleckmann et
al. (2006), found that
GRE scores were a
fairly accurate
predictor of graduate
school success
• Nathan Kuncel
Cooksey, L., & Stenning, W. F. (1981). The Empirical Impact of the Graduate Record Examination and Grade Point Average on
Entry and Success in Graduate School at Texas A&M University. ERIC.
Holt, D. T., Bleckmann, C. A., & Zitzmann, C. C. (2006). The Graduate Record Examination and Success in an Engineering
Management Program: A Case Study. Engineering Management Journal, 18(1), 10-16.
Morrison, T., & Morrison, M. (1995). A Meta-Analytic Assessment of the Predictive Validity of the Quantitative and Verbal
Components of the Graduate Record Examination with Graduate Grade Point Average Representing the Criterion of Graduate
Success. Educational and Psychological Measurement, 55(2), 309-316. doi: 10.1177/0013164495055002015
Powers, D. E. (2004). Validity of Graduate Record Examinations (GRE) General Test Scores for Admissions to Colleges of
Veterinary Medicine. Journal of Applied Psychology, 89(2), 208-219.

6. Existing literature
• Orlando (2005)
– Reviewed a range of studies between 1977 and 2001
– Different findings regarding validity
– Some correlation, though there is disagreement to the extent of
correlation
– Difference in disciplines, skills demanded, type of admission,
age
Orlando, J. (2005). The reliability of GRE scores in predicting graduate school success: a meta-analytic, cross-functional,
regressive, unilateral, post-kantian, hyper-empirical, quadruple blind, verbiage-intensive and hemorrhoid-inducing study.
Ubiquity, 2005, 1-1. doi: 10.1145/1071916.1071921

7. Existing Literature
• Nathan Kuncel – extensive research on GRE scores as predictor of
success
– Meta-analysis of different factors influencing graduate school success
(2001)
– Synthesized studies done on all standardized tests (2007)
– Another meta-analytic study on GRE scores for Master’s and Doctoral
programs (2010)
• Existing studies discussed the validity of GRE scores, but what can
we do with this knowledge?
Kuncel, N. R., & Hezlett, S. A. (2007). Standardized Tests Predict Graduate Students' Success. Science, 315(5815),
1080-1081. doi: 10.1126/science.1136618
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the
Graduate Record Examinations: Implications for graduate student selection and performance. Psychological Bulletin,
127(1), 162-181. doi: http://dx.doi.org/10.1037/0033-2909.127.1.162
Kuncel, N. R., Wee, S., Serafin, L., & Hezlett, S. A. (2010). The Validity of the Graduate Record Examination for Master’s
and Doctoral Programs: A Meta-Analytic Investigation. Educational and Psychological Measurement, 70(2), 340-352. doi:
10.1177/0013164409344508

8. Questions
• Is there a relationship between GRE scores and GPA?
Are GRE scores an accurate predictor of success?
• If GRE scores could predict success, how do we use this
knowledge to allocate funding?

9. Can GRE scores be used as an
accurate determinant of success in
graduate school?

10. Profile
• Civil engineering graduate students at a
large, research university
• Enrolled each Fall from 2003-2013
• Enrolments at three different degree
programs, Master of Engineering, Master
of Science, and Doctoral program

11. Citizenship status
71
28
Citizenship status (% of total enrolment)
International
Domestic

12. Degree program
24
31
43 MEN
MS
PhD
Percentage of students enrolled in each degree program

13. Is there a correlation between GRE
scores and graduate school GPA?
• GRE scores and graduate school GPA were
taken from the Fall enrolments between 2003-
2013
• Total of 1,083 students sampled over this period
• Multiple regression was applied across
enrollments during the 10 year period
• GRE scores as the independent variable
• Graduate school GPA as dependent variable

14. Correlation between GRE scores
and GPA
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Correlati
on 0.379 0.108 0.114 0.246 0.403 0.182 0.196 0.190 0.262 0.221 0.209
Coeffici
ent 0.143 0.012 0.013 0.061 0.162 0.033 0.038 0.036 0.069 0.049 0.044
Significa
nce -F 0.0065 0.732 0.634 0.099 0.0002 0.131 0.102 0.129 0.033 0.069 0.043
Sample
size 68 56 72 77 100 123 120 114 99 110 144

15. Correlation between GRE scores
and GPA (international students)
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Correlation 0.144 0.216 0.257 0.611 0.117 0.156 0.147 0.270 0.243 0.288
Coefficient 0.021 0.047 0.066 0.374 0.013 0.024 0.022 0.073 0.059 0.083
Significanc
e -F 0.632 0.376 0.201 0.003 0.562 0.540 0.485 0.130 0.177 0.051
Sample size 47 44 50 28 86 53 69 57 60 72

16. Correlation between GRE scores
and GPA (domestic)
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Correlation 0.174 0.083 0.347 0.611 0.343 0.156 0.174 0.317 0.256 0.166
Coefficient 0.030 0.007 0.120 0.374 0.118 0.024 0.030 0.100 0.065 0.028
Significance
-F 0.912 0.918 0.215 0.003 0.119 0.540 0.522 0.127 0.204 0.495
Sample size 9 28 27 28 37 53 45 42 50 53

17. Is there a correlation between GRE
scores and graduate school GPA?
• Positive correlation, but fairly weak
• Corresponds to earlier studies between
GRE scores and graduate school GPA
• Kuncel’s studies support that GRE scores
have some validity with CGPA
Kuncel, N. R., & Hezlett, S. A. (2007). Standardized Tests Predict Graduate Students' Success. Science, 315(5815),
1080-1081. doi: 10.1126/science.1136618
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the
Graduate Record Examinations: Implications for graduate student selection and performance. Psychological Bulletin,
127(1), 162-181. doi: http://dx.doi.org/10.1037/0033-2909.127.1.162
Kuncel, N. R., Wee, S., Serafin, L., & Hezlett, S. A. (2010). The Validity of the Graduate Record Examination for Master’s
and Doctoral Programs: A Meta-Analytic Investigation. Educational and Psychological Measurement, 70(2), 340-352. doi:
10.1177/0013164409344508

18. Can GRE scores therefore be used
to help allocate funding?

19. 1.Can GRE scores be used as an accurate
determinant for success in graduate
school?
2.Can we identify and allocate funding to
those most likely to succeed?

20. Questions
1. Are we using GRE scores in the selection of our
graduate assistants?
2. How does our graduate assistant scores match
up against those who don’t have assistantships?
3. Are we directing funding to the most successful
students? (how does their GPA look against
others?)
4. If we are not awarding assistantships to those
who are not most likely to succeed based off
GPA, then is the assistantship helping them or
hurting them?

21. Does having an assistantship
make a difference?
• Assistantship data from 2009 to 2013 is
used for this determination
• Students offered either a teaching
assistantship or graduate assistantship in
their first year of enrolment in the program
• GRE scores and Graduate school GPA
• Undergraduate GPA and Graduate school
GPA

22. Assistantships - Correlation
between GRE scores and GPA
• Total of 96 students were sampled
between 2009-2013
• Only students who were offered
assistantships in their first semester
• Correlation of 0.141. Positive, but a weak
correlation between GRE scores and the
GPAs of those given assistantships

23. Relationship between U-GPA and
G-GPA
• Correlation of 0.04
• Extremely weak correlation between
undergraduate GPA and graduate GPA for
those who are on assistantships
• How else would we determine who is best
suited to be teaching and research
assistants?

24. Questions
1. Are we using GRE scores in the selection of our
graduate assistants?
2. How does our graduate assistant scores
match up against those who don’t have
assistantships?
3. Are we directing funding to the most successful
students? (how does their GPA look against
others?)
4. If we are not awarding assistantships to those
who are not most likely to succeed based off
GPA, then is the assistantship helping them or
hurting them?

25. Average GRE Verbal scores for
those on Assistantships
• For each of the years analyzed, it was
found that:
Year Median GRE
- Verbal
Total number
of GAs
Above Below
2009 470 32 13 17
2010 480 20 12 6
2011 490 24 12 11
2012 530 10 4 5
2013 490 12 6 4

26. Average GRE Quantitative scores for
those on Assistantships
• For each of the years analyzed, it was
found that:
Year Median GRE
- Quantitative
Total number
of GAs
Above Below
2009 750 32 17 11
2010 760 20 8 8
2011 760 24 12 11
2012 780 10 4 5
2013 770 12 6 4

27. Questions
1. Are we using GRE scores in the selection of our
graduate assistants?
2. How does our graduate assistant scores match
up against those who don’t have assistantships?
3. Are we directing funding to the most
successful students? (how does their GPA
look against others?)
4. If we are not awarding assistantships to those
who are not most likely to succeed based off
GPA, then is the assistantship helping them or
hurting them?

28. Average GPA
Average GPA With assistantships No assistantships
MEN 3.69 3.58
MS 3.7 3.7
PhD 3.75 3.68

29. Questions
1. Are we using GRE scores in the selection of our
graduate assistants?
2. How does our graduate assistant scores match
up against those who don’t have assistantships?
3. Are we directing funding to the most successful
students? (how does their GPA look against
others?)
4. If we are not awarding assistantships to those
who are not most likely to succeed based off
GPA, then is the assistantship helping them
or hurting them?

30. Average time to degree
(semesters)
Average time to
degree With Assistantships No Assistantships
MEN 4.5 4.37
MS 6.58 6.05
PhD 11.36 11.82

31. Conclusions
• We cannot rely on GRE alone in predicting
future success in graduate school
(determined by graduate school GPA and
time to degree)
• Students on assistantships tend to do
better in terms of overall grades and time
taken to complete their degrees

32. Further research
• More study is required on how we admit students and
distribute funding to students
• Bigger sample size will help give a more accurate picture of
how and why students are admitted
• Indicators of success for graduate students
• Can we create a matrix of sorts to help better allocate funding?
• Best practices in admission and funding
• Eliman (1991) and Suhayda, Hicks and Fogg (2008) have also explored
this possibility in other disciplines
• Is graduate school admission an art or a science?
• Considering the different aspects of evaluating applicants,
particularly the non-measurable factors, can we reduce each and
every applicant in graduate school to a data point?
Eliman, A. A. (1991). A decision support system for Univeristy admission policies. European Journal of
Operational Research, 50(2), 140-156. doi: http://dx.doi.org/10.1016/0377-2217(91)90237-P
Suhayda, R., Hicks, F., & Fogg, L. (2008). A Decision Algorithm for Admitting Students to Advanced Practice
Programs in Nursing. Journal of Professional Nursing, 24(5), 281-284. doi:
http://dx.doi.org/10.1016/j.profnurs.2007.10.002

33. References
Attiyeh, G., & Attiyeh, R. (1997). Testing for Bias in Graduate School Admissions. Journal of Human Resources, 32(3), 524-548.
Cieboter, F. J. (1969). Factors Related to the Performance of Foreign Graduate Students. The Journal of Educational Research, 62(8),
360-365. doi: 10.2307/27532232
Cooksey, L., & Stenning, W. F. (1981). The Empirical Impact of the Graduate Record Examination and Grade Point Average on Entry
and Success in Graduate School at Texas A&M University. ERIC.
Eliman, A. A. (1991). A decision support system for Univeristy admission policies. European Journal of Operational Research, 50(2),
140-156. doi: http://dx.doi.org/10.1016/0377-2217(91)90237-P
Feeley, T. H., Williams, V., & Wise, T. (2005). Testing the Predictive Validity of the GRE Exam on Communication Graduate Student
Success: A Case Study at University at Buffalo. Communication Quarterly, 53(2), 229-245. doi: 10.1080/01463370500090209
Freeman, R. B. (2005). Fellowship Stipend Support and the Supply of Science and Engineering Students: NSF Graduate Research
Fellowships. American Economic Review, 95(2), 61-65.
Holt, D. T., Bleckmann, C. A., & Zitzmann, C. C. (2006). The Graduate Record Examination and Success in an Engineering
Management Program: A Case Study. Engineering Management Journal, 18(1), 10-16.
House, J. D., & Johnson, J. J. (1993). Graduate Record Examination Scores and Academic Background Variables as Predictors of
Graduate Degree Completion. Educational and Psychological Measurement, 53(2), 551-556. doi: 10.1177/0013164493053002025
Ingram, R. E. (1983). The GRE in the graduate admissions process: Is how it is used justified by the evidence of its validity?
Professional Psychology: Research and Practice, 14(6), 711-714. doi: http://dx.doi.org/10.1037/0735-7028.14.6.711
Kuncel, N. R., & Hezlett, S. A. (2007). Standardized Tests Predict Graduate Students' Success. Science, 315(5815), 1080-1081. doi:
10.1126/science.1136618
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the Graduate Record
Examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127(1), 162-181. doi:
http://dx.doi.org/10.1037/0033-2909.127.1.162
Marston, A. R. (1971). It is time to reconsider the Graduate Record Examination. American Psychologist, 26(7), 653-655. doi:
http://dx.doi.org/10.1037/h0032124
Morrison, T., & Morrison, M. (1995). A Meta-Analytic Assessment of the Predictive Validity of the Quantitative and Verbal Components
of the Graduate Record Examination with Graduate Grade Point Average Representing the Criterion of Graduate Success. Educational
and Psychological Measurement, 55(2), 309-316. doi: 10.1177/0013164495055002015

34. References
Orlando, J. (2005). The reliability of GRE scores in predicting graduate school success: a meta-analytic,
cross-functional, regressive, unilateral, post-kantian, hyper-empirical, quadruple blind, verbiage-intensive and
hemorrhoid-inducing study. Ubiquity, 2005, 1-1. doi: 10.1145/1071916.1071921
Powers, D. E. (2004). Validity of Graduate Record Examinations (GRE) General Test Scores for Admissions
to Colleges of Veterinary Medicine. Journal of Applied Psychology, 89(2), 208-219.
Robertson, M., & Nielsen, W. (1961). The graduate record examination and selection of graduate students.
American Psychologist, 16(10), 648-650. doi: http://dx.doi.org/10.1037/h0043209
Rosser, M. (1970). Criteria used for admission by graduate departments. American Psychologist, 25(6), 558-
560. doi: http://dx.doi.org/10.1037/h0037862
Rubio, D. M., Rubin, R. S., & Brennan, D. G. (2003). How Well Does the GRE Work for Your University? An
Empirical Institutional Case Study of the Graduate Record Examination across Multiple Disciplines. College
and University, 78(4), 11-17.
Saaty, T. L., France, J. W., & Valentine, K. R. (1991). Modeling the graduate business school admissions
process. Socio-Economic Planning Sciences, 25(2), 155-162. doi: http://dx.doi.org/10.1016/0038-
0121(91)90013-H
Sternberg, R. J., & Williams, W. M. (1997). Does the Graduate Record Examination predict meaningful
success in the graduate training of psychology? A case study. American Psychologist, 52(6), 630-641. doi:
http://dx.doi.org/10.1037/0003-066X.52.6.630
Suhayda, R., Hicks, F., & Fogg, L. (2008). A Decision Algorithm for Admitting Students to Advanced Practice
Programs in Nursing. Journal of Professional Nursing, 24(5), 281-284. doi:
http://dx.doi.org/10.1016/j.profnurs.2007.10.002
Thornell, J. G., & Mccoy, A. (1985). The Predictive Validity of the Graduate Record Examinations for
Subgroups of Students in Different Academic Disciplines. Educational and Psychological Measurement, 45(2),
415-419. doi: 10.1177/001316448504500229

35. Thank you!