Slides from my talk in the FMRIB group meeting at Oxford University.

1. Practical Normative Modeling:
Bridging the Technical-Clinical Gap
Saige Rutherford
FMRIB Group Meeting, Oxford University
March 15th, 2022
@being_saige

2. Chandra Sripada, Mike Angstadt, Daniel Kessler, Liza Levina,
Ivy Tso, Stephan Taylor, Jenna Wiens, Pascal Sturmfels
PhD supervisors: Andre Marquand, Eric Ruhé, & Christian
Beckmann. Lab members: Seyed Mostafa Kia, Thomas
Wolfers, Mariam Zabihi, Charlotte Fraza, Richard Dinga
Acknowledgements
Donders Institute, Nijmegen UMich, Ann Arbor
@being_saige
Charlie-Mop

3. Road Map
• Brief introduction + prior work
• Contextualize the vision for my Ph.D. work
• Practical considerations for normative modeling
• Dataset (eLife) paper
• Tutorial (Nature Protocols) paper
• In the works
• Evidence for embracing normative modeling

4. Background
@being_saige
www.beingsaige.com

5. Background
Detroit, MI
Ann Arbor, MI
Nijmegen, NL

6. • Methods Core to support clinical non-technical trained scientists.
• Realization that researchers in psychiatry have very different perspectives and questions.
• However, most need a common set of tools for curating data and performing analysis.

7. Visual processing and social cognition in
schizophrenia, psychosis, bipolar disorder
Mindfulness training for post
traumatic stress disorder
Speech disorders (stuttering) in children
Childhood obsessive compulsive disorder
Substance use disorder
Social anxiety disorder

8. Open science is not necessarily accessible science

9. Prior Work @ UMich
Rutherford, S., Sturmfels, P., Angstadt, M. et al. Automated Brain Masking of Fetal Functional MRI with Open Data. Neuroinformatics (2021). https://doi.org/10.1007/s12021-021-09528-5
Sturmfels, P., Rutherford, S., Angstadt, M., Peterson, M., Sripada, C., Wiens, J.. (2018). A Domain Guided CNN Architecture for Predicting Age from Structural Brain Images.
Proceedings of the 3rd Machine Learning for Healthcare Conference. Proceedings of Machine Learning Research https://proceedings.mlr.press/v85/sturmfels18a.html.
Aman Taxali, Mike Angstadt, Saige Rutherford, Chandra Sripada, Boost in Test–Retest Reliability in Resting State fMRI with Predictive Modeling, Cerebral Cortex, (2021).
https://doi.org/10.1093/cercor/bhaa390
Sripada, C., Angstadt, M., Rutherford, S. et al. Basic Units of Inter-Individual Variation in Resting State Connectomes. Sci Rep 9, 1900 (2019). https://doi.org/10.1038/s41598-018-38406-5
Sripada, C., Rutherford, S., Angstadt, M. et al. Prediction of neurocognition in youth from resting state fMRI. Mol Psychiatry 25, 3413–3421 (2020). https://doi.org/10.1038/s41380-019-0481-6
Sripada, C., Angstadt, M., Taxali, A. et al. Brain-wide functional connectivity patterns support general cognitive ability and mediate effects of socioeconomic status in youth.
Transl Psychiatry 11, 571 (2021). https://doi.org/10.1038/s41398-021-01704-0
Principal Component Regression + Psychometrics
Test-Retest Reliability of fMRI Predictive Modeling: CIFTI vs NIFTI, Scan Length, Model Choice
Tackling Spatial Invariance of CNNs using Normalized Brain Images
Fetal fMRI U-Net CNN Brain Masking w/ Open Training Dataset

10. Prior Work @ UMich
Principal Component Regression + Psychometrics
Test-Retest Reliability of fMRI Predictive Modeling
Tackling Spatial Invariance of CNNs using Normalized Brain Images
Fetal fMRI U-Net CNN Brain Masking w/ Open Training Dataset
Cool technical progress,
but where are the clinical results?

11. Controls
Group Average
Patients
Group Average
Controls vs. Patients
vs. =
Where are the clinical results?

13. Bridging the Technical-Clinical Gap

14. N= 60,000
N= 60

15. https://elifesciences.org/articles/72904 https://www.biorxiv.org/content/10.1101/2021.08.08.455583v1
Tutorial Paper
Dataset Paper
Chapter 1: Practical Considerations for Normative Modeling

16. https://pcntoolkit.readthedocs.io/en/latest/index.html

17. Normative Modeling Overview

19. A hurdle to the widespread application of normative modeling is …
a lack of well-defined reference models to quantify variability across the lifespan and to compare
results from different studies.
Such models should:
(1) Accurately model population variation across large samples
(2) Be derived from widely accessible measures
(3) Provide the ability to be updated as additional data come on-line
(4) Be supported by easy-to-use software tools
(5) Should quantify brain development and ageing at a high spatial resolution, so that different patterns of
atypicality can be used to stratify cohorts and predict clinical outcomes with maximum spatial precision.

20. 58,836 individuals across 82 scan sites covering the
human lifespan (aged 2-100).
Normative models for cortical thickness and
subcortical volumes derived from Freesurfer (v6).
N
(subjects)
N
(sites)
Sex
(%F/%M)
Age
(Mean, s.d)
Full All 58,836 82
Training set 29,418 82 51.1/48.9 46.9, 24.4
Test set 29,418 82 50.9/49.1 46.9, 24.4
mQC All 24,354 59
Training set 12,177 59 50.2/49.8 30.2, 24.1
Test set 12,177 59 50.4/49.4 30.1, 24.2
Clinical Test set 1,985 24 38.9/61.1 30.5, 14.1
Transfer Test set 546 6 44.5/55.5 24.8, 13.7
https://github.com/predictive-clinical-neuroscience/braincharts

24. Generalization:
across
sites

25. Generalization:
resampling the full
controls test set

26. https://brainviz-app.herokuapp.com/

30. Evidence for embracing normative modeling:
SVM Classification (HC vs. SZ) 5-fold cross validation
https://pcntoolkit.readthedocs.io/en/latest/pages/post_hoc_analysis.html
AUC = 0.50 +/- 0.15
AUC = 0.72 +/- 0.07
SchizConnect.org dataset (not included in eLife paper  can be considered another transfer dataset. HC = 263 , SZ = 261

31. https://pcntoolkit.readthedocs.io/en/latest/pages/other_predictive_models.html
HCP-Young Adult Sample: 946 subjects with structural MRI data and all behavioral variables needed to create g-factor
Use deviation (Z) scores and true cortical thickness data as the input features to predict g-factor
Evidence for embracing normative modeling:
Regression: predicting general cognitive ability (g-factor)
Method Deviation scores as
input features (MSE)
Cortical Thickness as
input features (MSE)
Principal Component Regression 0.741 0.830
Connectome Predictive Modeling 0.697 0.701
Lasso (Linear regression + L1 regularization) 0.682 0.697
Ridge (Linear regression + L2 regularization) 0.734 0.692
Elastic Net (Linear regression + L1/L2 regularization) 0.680 0.692

32. Deviations scores
Cortical thickness
Dark Blue = FDR corrected p-value < 0.05
Yellow = non-significant
left hemi right hemi
Evidence for embracing normative modeling:
Classical case-control testing (mass univariate t-tests) controls vs. schizophrenia
https://pcntoolkit.readthedocs.io/en/latest/pages/post_hoc_analysis.html

33. Chandra Sripada, Mike Angstadt, Daniel Kessler, Liza Levina,
Ivy Tso, Stephan Taylor, Jenna Wiens, Pascal Sturmfels
PhD supervisors: Andre Marquand, Eric Ruhé, & Christian
Beckmann. Lab members: Seyed Mostafa Kia, Thomas
Wolfers, Mariam Zabihi, Charlotte Fraza, Richard Dinga
Acknowledgements
Donders Institute, Nijmegen UMich, Ann Arbor
@being_saige
Charlie-Mop