20150128 Mayo Clinic Department of Biomedical Statistics and Informatics

1. brian m. bot | sage bionetworks | @BrianMBot
mayo clinic - 2015 jan 28
tools for reproducible research
in an increasingly digital world

2. sage bionetworks
~40 FTEs
1/2 research - 1/3 platform - 1/6 leadership/support

3. sage bionetworks
focused on a world where biomedical research
will fundamentally change to be
more open and collaborative

5. production

6. distribution

7. aggregation

13. 6%
21%
8%
11%
54%
cannot
reproduce
can
reproduce in
principle
can reproduce
w/discrepancies
can reproduce from
processed data w/
discrepancies
can
reproduce
partially
the status quo tolerates
poor communication of
findings
Ioannidis A. et al. Nature Genetics 2009

14. 208,294,724
datapoints
124 pages
supplemental material
?? lines
unobtainable source code
?? version or architecture of
statistical analysis program (R)
enumerable R packages
and package dependencies
key R package “ClaNC”
no longer available
1231 citations
often what is in principle
reproducible, is not
practically reproducible
unidentified publication
‣ from journal with 5 year impact factor of 27
‣ article freely available for download
‣ data freely available for download

15. “Scientists often study the past as
obsessively as historians because few
other professions depend so acutely on it.
Every experiment is a conversation with
a prior experiment,
every new theory a refutation of the old”
-Siddhartha Mukherjee, The Emperor of All Maladies

16. scientific method
1. define a question
2. gather information and resources (background research)
3. form a hypothesis
8. retest (frequently done by other scientists)
4. test hypothesis experimentally
5. analyze experimental data
7. publish results
6. draw conclusions based on data

17. 7. publish results

18. finitein
∞
...

19. conducting research for others to consume
(even if the ‘other’ is future you)
reproducible research

20. tools for reproducible research
code
data
analysis

21. tools for reproducible research
code
version control

22. tools for reproducible research
code
version control
client-server
distributed

23. tools for reproducible research
code
version control
client-server
(e.g svn, cvs)

24. tools for reproducible research
code
version control
client-server
distributed

25. tools for reproducible research
code
version control
(e.g git, mercurial)
distributed

26. tools for reproducible research
code
version control
distributed

27. tools for reproducible research
code
data
analysis

28. tools for reproducible research
data
generic
domain
repositories
results

29. tools for reproducible research
data
digital object identifier (doi)
a unique identifier which remains fixed over the
lifetime of a web-accessible object
metadata, including the object’s location, is stored in
association with the doi and may change over time
referring to an online document by its doi provides
more stable linking than simply referring to a url

30. tools for reproducible research
code
data
analysis

31. tools for reproducible research
analysis
R Sweave knitr
great if you know LaTeX

32. tools for reproducible research
analysis
R Sweave knitr
great if you are lazy
(like me)

33. tools for reproducible research
analysis
knitr
# Hello World Title
### Author: Brian M. Bot
This is a narrative with inline
code execution to tell me that pi
is equal to `r pi`. And a plot to
show a simple function.
```{r}
x <- 1:100
y <- log(x)/x
plot(x,y)
```

34. tools for reproducible research
analysis
knitr
# Hello World Title
### Author: Brian M. Bot
This is a narrative with inline
code execution to tell me that pi
is equal to `r pi`. And a plot to
show a simple function.
```{r}
x <- 1:100
y <- log(x)/x
plot(x,y)
```

35. tools for reproducible research
analysis
ipython
notebook

36. tools for reproducible research
other tools
galaxy
docker
packrat
shiny

37. tools for reproducible research
other tools
enables sharing of all resources (data, code, results)
and their relationships to one another

38. tools for reproducible research

39. tools for reproducible research

40. tools for reproducible research

41. Go Hawks!

42. mayo clinic - 2015 jan 28
in an increasingly digital world
brian m. bot
——————
brian.bot@sagebase.org
@BrianMBot
sage bionetworks
tools for reproducible research