NA

1. Presented
by
Pramod Kumar
(Environmental Monitoring Lab)

2. Sciences and affecting factors.
New Idea, R and R.
Difference of two test results under ideal conditions should not more than 5%.
Introduction

3. History
 Aristotle: Father of Reproducibility.
 First importance of reproducibility in science by Irish
chemist Robert Boyle.
 Christiaan Huygens built his own air pump in Amsterdam,
which was a replicate of Boyle’s air pump.
Steven Shapin and Simon Schaffer (1985), book "Leviathan
and the Air-pumps” describe debate between Boyle and
Philosophers Hobbes.
Boyle’s air pump ,17th Century
www.wikipedia.org/reproducibility

4. “Closeness of agreement between results of successive
measurements of same measurand carried out under same
conditions of measurement”.
Criteria for repeatability
Same measurement procedure.
Same observer.
Short time.
Same location.
Same measuring instrument used under same
condition.
Measurement
average
Repeatability
Repeatability
http://www.raytheon.com

5. Problems associated with repeatability
Unaffordable cost of experimental procedure
Genetic differences in experimental model
Variation in experimental condition
More variable involve, more error
Restriction of using same instrument
 Biological process provide additional source of variability

6. Repeatability assessment result
Repeatability experiment of SIGMOD, 2008

7. “Scientific process relies on Reproducibility of analysis”
Aristotle,322 BC
“Reproducibility, getting bitwise identical results from run to run”
Demmel,2013
Closeness of agreement between results of measurements of same measurand
carried out under changed conditions of measurement.
Reproducibility

8.  Method of measurement
 Principle of measurement
 Observer
 Measuring instrument
 Reference standard
 Location
 Conditions of use
 Time
Criteria for Reproducibility
Appraiser 1 Measurement
Appraiser 2
Appraiser n
Reproducibility
Distribution of Appraiser Averages
http://www.raytheon.com

9. Reproducible research ?
Two basic reasons :
 First is to show evidence of correctness of your results.
 Second reason to aspire to reproducibility is to enable others to make use of our
methods and results.
Improve gage capability ?
 Repeatability: Gage maintain and Gage redesign,
 Reproducibility: Operator training ,More clearly define measurement scale
available to operator.
Stodden et al.2013

10. Repeatability V/s reproducibility
PublicationPublication
Provenance
Attribution Credit
Provenance
Attribution Credit
Context
Investigation
Study
Experiment
Context
Investigation
Study
Experiment
http://www.slideshare.net/sjcockell/reproducibility

11. http://forum.thefreedictionary.com
Reproducibility is one of main principle of the Scientific method.
Significance of repeatability and reproducibility in science
the Scientific method

12. Computer Science
Reproducible computational research is to expose more of search work
flow to our audience.
It was considered a challenge in translation from theoretical CS to
implementation of actual algorithms.
Toxicological Science
To check toxicity of a chemical waste by biological assays of
bioluminescent bacteria.
Impact assessment of a toxicant through models.

13. Medical Science
Reproducibility research is key to discoveries of new medicine .
A phase first discovery will be following by phase second reproductions
as a drug develops to commercial production.
In recent decades phase second has fallen from 28% to 18%.
A study in 2012 , found that 65% medical studies were inconsistent when
re-test ,and only 6% were completely reproducible.

14. Biological Science
 Synthesis of research results is of fundamental importance in seeking to
develop ecological and construct theories
 It is critical that published studies contain sufficient detail to allow their
methods to be replicated and their results compared.
 In response to this need, growing attention is being paid to publication and
presentation of analytical results in Ecology.

15. Digital Science :Astronomy
 In Astronomy, from observing proposals to final publications, data and
software.
 Current challenges in Astronomy is e cient exploitation of huge volumeffi
of data.
 Which is generated by experiments/ observations, computed by means of
numerical simulations.

16. Bayer:* Bayer halted in 2011 nearly two third of its early drug target project
because in house experiments failed to match claims made in literature.
Pfizer:** Pfizer wrote off $ 750 million in 2012 after results of a study with
dimebon for Alzheimer, published originally in journal lancet, could not
reproduced.
Solution:
Establish a common quality standard for biomedical research in drug
development.
WHO and ASQ published quality guidelines for biomedical research.
Pharmaceutical industry should promote these guidelines to improve R and D
productivity.
Source: *WSJ, December 2, 2011; **Fierce Biotech, January 17,2012
Impact on Industrial productivity

18. Repeatability of capillary absorption tests

19.  R and R provide validity of your experiment.
 Enhances quality in work.
 Recent years, increase in attempts to replicate the results of studies in
fields from Chemistry to Medicine .
 Discussion about repeatability and benefits to be gained from repeating
studies, with out necessarily expecting to replicate original results.
 Its have limitation in field-oriented disciplines e.g. Ecology, Archeology,
Psychology,and Social Science.

20. Clear experimental goal.
Use proper experimental methodology.
Standard statistical analysis
Enhance collaboration and open communication
Changes to regulations of funding bodies.
 Random audits .
 Give researchers a greater incentive.
Reduce unrepresentative workloads.
Measuring right object.
Future perspective

21. In 2012, a study by Begley and Ellis was published in Nature that reviewed a
decade of research. That study found that 47 out of 53 medical research paper
focused on cancer research were irreproducible.
Case study