Choosing the correct statistical test for data analysis

1. When to use,
What Statistical Test for data Analysis
Prof. Asokan R
I/C of R&D, KINS, KIIT (DU), BBSR.

2. Outline
1.Some key Terminology
2.Types of variables
3.Hypothesis Testing
4.Statistical Assumptions
5.Parametric tests
6.Non-Parametric tests
7.Parametric test Flowchart

3. Introduction

4. What is Statistics?

5. • "Statistics is the practice or science of collecting and analyzing numerical
data in large quantities, especially to infer proportions in a whole from
those in a representative sample.“
- Oxford Dictionary
• Statistics are used to interpret data and solve complicated real-world
issues.
• Statistics can be used to carry out mathematical computations to extract
useful insights from data.

6. What is the purpose of statistical tests?

7. Clinical studies often compare the efficacy of a new preparation in a
study group with the efficacy of an established preparation, or a placebo, in a
control group.
Aside from a pure description, we would like to know whether the
observed differences between the treatment groups are just random or
are really present.
This is because differences can be due to chance variability (scatter) in
a parameter, such as the success of the treatment in the study group.

8. What are the steps in a statistical test?

9. ● Statement of the question to be answered by the study
● Formulation of the null and alternative hypotheses
● Decision for a suitable statistical test
● Specification of the level of significance (for example, 0.05)
● Performance of the statistical test analysis: calculation of the p-value
● Statistical decision: for example – p

10. • The decision for a statistical test is based on the scientific question to be answered,
the data structure, and the study design.
• Before the data are recorded and the statistical test is selected, the question to be
answered and the null hypothesis must be formulated.
• The test and the level of significance must be specified in the study protocol before
the study is performed. It must be decided whether the test should be one-tailed or
two-tailed.
• If the test is two-tailed, this means that the direction of the expected difference is
unclear. (The new drug might even work less well than the placebo).
• A one-tailed test should only be performed when there is clear evidence that the
intervention should only act in one direction.

12. Survival time analysis
• If the point of interest is not the endpoint itself, but the time till it is reached,
survival time analysis is the most suitable procedure.
• This compares two or more groups with respect to the time when an endpoint is
reached (within the period of observation).
• One example is the comparison of the survival time of two groups of cancer
patients given different therapies. The endpoint here is death, although it could
just as well be the occurrence of metastases.
• In contrast to the previous tests, it almost never happens that all subjects reach
the endpoint in survival time analysis, as the period of observation is limited.
• For this reason, the data are also described as (right) censored, as it is still unclear
when all subjects will reach the endpoint when the study ends. The log rank test is
the usual statistical test.

13. Correlation analysis
• Correlation analysis examines the strength of the correlation between two test
variables, for example, the strength of the correlation between the body weight of a
neonate and its body length.
• The parametric variant (Pearson correlation coefficient) exclusively tests for a
linear correlation between continuous parameters.
• On the other hand, the non-parametric variant (the Spearman correlation
coefficient) solely tests for monotonous relationships for at least ordinally scaled
parameters.
• Correlation coefficients measure the strength of association and can have values
between –1 and +1.
• The closer they are to 1, the stronger is the association.
• The null hypothesis to be tested is then that there is no linear (or monotonous)
correlation.

14. You should plan your statistical approach at the start of your
project, before you collect any data.
Different statistical tests have different requirements and planning in
advance has various benefits:
• Knowing the statistical approach will allow you to plan the way you
collect your data.
• You will save time because you’ll only collect relevant data.
• You will save effort.

15. • If you simply collect data and then look for a way to carry out an
analysis you may find that you do not have quite what you need to
answer your research question.
• Knowing what type of project you have and what sort of data you
will collect can be useful in determining the best analytical
approach.

16. • In terms of selecting a statistical test, the most important question
is "what is the main study hypothesis?".
• The correct statistical test to use not only depends on your study
design, but also the characteristics of your data.
• This will be a result of your research questions/hypotheses you are
trying to answer.

17. When choosing the correct test, ask yourself the following
questions:
What kind of data have you collected?
What is your goal?

18. Types of Variables

19. DEPENDENT AND INDEPENDENT VARIABLES
An independent variable often called “predictor variable”, is a variable
that is being manipulated in order to observe the effect on a dependent variable,
sometimes called an outcome/output variable.
Independent variable(s)-> Predictor variable(s) (or factor, or grouping) -
These are the variables (factors) that affect the response variable(s).
Dependent variable(s) -> Outcome/Output variable(s)/ Response - These are
the “things” that are affected by the other variables and/or experimental situation.

20. TYPES OF VARIABLES
• It is important to distinguish the difference between the type of variables
because this plays a key role in determining the correct type of statistical test
to adopt.
There are two main categories:
• QUANTITATIVE
• CATEGORICAL

22. QUANTITATIVE: express the amounts of things or represents thus a measure and is
numerical.(e.g. the number of cigarettes in a pack, height, weight).
The two different types of quantitative variables are:
CONTINOUS ( Ratio): which the values are not countable and have an infinite
number of possibilities.
For example: Age, Weight, Height
DISCRETE (Interval): which the values are countable and have a finite
number of possibilities. The values are often (but not always) integers.
Examples : Number of children per family, Number of students in a class, Number of
citizens of a country.

23. QUALITATIVE or CATEGORICAL: that are not numerical and which values fits
into categories or express groupings of things (e.g. the different type of fruits, these are
not measurements but classifications, e.g. red, blue, green).
The three different types of categorical variables are:
ORDINAL: variable with an order implied in the levels or represent data with
an order (e.g. rankings). For example: Large, Medium, Small.
For instance, if the severity of road accidents has been measured on a scale such as light,
moderate and fatal accidents.

24. NOMINAL: where no ordering is possible or implied in the levels. (e.g.
brands or species names).
• For example: Eye color is a nominal variable because there is no order
among blue, brown or green eyes.
BINARY or dichotomous : A nominal variable can have between two
levels (e.g., do you smoke? Yes/No or what is your gender? Female/Male)
Represent data with a yes/no or 1/0 outcome (e.g. LEFT or RIGHT).

26. What is a test statistic?

27. • A test statistic is a number calculated by a statistical test.
• It describes how far your observed data is from the null
hypothesis of no relationship between variables or no difference
among sample groups.
• The test statistic tells you how different two or more groups are
from the overall population mean.

28. What is statistical significance?

29. • Statistical significance - it is unlikely their observations could have
occurred under the null hypothesis of a statistical test. Significance is
usually denoted by a p-value, or probability value.
• Statistical significance is arbitrary – it depends on the alpha value, chosen
by the researcher.
• The most common threshold is p < 0.05, which means that the data is
likely to occur less than 5% of the time under the null hypothesis.
• When the p-value falls below the chosen alpha value, then we say the
result of the test is statistically significant.

30. The decision of which statistical test to use depends on:
The research design
The distribution of the data
The type of variable
This will be a result of your research questions/hypotheses you are
trying to answer.

31. HYPOTHESIS TESTING

32. Using Hypothesis Testing, we try to interpret or draw conclusions
about the population using sample data, evaluating two mutually exclusive
statements about a population to determine which statement is best supported
by the sample data.
This is used to determine whether an experiment performed offers ample
evidence to reject a proposal.
Before we start to distinguish between various tests or experiments, we
need to gain a clear understanding of what a null hypothesis is.

33. What is null hypothesis?

34. • A Null Hypothesis implies that there is no strong difference in a given set of
observations.
• The basic assumption of a statistical test is called the null hypothesis.
• we can quantify and interpret statistical measurements to determine if the
null hypothesis should be accepted or not.

35. There are other types of test.
• “Trend tests” examine whether there is a tendency for increasing or decreasing
values in at least three groups.
• A superiority test examines whether an expensive new drug is better than the
conventional standard medication by a specific and medically relevant difference.
• A non-inferiority test might examine whether a cheaper new medicine is not much
worse than a conventional medicine. The acceptable level of activity is specified
before the start of the study on the basis of expert medical knowledge.
• An equivalence test is intended to show that a medication has approximately the
same activity as a conventional standard medication.

36. • To make a statement on whether to deny the null hypothesis a test
statistic will be determined.
• The decision is taken based on the test statistical quantitative
value.
There are two methods of how this decision can be derived:
1. Critical Value
2. p-Value

37. Critical Value Method
In this the idea is to find out whether or not the test statistic observed is
more extreme than a given critical value. - As stated in Wikipedia.
A critical value is the value of the test statistic which defines the upper and
lower bounds of a confidence interval, or which defines the threshold of statistical
significance in a statistical test.
The acceptance region is the set of values of the test statistic for which the null
hypothesis is not rejected.

39. The critical value for a 95% confidence interval is 1.96, where (1-0.95)/2 =
0.025.
The critical value separates the area in the rejection region(s) and not-
rejection region within the probability distribution curve.
• In a two-sided test, if the test numbers are either too small or too high the null
hypothesis is rejected.
• In a left-tailed test, if the test results are too low the null hypothesis is rejected.
• In a right-tailed test, if the test numbers are too high the null hypothesis is
rejected.

40. P-Value
• The P-value method is used in Hypothesis Testing to check the significance
of the given Null Hypothesis. Then, deciding to reject or support it is based
upon the specified significance level or threshold.
• The p-value refers to the probability that sample data were observed at least
as extreme as the test statistics obtained.
• The lower the p-value (closer to 0), the stronger is proof against the null
hypothesis.

42. The main two criteria that need to be taken into consideration before
determining which statistical test to use are:
1.If the data meets those defined assumptions
2.Type of Variable you have to do the test
On the first point, here are some common assumptions:
• Homogenous: The variation is identical across all groups within each
group being compared. When one group has much more variability than
others, the efficacy of the test will be reduced.

43. • No autocorrelation: Independence of observations. There is no relation
between the observations / variables you use in your test (for example,
multiple measurements of a single test subject are not independent, whereas
measurements of several different test subjects are independent).
• Normality: The data follows a normal distribution, the famous bell curve.
If your data does not meet the normality or homogeneity of variance
assumptions, you may be able to perform a non-parametric statistical test which
enables you to make comparisons without any assumptions about the
distribution of data.

45. Choosing a Statistical test

46. Choosing a Statistical test-
• Parametric tests are used if the data is normally distributed.
A parametric statistical test makes an assumption about the population
parameters and the distributions that the data came from. These types of test
includes t-tests, z-tests and anova tests, which assume data is from normal
distribution.
• Non parametric statistical test- Non parametric tests are used when data is
not normally distributed. Non parametric tests include chi-square test.

47. • Chi-squared test – used to compare the distributions of two or more
sets of categorical or ordinal data.
• t-tests – used to compare the means of two sets of data.
• Wilcoxon U test – non-parametric equivalent of the t-test. Based on
the rank order of the data, it may also be used to compare medians.
• ANOVA – analysis of variance, to compare the means of more than
two groups of data.

49. Tests to address the question: Is there a difference between groups – unpaired
(parallel and independent groups) situation?

50. Tests to address the question: Is there a difference between groups
– paired situation?

51. • Pairing signifies that data sets are derived by repeated measurements
(e.g. before-after measurements or multiple measurements across time)
on the same set of subjects.
• A crossover study design also calls for the application of paired group
tests for comparing the effects of different interventions on the same
subjects.
• Sometimes subjects are deliberately paired to match baseline
characteristics such as age, sex, severity or duration of disease.

52. Tests to address the question: Is there an association between
variables?

53. • These are correlation tests and they express the strength of the association
as a correlation coefficient.
• All correlation coefficients vary in magnitude from 0 (no correlation at
all) to 1 (perfect correlation).
• A perfect correlation may indicate but does not necessarily mean causality.
• When two numerical variables are linearly related to each other, a linear
regression analysis can generate a mathematical equation, which can predict
the dependent variable based on a given value of the independent variable.

54. Tests to address the question: Is there an agreement between assessment
(screening / rating / diagnostic) techniques?

55. • This can be a comparison between a new screening technique against
the standard test, new diagnostic test against the available gold standard or
agreement between the ratings or scores given by different observers.
• In case of categorical data, the Cohen’s Kappa statistic is frequently
used, with kappa (which varies from 0 for no agreement at all to 1 for
perfect agreement) indicating strong agreement when it is > 0.7.

56. PARAMETRIC TESTS
Parametric tests are the ones that can only be run with data that
stick with the “three statistical assumptions” mentioned above.
The most common types of parametric tests are divided into three
categories.

57. • The most common types of parametric test include
regression tests,
comparison tests, and
correlation tests.
Regression tests
• Regression tests look for cause-and-effect relationships. They can be
used to estimate the effect of one or more continuous variables on
another variable.

58. Regression tests:
These tests are used test cause-and-effect relationships, if the change in one or more
continuous variable predicts change in another variable.
• Simple linear regression: tests how a change in the predictor variable predicts the
level of change in the outcome variable.
• Multiple linear regression: tests how changes in the combination of two or more
predictor variables predict the level of change in the outcome variable
• Logistic regression: is used to describe data and to explain the relationship
between one dependent (binary) variable and one or more nominal, ordinal,
interval or ratio-level independent variable(s).

60. Comparison tests:
These tests look for the difference between the means of variables: Comparison
of Means.
• T-tests are used when comparing the means of precisely two groups (e.g. the
average heights of men and women).
• Independent t-test: Tests the difference between the same variable from
different populations (e.g., comparing dogs to cats)
• ANOVA and MANOVA tests are used to compare the means of more than
two groups or more(e.g. the average weights of children, teenagers, and
adults).

62. Correlation tests:
These tests look for an association between variable checking whether two
variables are related.
• Pearson Correlation: Tests for the strength of the association between two
continuous variables.
• Spearman Correlation: Tests for the strength of the association between two
ordinal variables (it does not rely on the assumption of normally distributed
data)
• Chi-Square Test: Tests for the strength of the association between two
categorical variables.

63. • Correlation tests check whether variables are related without
hypothesizing a cause-and-effect relationship.
• These can be used to test whether two variables to use in (for example)
a multiple regression test are autocorrelated.

65. NON- NORMAL DISTRIBUTIONS
• Although the normal distribution takes centre part in statistics, many
processes follow non-normal distributions.
Many datasets naturally fit a non-normal model:
• -The number of accidents tends to fit a “Poisson distribution”
• -The Lifetimes of products usually fit a “Weibull distribution”.

66. • The Poisson distribution is used to describe the distribution of rare
events in a large population.
• For example, at any particular time, there is a certain probability that a
particular cell within a large population of cells will acquire a mutation.
• In probability theory and statistics, the Poisson distribution is a discrete
probability distribution that expresses the probability of a given number of
events occurring in a fixed interval of time or space if these events occur
with a known constant mean rate and independently of the time since the
last event.

67. Weibull distribution
• Weibull models are used to describe various types of observed failures of
components and phenomena. They are widely used in reliability and
survival analysis.
• The Weibull Distribution is a continuous probability distribution used to
analyse life data, model failure times and access product reliability.

68. Example of Non-Normal Distributions
1.Beta Distribution.
2.Exponential Distribution.
3.Gamma Distribution.
4.Inverse Gamma Distribution.
5.Log-Normal Distribution.
6.Logistic Distribution.
7.Maxwell-Boltzmann Distribution.
8.Poisson Distribution.
9.Skewed Distribution.
10.Symmetric Distribution.
11.Uniform Distribution.
12.Unimodal Distribution.
13.Weibull Distribution.

69. How do we deal with non-Normal-Distributions?
When your data is supposed to fit a normal distribution but doesn’t, we
could do a few things to handle them:
• We may still be able to run parametric tests if your sample size is large
enough (usually over 20 items) and try to interpret the results accordingly.
• We may choose to transform the data with different statistical techniques,
forcing it to fit a normal distribution.
• If the sample size is small, skewed or if it represents another distribution
type, you might run a non-parametric test.

70. Non-Parametric Tests
• Non-parametric tests don’t make as many assumptions about the data
and are useful when one or more of the three statistical assumptions
are violated.
Note that: The inferences that non-parametric tests make aren’t as
strong as the parametric tests.

75. Statistical tests used frequently in medical studies In order to assess
which statistical tests are most often used in medical publications, 1828
publications were taken from six medical journals in general medicine,
obstetrics and gynecology, or emergency medicine.
The result showed that a reader who is familiar with descriptive
statistics, Pearson’s chi-square test, Fisher’s exact test and the t-test,
should be capable of correctly interpreting the statistics in at least 70% of
the articles.

79. References
1. Statistics How To. 2020. Non-Normal Distribution — Statistics How To. [online] Available at:
https://www.statisticshowto.com/probability-and-statistics/non-normal-distributions
2. Scribbr. 2020. Choosing The Right Statistical Test | Types And Examples. [online] Available
at: https://www.scribbr.com/statistics/statistical-tests
3. Statistical Testing: How to select the best test for your data?. (Online) Available at:
https://towardsdatascience.com/statistical-testing-understanding-how-to-select-the-best-
test-for-your-data-52141c305168
4. McDonald, J.H. 2014. Handbook of Biological Statistics (3rd ed.). Sparky House Publishing,
Baltimore, Maryland.
Campbell MJ, Machin D. In: Medical Statistics: A Common-sense Approach , 2nd edn.
Chichester: Wiley, 1993:2.

80. 5. Parikh MN, Hazra A, Mukherjee J, Gogtay N, editors. Research methodology simplified: Every clinician
a researcher. New Delhi: Jaypee Brothers; 2010. Hypothesis testing and choice of statistical tests; pp. 121–
8.
6. Petrie A, Sabin C, editors. Medical statistics at a glance. 2 nd. London: Blackwell Publishing; 2005. The
theory of linear regression and performing a linear regression analysis; pp. 70–3.
7. Wang D, Clayton T, Bakhai A. Analysis of survival data. In: Wang D, Bakhai A, editors. Clinical trials:
A practical guide to design, analysis and reporting. London: Remedica; 2006. pp. 235–52.
8. https://stats.idre.ucla.edu/spss/whatstat/what-statistical-analysis-should-i-usestatistical-analyses-using-
spss/
9. https://www.analyticsvidhya.com/blog/2018/01/anova-analysis-of-variance/
10. https://www.analyticsvidhya.com/blog/2015/09/hypothesis-testing-explained/
11.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3116565/#:~:text=If%20distribution%20of%20the%20
data,test%20should%20be%20used%20first.

81. 12. Reed JF 3rd, Salen P, Bagher P. Methodological and statistical techniques: what
do residents really need to know about statistics? J Med Syst 2003; 27: 233–8.
13. Dtsch Arztebl Int 2010; 107(19): 343–8
14.https://datascience.foundation/sciencewhitepaper/statistical-testing-
understanding-which-testing-methods-to-use
15. http://www.statisticshowto.com/probability-and-statistics/chi-square/
16. http://stattrek.com/chi-square-test/independence.aspx?Tutorial=AP
17. https://www.investopedia.com/terms/n/null_hypothesis.asp
18. https://en.wikipedia.org/wiki/Statistical_hypothesis_testing