SlideShare una empresa de Scribd logo
1 de 23
Descargar para leer sin conexión
Classes without dependencies
Teaching the tidyverse to first year science students
Sam Clifford, Iwona Czaplinski, Brett Fyfield, Sama Low-Choy, Belinda
Spratt, Amy Stringer, Nicholas Tierney
2018-07-12
The student body’s got a bad preparation
SEB113 a core unit in QUT’s 2013 redesign of Bachelor of Science
Introduce key math/stats concepts needed for first year science
OP 13 cutoff (ATAR 65)
Assumed knowledge: Intermediate Mathematics
Some calculus and statistics
Not formally required
Diagnostic test and weekly prep material
Basis for further study in disciplines (explicit or embedded)
Still needs to be a self-contained unit that teaches skills
What they need is adult education
Engaging students with use of maths/stats in science
Build good statistical habits from the start
Have students doing analysis
that is relevant to their needs
as quickly as possible
competently
with skills that can be built on
Introduction to programming
reproducibility
separating analysis from the raw data
flexibility beyond menus
correcting mistakes becomes easier
You go back to school
Bad old days
Manual calculation of test statistics
Reliance on statistical tables
Don’t want to replicate senior high school study
Reduce reliance on point and click software that only does
everything students need right now (Excel, Minitab)
Students don’t need to become R developers
Focus on functionality rather than directly controlling every element,
e.g. LATEXvs Word
It’s a bad situation
Initial course development was not tidy
New B Sc course brought forward
Grab bag of topics at request of science academics
Difficult to find tutors who could think outside “traditional” stat. ed.
very low student satisfaction initially
Rapid and radical redesign required
tidyverse an integrated suite focused on transforming data frames
Vectorisation > loops
RStudio > JGR > Rgui.exe
What you want is an adult education (Oh yeah!)
Compassion and support for learners
Problem- and model-based
Technology should support learning goals
Go further, quicker by not focussing on mechanical calculations
Workflow based on functions rather than element manipulation
Statistics is an integral part of science
Statistics isn’t about generating p values
see Cobb in Wasserstein and Lazar [2016]
Machines do the work so people have time to think – IBM (1967)
All models are wrong, but some are useful – Box (1987)
Now here we go dropping science, dropping it all over
Within context of scientific method:
Aims
Methods and Materials
1. Get data/model into an analysis environment
2. Data munging
Results
3. Exploration of data/model
4. Compute model
5. Model diagnostics
Conclusion
6. Interpret meaning of results
I said you wanna be startin’ somethin’
Redesign around ggplot2
ggplot2 introduced us to tidy data requirements
Redesign based on Year 11 summer camp
This approach not covered by textbooks at the time
Tried using JGR and Plot Builder for one semester
Extension to wider tidyverse
Replace unrelated packages/functions with unified approach
Focus on what you want rather than directly coding how to do it
Good effort-reward with limited expertise
Summer(ise) loving, had me a blast; summer(ise) loving,
happened so fast
R is a giant calculator that can operate on objects
ggplot() requires a data frame object
dplyr::summarise() to summarise a column variable
dplyr::group_by() to do summary according to specified
structure
Copy-paste or looping not guaranteed to be MECE
Group-level summary stats leads to potential statistical models
Easier, less error prone, than repeated usage of =AVERAGE()
We want the funk(tional programming paradigm)
Tidy data as observations of variables with structure [Wickham,
2014b]
R as functional programming [Wickham, 2014a]
Actions on entire objects to do things to data and return useful
information
Students enter understanding functions like y(x) = x2
function takes input
function returns output
e.g. mean(x) = i xi/n
Week 4: writing functions to solve calculus problems
magrittr::%>% too conceptually similar to ggplot2::+ for
novices to grasp in first course
Like Frankie sang, I did it my way
What’s the mean gas mileage for each engine geometry and
transmission type for the 32 cars listed in 1974 Motor Trends
magazine?
Loops For each of the pre-computed number of
groups, subset, summarise and store how
you want
tapply() INDEX a list of k vectors, 1 summary
FUNction, returns k-dimensional array
dplyr specify grouping variables and which sum-
mary statistics, returns tidy data frame ready
for model/plot
Night of the living baseheads
Like all procedural languages, plot() has one giant list of
arguments
Focus is on how plot is drawn rather than what you want to plot
Inefficiency of keystrokes
re-stating the things being plotted
setting up plot axis limits
loop counters for small multiples, etc.
Toot toot, chugga chugga, big red car
Say we want to plot cars’ fuel efficiency against weight
library(tidyverse)
data(mtcars)
mtcars <- mutate(
mtcars, l100km = 235.2146/mpg,
wt_T = wt/2.2046,
am = factor(am, levels = c(0,1),
labels=c("Auto", "Manual")),
vs = factor(vs, levels = c(0,1),
labels=c("V","S")))
plot(y=mtcars$l100km, x=mtcars$wt_T)
1.0 1.5 2.0 2.5
101520
mtcars$wt_T
mtcars$l100km
Fairly quick to say what
goes on x and y axes
More arguments → better
graph
xlim, ylim
xlab, ylab
main
type, pch
What if we want to see how
it varies with
engine geometry
transmission type
The wisdom of the fool won’t set you free
yrange <- range(mtcars$l100km)
xrange <- range(mtcars$wt_T)
levs <- expand.grid(vs = c("V", "S"),
am = c("Auto", "Manual"))
par(mfrow = c(2,2))
for (i in 1:nrow(levs)){
dat_to_plot <- merge(levs[i, ], mtcars)
plot(dat_to_plot$l100km ~ dat_to_plot$wt_T, pch=16,
xlab="Weight (t)", xlim=xrange,
ylab="Fuel efficiency (L/100km)",
ylim=yrange,
main = sprintf("%s-%s", levs$am[i],
levs$vs[i]))}
1.0 1.5 2.0 2.5
101520
Auto−V
Weight (t)
Fuelefficiency(L/100km)
1.0 1.5 2.0 2.5
101520
Auto−S
Weight (t)
Fuelefficiency(L/100km)
1.0 1.5 2.0 2.5
101520
Manual−V
Weight (t)
Fuelefficiency(L/100km)
1.0 1.5 2.0 2.5
101520
Manual−S
Weight (t)
Fuelefficiency(L/100km)
ggplot(data = mtcars,
aes(x = wt_T,
y = l100km)) +
geom_point() +
facet_grid(am ~ vs) +
theme_bw() +
xlab("Weight (t)") +
ylab("Fuel efficiency (L/100km)")
V S
AutoManual
1.0 1.5 2.0 2.5 1.0 1.5 2.0 2.5
10
15
20
10
15
20
Weight (t)
Fuelefficiency(L/100km)
One, two, princes kneel before you
Both approaches do the same thing
Idea base ggplot2
Plot variables Specify vectors Coordinate system de-
fined by variables
Small multiples Loops, subsets, par facet_grid
Common axes Pre-computed Inherited from data
V/S A/M annotation Strings Inherited from data
Axis labels Per axis set For whole plot
Focus on putting things on the page vs representing variables
I got a grammar Hazel and a grammar Tilly
Plots are built from [Wickham, 2010]
data – which variables are mapped to aesthetic elements
geometry – how do we draw the data?
annotations – what is the context of these shapes?
Build more complex plots by adding commands and layering elements,
rather than by stacking individual points and lines e.g.
make a scatter plot, THEN
add a trend line (with inherited x, y), THEN
facet by grouping variable, THEN
change axis information
When I’m good, I’m very good; but when I’m bad, I’m better
Want to make good plots as soon as possible
Learning about Tufte’s principles [Tufte, 1983, Pantoliano, 2012]
Discuss what makes a plot good and bad
Seeing how ggplot2 code translates into graphical elements
Week 2 workshop has students making best and worst plots for a
data set, e.g.
Sie ist ein Model und sie sieht gut aus
Make use of broom package to get model summaries
Get data frames rather than summary.lm() text vomit
tidy()
parameter estimates
CIs
t test info [Greenland et al., 2016]
glance()
everything else
ggplot2::fortify()
regression diagnostic info instead of plot.lm()
stat_qq(aes(x=.stdresid)) for residual quantiles
geom_point(aes(x=.fitted, y=.resid)) for fitted vs
residuals
When you hear some feedback keep going take it higher
Positives
More confidence and students see use of maths/stats in science
Students enjoy group discussions in workshops
Some students continue using R over Excel in future units
Labs can be done online in own time
Negatives
Request for more face to face help rather than online
Labs can be done online in own time (but are they?)
Downloading of slides rather than attending/watching lectures
Things can only get better
Focus on what you want from R rather than how you do it
representing variables graphically
summarising over structure in data
tidiers for models
Statistics embedded in scientific theory [Diggle and Chetwynd, 2011]
Problem-based learning
groups of novices
supervised by tutors
discussion of various approaches
Peter J. Diggle and Amanda G. Chetwynd. Statistics and Scientific
Method: An Introduction for Students and Researchers. Oxford
University Press, 2011.
Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin,
Charles Poole, Steven N. Goodman, and Douglas G. Altman.
Statistical tests, p values, confidence intervals, and power: a guide to
misinterpretations. European Journal of Epidemiology, 31(4):337–350,
apr 2016. URL https://doi.org/10.1007/s10654-016-0149-3.
Mike Pantoliano. Data visualization principles: Lessons from Tufte, 2012.
URL https:
//moz.com/blog/data-visualization-principles-lessons-from-tufte.
Edward Tufte. The Visual Display of Quantitative Information. Graphics
Press, 1983.
Ronald L. Wasserstein and Nicole A. Lazar. The ASA's statement on
p-values: Context, process, and purpose. The American Statistician, 70
(2):129–133, Apr 2016. URL
https://doi.org/10.1080/00031305.2016.1154108.
H. Wickham. Advanced R. Chapman & Hall/CRC The R Series. Taylor &
Francis, 2014a. ISBN 9781466586963. URL
https://books.google.com.au/books?id=PFHFNAEACAAJ.
Hadley Wickham. A layered grammar of graphics. Journal of
Computational and Graphical Statistics, 19(1):3–28, 2010. doi:
10.1198/jcgs.2009.07098.
Hadley Wickham. Tidy data. Journal of Statistical Software, 59(1):1–23,
2014b. ISSN 1548-7660. URL
https://www.jstatsoft.org/index.php/jss/article/view/v059i10.

Más contenido relacionado

La actualidad más candente

Data Structures 2004
Data Structures 2004Data Structures 2004
Data Structures 2004Sanjay Goel
 
Graph Neural Network for Phenotype Prediction
Graph Neural Network for Phenotype PredictionGraph Neural Network for Phenotype Prediction
Graph Neural Network for Phenotype Predictiontuxette
 
Spsshelp 100608163328-phpapp01
Spsshelp 100608163328-phpapp01Spsshelp 100608163328-phpapp01
Spsshelp 100608163328-phpapp01Henock Beyene
 
Solving dynamics problems with matlab
Solving dynamics problems with matlabSolving dynamics problems with matlab
Solving dynamics problems with matlabSérgio Castilho
 
Differences-in-Differences
Differences-in-DifferencesDifferences-in-Differences
Differences-in-DifferencesJaehyun Song
 
Principal component analysis and lda
Principal component analysis and ldaPrincipal component analysis and lda
Principal component analysis and ldaSuresh Pokharel
 
La statistique et le machine learning pour l'intégration de données de la bio...
La statistique et le machine learning pour l'intégration de données de la bio...La statistique et le machine learning pour l'intégration de données de la bio...
La statistique et le machine learning pour l'intégration de données de la bio...tuxette
 
Intuition – Based Teaching Mathematics for Engineers
Intuition – Based Teaching Mathematics for EngineersIntuition – Based Teaching Mathematics for Engineers
Intuition – Based Teaching Mathematics for EngineersIDES Editor
 
ALTERNATIVE METHOD TO LINEAR CONGRUENCE
ALTERNATIVE METHOD TO LINEAR CONGRUENCEALTERNATIVE METHOD TO LINEAR CONGRUENCE
ALTERNATIVE METHOD TO LINEAR CONGRUENCEPolemer Cuarto
 
Reproducibility and differential analysis with selfish
Reproducibility and differential analysis with selfishReproducibility and differential analysis with selfish
Reproducibility and differential analysis with selfishtuxette
 
Kernel methods for data integration in systems biology
Kernel methods for data integration in systems biologyKernel methods for data integration in systems biology
Kernel methods for data integration in systems biologytuxette
 
Principal Component Analysis and Clustering
Principal Component Analysis and ClusteringPrincipal Component Analysis and Clustering
Principal Component Analysis and ClusteringUsha Vijay
 
Lect4 principal component analysis-I
Lect4 principal component analysis-ILect4 principal component analysis-I
Lect4 principal component analysis-Ihktripathy
 
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...Prashant Borkar
 
Selective inference and single-cell differential analysis
Selective inference and single-cell differential analysisSelective inference and single-cell differential analysis
Selective inference and single-cell differential analysistuxette
 
Decision Tree Algorithm Implementation Using Educational Data
Decision Tree Algorithm Implementation  Using Educational Data Decision Tree Algorithm Implementation  Using Educational Data
Decision Tree Algorithm Implementation Using Educational Data ijcax
 

La actualidad más candente (20)

Data Structures 2004
Data Structures 2004Data Structures 2004
Data Structures 2004
 
Pca
PcaPca
Pca
 
Graph Neural Network for Phenotype Prediction
Graph Neural Network for Phenotype PredictionGraph Neural Network for Phenotype Prediction
Graph Neural Network for Phenotype Prediction
 
Spsshelp 100608163328-phpapp01
Spsshelp 100608163328-phpapp01Spsshelp 100608163328-phpapp01
Spsshelp 100608163328-phpapp01
 
Solving dynamics problems with matlab
Solving dynamics problems with matlabSolving dynamics problems with matlab
Solving dynamics problems with matlab
 
Differences-in-Differences
Differences-in-DifferencesDifferences-in-Differences
Differences-in-Differences
 
PCA
PCAPCA
PCA
 
Principal component analysis and lda
Principal component analysis and ldaPrincipal component analysis and lda
Principal component analysis and lda
 
La statistique et le machine learning pour l'intégration de données de la bio...
La statistique et le machine learning pour l'intégration de données de la bio...La statistique et le machine learning pour l'intégration de données de la bio...
La statistique et le machine learning pour l'intégration de données de la bio...
 
Intuition – Based Teaching Mathematics for Engineers
Intuition – Based Teaching Mathematics for EngineersIntuition – Based Teaching Mathematics for Engineers
Intuition – Based Teaching Mathematics for Engineers
 
ALTERNATIVE METHOD TO LINEAR CONGRUENCE
ALTERNATIVE METHOD TO LINEAR CONGRUENCEALTERNATIVE METHOD TO LINEAR CONGRUENCE
ALTERNATIVE METHOD TO LINEAR CONGRUENCE
 
Reproducibility and differential analysis with selfish
Reproducibility and differential analysis with selfishReproducibility and differential analysis with selfish
Reproducibility and differential analysis with selfish
 
Kernel methods for data integration in systems biology
Kernel methods for data integration in systems biologyKernel methods for data integration in systems biology
Kernel methods for data integration in systems biology
 
Principal Component Analysis and Clustering
Principal Component Analysis and ClusteringPrincipal Component Analysis and Clustering
Principal Component Analysis and Clustering
 
Lect4 principal component analysis-I
Lect4 principal component analysis-ILect4 principal component analysis-I
Lect4 principal component analysis-I
 
Chap011
Chap011Chap011
Chap011
 
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...
Grouping and Displaying Data to Convey Meaning: Tables & Graphs chapter_2 _fr...
 
Presentation1
Presentation1Presentation1
Presentation1
 
Selective inference and single-cell differential analysis
Selective inference and single-cell differential analysisSelective inference and single-cell differential analysis
Selective inference and single-cell differential analysis
 
Decision Tree Algorithm Implementation Using Educational Data
Decision Tree Algorithm Implementation  Using Educational Data Decision Tree Algorithm Implementation  Using Educational Data
Decision Tree Algorithm Implementation Using Educational Data
 

Similar a Classes without Dependencies - UseR 2018

Machine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptxMachine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptxVenkateswaraBabuRavi
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401butest
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401butest
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401butest
 
Technology Lesson Plan Assignment: Quadratice Functions
Technology Lesson Plan Assignment: Quadratice FunctionsTechnology Lesson Plan Assignment: Quadratice Functions
Technology Lesson Plan Assignment: Quadratice Functionsdart11746
 
Ict Tools In Mathematics Instruction
Ict Tools In Mathematics InstructionIct Tools In Mathematics Instruction
Ict Tools In Mathematics InstructionMiracule D Gavor
 
EE-232-LEC-01 Data_structures.pptx
EE-232-LEC-01 Data_structures.pptxEE-232-LEC-01 Data_structures.pptx
EE-232-LEC-01 Data_structures.pptxiamultapromax
 
Automatically Answering And Generating Machine Learning Final Exams
Automatically Answering And Generating Machine Learning Final ExamsAutomatically Answering And Generating Machine Learning Final Exams
Automatically Answering And Generating Machine Learning Final ExamsRichard Hogue
 
AlgorithmsModelsNov13.pptx
AlgorithmsModelsNov13.pptxAlgorithmsModelsNov13.pptx
AlgorithmsModelsNov13.pptxPerumalPitchandi
 
[DOLAP2023] The Whys and Wherefores of Cubes
[DOLAP2023] The Whys and Wherefores of Cubes[DOLAP2023] The Whys and Wherefores of Cubes
[DOLAP2023] The Whys and Wherefores of CubesUniversity of Bologna
 
An alternative learning experience in transition level mathematics
An alternative learning experience in transition level mathematicsAn alternative learning experience in transition level mathematics
An alternative learning experience in transition level mathematicsDann Mallet
 
ch12lectPP420
ch12lectPP420ch12lectPP420
ch12lectPP420fiegent
 
22_RepeatedMeasuresDesign_Complete.pptx
22_RepeatedMeasuresDesign_Complete.pptx22_RepeatedMeasuresDesign_Complete.pptx
22_RepeatedMeasuresDesign_Complete.pptxMarceloHenriques20
 
Rd1 r17a19 datawarehousing and mining_cap617t_cap617
Rd1 r17a19 datawarehousing and mining_cap617t_cap617Rd1 r17a19 datawarehousing and mining_cap617t_cap617
Rd1 r17a19 datawarehousing and mining_cap617t_cap617Ravi Kumar
 
Course Syllabus For Operations Management
Course Syllabus For Operations ManagementCourse Syllabus For Operations Management
Course Syllabus For Operations ManagementYnal Qat
 
313 IDS _Course_Introduction_PPT.pptx
313 IDS _Course_Introduction_PPT.pptx313 IDS _Course_Introduction_PPT.pptx
313 IDS _Course_Introduction_PPT.pptxsameernsn1
 
A data science observatory based on RAMP - rapid analytics and model prototyping
A data science observatory based on RAMP - rapid analytics and model prototypingA data science observatory based on RAMP - rapid analytics and model prototyping
A data science observatory based on RAMP - rapid analytics and model prototypingAkin Osman Kazakci
 
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...IJCSES Journal
 
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...ijcseit
 

Similar a Classes without Dependencies - UseR 2018 (20)

Machine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptxMachine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptx
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
 
Technology Lesson Plan Assignment: Quadratice Functions
Technology Lesson Plan Assignment: Quadratice FunctionsTechnology Lesson Plan Assignment: Quadratice Functions
Technology Lesson Plan Assignment: Quadratice Functions
 
Ict Tools In Mathematics Instruction
Ict Tools In Mathematics InstructionIct Tools In Mathematics Instruction
Ict Tools In Mathematics Instruction
 
EE-232-LEC-01 Data_structures.pptx
EE-232-LEC-01 Data_structures.pptxEE-232-LEC-01 Data_structures.pptx
EE-232-LEC-01 Data_structures.pptx
 
Automatically Answering And Generating Machine Learning Final Exams
Automatically Answering And Generating Machine Learning Final ExamsAutomatically Answering And Generating Machine Learning Final Exams
Automatically Answering And Generating Machine Learning Final Exams
 
AlgorithmsModelsNov13.pptx
AlgorithmsModelsNov13.pptxAlgorithmsModelsNov13.pptx
AlgorithmsModelsNov13.pptx
 
[DOLAP2023] The Whys and Wherefores of Cubes
[DOLAP2023] The Whys and Wherefores of Cubes[DOLAP2023] The Whys and Wherefores of Cubes
[DOLAP2023] The Whys and Wherefores of Cubes
 
An alternative learning experience in transition level mathematics
An alternative learning experience in transition level mathematicsAn alternative learning experience in transition level mathematics
An alternative learning experience in transition level mathematics
 
4.80 sy it
4.80 sy it4.80 sy it
4.80 sy it
 
ch12lectPP420
ch12lectPP420ch12lectPP420
ch12lectPP420
 
22_RepeatedMeasuresDesign_Complete.pptx
22_RepeatedMeasuresDesign_Complete.pptx22_RepeatedMeasuresDesign_Complete.pptx
22_RepeatedMeasuresDesign_Complete.pptx
 
Rd1 r17a19 datawarehousing and mining_cap617t_cap617
Rd1 r17a19 datawarehousing and mining_cap617t_cap617Rd1 r17a19 datawarehousing and mining_cap617t_cap617
Rd1 r17a19 datawarehousing and mining_cap617t_cap617
 
Course Syllabus For Operations Management
Course Syllabus For Operations ManagementCourse Syllabus For Operations Management
Course Syllabus For Operations Management
 
313 IDS _Course_Introduction_PPT.pptx
313 IDS _Course_Introduction_PPT.pptx313 IDS _Course_Introduction_PPT.pptx
313 IDS _Course_Introduction_PPT.pptx
 
A data science observatory based on RAMP - rapid analytics and model prototyping
A data science observatory based on RAMP - rapid analytics and model prototypingA data science observatory based on RAMP - rapid analytics and model prototyping
A data science observatory based on RAMP - rapid analytics and model prototyping
 
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
 
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
 

Último

THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONTHEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONHumphrey A Beña
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfJemuel Francisco
 
Textual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSTextual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSMae Pangan
 
TEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docxTEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docxruthvilladarez
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxVanesaIglesias10
 
Choosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for ParentsChoosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for Parentsnavabharathschool99
 
Millenials and Fillennials (Ethical Challenge and Responses).pptx
Millenials and Fillennials (Ethical Challenge and Responses).pptxMillenials and Fillennials (Ethical Challenge and Responses).pptx
Millenials and Fillennials (Ethical Challenge and Responses).pptxJanEmmanBrigoli
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmStan Meyer
 
Presentation Activity 2. Unit 3 transv.pptx
Presentation Activity 2. Unit 3 transv.pptxPresentation Activity 2. Unit 3 transv.pptx
Presentation Activity 2. Unit 3 transv.pptxRosabel UA
 
Keynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designKeynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designMIPLM
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Seán Kennedy
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...Nguyen Thanh Tu Collection
 
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Celine George
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management SystemChristalin Nelson
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataBabyAnnMotar
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Projectjordimapav
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Celine George
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 

Último (20)

THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONTHEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
 
Textual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSTextual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHS
 
TEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docxTEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docx
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptx
 
Choosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for ParentsChoosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for Parents
 
Millenials and Fillennials (Ethical Challenge and Responses).pptx
Millenials and Fillennials (Ethical Challenge and Responses).pptxMillenials and Fillennials (Ethical Challenge and Responses).pptx
Millenials and Fillennials (Ethical Challenge and Responses).pptx
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and Film
 
Presentation Activity 2. Unit 3 transv.pptx
Presentation Activity 2. Unit 3 transv.pptxPresentation Activity 2. Unit 3 transv.pptx
Presentation Activity 2. Unit 3 transv.pptx
 
Keynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designKeynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-design
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 
Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
 
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management System
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped data
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Project
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 

Classes without Dependencies - UseR 2018

  • 1. Classes without dependencies Teaching the tidyverse to first year science students Sam Clifford, Iwona Czaplinski, Brett Fyfield, Sama Low-Choy, Belinda Spratt, Amy Stringer, Nicholas Tierney 2018-07-12
  • 2. The student body’s got a bad preparation SEB113 a core unit in QUT’s 2013 redesign of Bachelor of Science Introduce key math/stats concepts needed for first year science OP 13 cutoff (ATAR 65) Assumed knowledge: Intermediate Mathematics Some calculus and statistics Not formally required Diagnostic test and weekly prep material Basis for further study in disciplines (explicit or embedded) Still needs to be a self-contained unit that teaches skills
  • 3. What they need is adult education Engaging students with use of maths/stats in science Build good statistical habits from the start Have students doing analysis that is relevant to their needs as quickly as possible competently with skills that can be built on Introduction to programming reproducibility separating analysis from the raw data flexibility beyond menus correcting mistakes becomes easier
  • 4. You go back to school Bad old days Manual calculation of test statistics Reliance on statistical tables Don’t want to replicate senior high school study Reduce reliance on point and click software that only does everything students need right now (Excel, Minitab) Students don’t need to become R developers Focus on functionality rather than directly controlling every element, e.g. LATEXvs Word
  • 5. It’s a bad situation Initial course development was not tidy New B Sc course brought forward Grab bag of topics at request of science academics Difficult to find tutors who could think outside “traditional” stat. ed. very low student satisfaction initially Rapid and radical redesign required tidyverse an integrated suite focused on transforming data frames Vectorisation > loops RStudio > JGR > Rgui.exe
  • 6. What you want is an adult education (Oh yeah!) Compassion and support for learners Problem- and model-based Technology should support learning goals Go further, quicker by not focussing on mechanical calculations Workflow based on functions rather than element manipulation Statistics is an integral part of science Statistics isn’t about generating p values see Cobb in Wasserstein and Lazar [2016]
  • 7. Machines do the work so people have time to think – IBM (1967) All models are wrong, but some are useful – Box (1987)
  • 8. Now here we go dropping science, dropping it all over Within context of scientific method: Aims Methods and Materials 1. Get data/model into an analysis environment 2. Data munging Results 3. Exploration of data/model 4. Compute model 5. Model diagnostics Conclusion 6. Interpret meaning of results
  • 9. I said you wanna be startin’ somethin’ Redesign around ggplot2 ggplot2 introduced us to tidy data requirements Redesign based on Year 11 summer camp This approach not covered by textbooks at the time Tried using JGR and Plot Builder for one semester Extension to wider tidyverse Replace unrelated packages/functions with unified approach Focus on what you want rather than directly coding how to do it Good effort-reward with limited expertise
  • 10. Summer(ise) loving, had me a blast; summer(ise) loving, happened so fast R is a giant calculator that can operate on objects ggplot() requires a data frame object dplyr::summarise() to summarise a column variable dplyr::group_by() to do summary according to specified structure Copy-paste or looping not guaranteed to be MECE Group-level summary stats leads to potential statistical models Easier, less error prone, than repeated usage of =AVERAGE()
  • 11. We want the funk(tional programming paradigm) Tidy data as observations of variables with structure [Wickham, 2014b] R as functional programming [Wickham, 2014a] Actions on entire objects to do things to data and return useful information Students enter understanding functions like y(x) = x2 function takes input function returns output e.g. mean(x) = i xi/n Week 4: writing functions to solve calculus problems magrittr::%>% too conceptually similar to ggplot2::+ for novices to grasp in first course
  • 12. Like Frankie sang, I did it my way What’s the mean gas mileage for each engine geometry and transmission type for the 32 cars listed in 1974 Motor Trends magazine? Loops For each of the pre-computed number of groups, subset, summarise and store how you want tapply() INDEX a list of k vectors, 1 summary FUNction, returns k-dimensional array dplyr specify grouping variables and which sum- mary statistics, returns tidy data frame ready for model/plot
  • 13. Night of the living baseheads Like all procedural languages, plot() has one giant list of arguments Focus is on how plot is drawn rather than what you want to plot Inefficiency of keystrokes re-stating the things being plotted setting up plot axis limits loop counters for small multiples, etc.
  • 14. Toot toot, chugga chugga, big red car Say we want to plot cars’ fuel efficiency against weight library(tidyverse) data(mtcars) mtcars <- mutate( mtcars, l100km = 235.2146/mpg, wt_T = wt/2.2046, am = factor(am, levels = c(0,1), labels=c("Auto", "Manual")), vs = factor(vs, levels = c(0,1), labels=c("V","S"))) plot(y=mtcars$l100km, x=mtcars$wt_T) 1.0 1.5 2.0 2.5 101520 mtcars$wt_T mtcars$l100km Fairly quick to say what goes on x and y axes More arguments → better graph xlim, ylim xlab, ylab main type, pch What if we want to see how it varies with engine geometry transmission type
  • 15. The wisdom of the fool won’t set you free yrange <- range(mtcars$l100km) xrange <- range(mtcars$wt_T) levs <- expand.grid(vs = c("V", "S"), am = c("Auto", "Manual")) par(mfrow = c(2,2)) for (i in 1:nrow(levs)){ dat_to_plot <- merge(levs[i, ], mtcars) plot(dat_to_plot$l100km ~ dat_to_plot$wt_T, pch=16, xlab="Weight (t)", xlim=xrange, ylab="Fuel efficiency (L/100km)", ylim=yrange, main = sprintf("%s-%s", levs$am[i], levs$vs[i]))} 1.0 1.5 2.0 2.5 101520 Auto−V Weight (t) Fuelefficiency(L/100km) 1.0 1.5 2.0 2.5 101520 Auto−S Weight (t) Fuelefficiency(L/100km) 1.0 1.5 2.0 2.5 101520 Manual−V Weight (t) Fuelefficiency(L/100km) 1.0 1.5 2.0 2.5 101520 Manual−S Weight (t) Fuelefficiency(L/100km) ggplot(data = mtcars, aes(x = wt_T, y = l100km)) + geom_point() + facet_grid(am ~ vs) + theme_bw() + xlab("Weight (t)") + ylab("Fuel efficiency (L/100km)") V S AutoManual 1.0 1.5 2.0 2.5 1.0 1.5 2.0 2.5 10 15 20 10 15 20 Weight (t) Fuelefficiency(L/100km)
  • 16. One, two, princes kneel before you Both approaches do the same thing Idea base ggplot2 Plot variables Specify vectors Coordinate system de- fined by variables Small multiples Loops, subsets, par facet_grid Common axes Pre-computed Inherited from data V/S A/M annotation Strings Inherited from data Axis labels Per axis set For whole plot Focus on putting things on the page vs representing variables
  • 17. I got a grammar Hazel and a grammar Tilly Plots are built from [Wickham, 2010] data – which variables are mapped to aesthetic elements geometry – how do we draw the data? annotations – what is the context of these shapes? Build more complex plots by adding commands and layering elements, rather than by stacking individual points and lines e.g. make a scatter plot, THEN add a trend line (with inherited x, y), THEN facet by grouping variable, THEN change axis information
  • 18. When I’m good, I’m very good; but when I’m bad, I’m better Want to make good plots as soon as possible Learning about Tufte’s principles [Tufte, 1983, Pantoliano, 2012] Discuss what makes a plot good and bad Seeing how ggplot2 code translates into graphical elements Week 2 workshop has students making best and worst plots for a data set, e.g.
  • 19. Sie ist ein Model und sie sieht gut aus Make use of broom package to get model summaries Get data frames rather than summary.lm() text vomit tidy() parameter estimates CIs t test info [Greenland et al., 2016] glance() everything else ggplot2::fortify() regression diagnostic info instead of plot.lm() stat_qq(aes(x=.stdresid)) for residual quantiles geom_point(aes(x=.fitted, y=.resid)) for fitted vs residuals
  • 20. When you hear some feedback keep going take it higher Positives More confidence and students see use of maths/stats in science Students enjoy group discussions in workshops Some students continue using R over Excel in future units Labs can be done online in own time Negatives Request for more face to face help rather than online Labs can be done online in own time (but are they?) Downloading of slides rather than attending/watching lectures
  • 21. Things can only get better Focus on what you want from R rather than how you do it representing variables graphically summarising over structure in data tidiers for models Statistics embedded in scientific theory [Diggle and Chetwynd, 2011] Problem-based learning groups of novices supervised by tutors discussion of various approaches
  • 22. Peter J. Diggle and Amanda G. Chetwynd. Statistics and Scientific Method: An Introduction for Students and Researchers. Oxford University Press, 2011. Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman. Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31(4):337–350, apr 2016. URL https://doi.org/10.1007/s10654-016-0149-3. Mike Pantoliano. Data visualization principles: Lessons from Tufte, 2012. URL https: //moz.com/blog/data-visualization-principles-lessons-from-tufte. Edward Tufte. The Visual Display of Quantitative Information. Graphics Press, 1983. Ronald L. Wasserstein and Nicole A. Lazar. The ASA's statement on p-values: Context, process, and purpose. The American Statistician, 70 (2):129–133, Apr 2016. URL https://doi.org/10.1080/00031305.2016.1154108.
  • 23. H. Wickham. Advanced R. Chapman & Hall/CRC The R Series. Taylor & Francis, 2014a. ISBN 9781466586963. URL https://books.google.com.au/books?id=PFHFNAEACAAJ. Hadley Wickham. A layered grammar of graphics. Journal of Computational and Graphical Statistics, 19(1):3–28, 2010. doi: 10.1198/jcgs.2009.07098. Hadley Wickham. Tidy data. Journal of Statistical Software, 59(1):1–23, 2014b. ISSN 1548-7660. URL https://www.jstatsoft.org/index.php/jss/article/view/v059i10.