This presentation describes TeachingWithData.org, a collection of resources for faculty who want to include data in their undergraduate social science courses. The presentation was given at the 2010 Annual Meeting of the American Sociological Association (Atlanta) by John Paul DeWitt (SSDAN) and Lynette Hoelter (ICPSR)

1. TeachingWithData.orgResources for Teaching Quantitative Literacy in the Social Sciences John Paul DeWitt & Lynette Hoelter University of Michigan ASA Annual Meeting, August 15, 2010

2. Presentation Outline: Introducing the project partners Quantitative Literacy Introducing TeachingWithData.org General overview (demo of Website) Sociology-related resources Future directions

3. Project Partners ICPSR SSDAN Others involved: American Economic Association Committee on Economic Education American Political Science Association American Sociological Association Association of American Geographers Science Education Resource Center, Carleton College

4. ICPSR World’s oldest and largest social science data archive Began in 1962 as ICPR Membership organization with 700+ members worldwide (non-members can use many resources) Summer Program in Quantitative Methods of Social Research

5. Current Snapshot of ICPSR Currently 7,880 studies (65,200 data sets) Grouped into Thematic Collections Available in multiple formats Federal funding allows parts of the collection to be openly available Data sources: Government Large data collection efforts Principal Investigators Repurposing Other organizations

6. ICPSR: Undergraduate Education Fairly recent attention Response to faculty Undergrad users are fastest growing segment Resources OLC, SETUPS, ICSC, EDRL NSF-funded projects TeachingWithData.org (NSDL) Course, Curriculum, & Laboratory Improvement project to assess the effect of using digital materials on students’ quantitative literacy skills

7. 7 SSDAN-OLC SSDAN’s primary focus is to assist in the dissemination of social data into the classroom with sites like DataCounts! and CensusScope ICPSRgreat track record in research, with a new attention on undergraduate education coming more recently with the welcomed Online Learning Center (OLC)

8. 8 SSDAN: Background Started in 1995 University-based organization that creates demographic media and makes U.S. census data accessible to policymakers, educators, the media, and informed citizens. web sites user guides hands-on classroom materials Integrating Data Analysis (IDA)

9. 9 SSDAN: Classroom Products DataCounts! (www.ssdan.net/datacounts) Collection of approximately 85 Data Driven Learning Modules (DDLMs) WebCHIP (simple contingency table software) Datasets (repackaged decennial census and American Community Survey) Target audience is lower undergraduate courses CensusScope (www.censusscope.org) Maps, charts, and tables Demographic data at local, region, and national levels Key indicators and trends back to 1960 for some variables

10. 10 SSDAN: DataCounts! Quickly connects users to datasets… ..or Data Driven Learning Modules

11. 11 SSDAN: DataCounts! Brief List of available dataset collections Menu for choosing a dataset for analysis

13. Forces faculty to create modules with specific learning goals in mind.

15. Subjects (e.g. Family, Sexuality and Gender)

16. Learning TimeTitle Author and Institution Brief Description

19. 16 SSDAN: DataCounts! Students can quickly run simple cross tabulations to see distributions and test hypotheses

20. 17 SSDAN: DataCounts! Controlling for an additional variable allows for deeper analysis

21. 18 SSDAN DataCounts! Collection of approximately 85 Data Driven Learning Modules (DDLMs) WebCHIP (simple contingency table software) Datasets (repackaged decennial census and American Community Survey) Target is lower undergraduate courses CensusScope Maps, charts, and tables Demographic data at local, region, and national levels Key indicators and trends back to 1960 for some variables

22. 19 SSDAN: CensusScope New ACS data with improved look & feel coming Fall 2010

23. 20 SSDAN: CensusScope Charts, Trends, and Tables All available for states, counties, and metropolitan areas

24. Thinking about Quantitative Literacy (QL) CCLI project to measure effectiveness of using online modules to teach QL First need to agree on skill set representing QL in the social sciences Most use data-based exercises to teach content QL/QR has gotten much recent attention in institutional assessment, many schools requiring a QL component

25. What is QL? “Statistical literacy, quantitative literacy, numeracy --Under the hood, it is what do we want people to be able to do: Read tables and graphs and understand English statements that have numbers in them. That’s a good start,” said Milo Schield, a professor of statistics at Augsburg College and a vice president of the National Numeracy Network. Shield was dismayed to find that, in a survey of his new students, 44 percent could not read a simple 100 percent row table and about a quarter could not accurately interpret a scatter plot of adult heights and weights. Chandler, Michael Alison. What is Quantitative Literacy?, Washington Post, Feb. 5, 2009

26. Similar to Critical Thinking: Students as participants in a democratic society Skills include: Questioning the source of evidence in a stated point Identifying gaps in information Evaluating whether an argument is based on data or opinion/inference/pure speculation Using data to draw logical conclusions

27. Quantitative Literacy Necessary for informed citizenry Skills learned & used within a context Skills: Reading and interpreting tables or graphs and to calculating percentages and the like Working within a scientific model (variables, hypotheses, etc.) Understanding and critically evaluating numbers presented in everyday lives Evaluating arguments based on data Knowing what kinds of data might be useful in answering particular questions For a straightforward definition/skill list, see Samford University’s (not social science specific)

28. Translating to Learning Outcomes Began with AAC&U rubric for quantitative reasoning QL in social sciences: Calculation Interpretation Representation Analysis Method selection Estimation/Reasonableness checks Communication Find/Identify/Generate data Research design Confidence

29. Learning Outcome Dimensions Calculation: Ability to perform mathematical operations Interpretation: Ability to explain information presented in a mathematical form (e.g., tables, equations, graphs, or diagrams) Representation: Ability to convert relevant information from one mathematical form to another (e.g., tables, equations, graphs or diagrams) Analysis: Ability to make judgments based on quantitative analysis

30. Learning Outcomes (con’t) Method selection: Ability to choose the mathematical operations required to answer a research question Estimation/Reasonableness Checks: Ability to recognize the limits of a method and to form reasonable predictions of unknown quantities Communication: Ability to use appropriate levels and types of quantitative information (data, reasoning, tools) to support a conclusion or explain a situation in a way that takes the audience into account.

31. Learning Outcomes (con’t) Find/Identify/Generate Data: Ability to identify or generate appropriate information to answer a question Research design: Understand the links between theory and data Confidence: Level of comfort in performing and interpreting a method of quantitative analysis

32. 29 Assessment Tools and Results

33. QL Skills Are Marketable Often cited by students as something “tangible” that they have learned Definable skill set useful in many career paths Easy to tie to everyday life

34. Including Data Builds QL and: Engages students with disciplines more fully Active learning Better picture of how social scientists work Prevents some of the feelings of “disconnect” between substantive and technical courses Piques student interest Opens the door to the world of data

35. TeachingWithData.org National Science Digital Library – only social science pathway Goal: Make it easier for faculty to use real data in classes Undergraduate (esp. “non-methods”) K(9)-12 efforts Includes survey of ~3600 social science faculty Repository of data-related materials Exercises, including games and simulations Static and dynamic maps, charts, tables Data Publications Tagged with metadata for easy searching

37. Guided Search from home page

38. Resources categorized by more general ‘resource type’ controlled vocabulary

39. Data  focused on tables and figures vs. data sets

40. Reference Shelf  Data Sources, events, pedagogy

41. Classroom Resources  Grouped like resources,

42. Search box with grade level

43. Spring Cleaning – removed hundreds of resources

44. Identified items at lower levels (higher granularity)

45. User log-in (OpenID) and submission

46. Local content

47. Data in the News blog

48. Data for Online Analysis

49. Reading list: ability to create, save, and share

50. Favorites

51. List of resources for course, project, or textbook

53. New Account Setup (OpenID)

54. New Account Setup

55. TeachingWithData.org

57. TeachingWithData.org

58. TeachingWithData.org

59. TeachingWithData.org

61. Submit, edit metadata, review resources

62. “Report” button for review and edit

63. Cleaner metadata, outdated links, etc

64. Comments

65. OpenStudy partnership?

66. Ratings

67. Recommendations

68. User Collaborations (Instructor-Instructor, Instructor-Student)

69. Instant feedback and help

71. Sociology Resources

72. Example Resources “Data in the News” feature – good way to bring in current events Lesson plans/lectures Data-driven exercises Data sources Tools

73. Lesson Plans (Example)

74. More Extensive Lesson Plans (Example)

75. International Data & Information for Comparison (Example)

76. Example: Short Video on Family Change in Canada

77. Static Tables (Example)

78. Graphs & Maps (Example)

79. Interactive Maps (Example)

80. Data-Based Exercises: “Low-Tech” (Example)

81. Data-Based Exercises: Online (Example)

82. Data-Based Exercises: No Stat Software Needed (Example)

83. Simulations (Example)

84. Data for Online Analysis: No Software Needed (Example)

85. Educational Data Extracts for Statistics Packages (Example)

86. Tools for Data Visualization (Example)

87. Future Directions: Include resources for high school teachers Ability to link data to analysis and/or visualization tools Ability for faculty to rate and comment on resources Peer-reviewed materials and capability for faculty to upload their own resources Community building through professional associations and networks of users

88. Your Turn! What have you tried? What has worked best? Favorites we should include in TwD?

89. Acknowledgements PI: George C. Alter, ICPSR Co-PI: William H. Frey, SSDAN Funded by National Science Foundation grant DUE-0840642