Data Collection & Analysis

A RESOURCE GUIDE FOR
UNDERSTANDING COMMUNITY SCHOOLS
Data Collection and Analysis

October 2012

Prepared by:

Iris Hemmerich
Urban Strategies Council

Data Collection and Analysis at Community Schools

Table of Contents
A Resource Guide for Understanding Community Schools .......................................................................... 2
Updating the Resource Guide ................................................................................................................... 4
Additional Community School Resources ................................................................................................. 4
Our Community School work with Oakland Unified School District ............................................................. 5
Data Collection and Analysis: Literature Review .......................................................................................... 6
Introduction .............................................................................................................................................. 6
Review ....................................................................................................................................................... 6
1. Outcomes Addressed by Community Schools ............................................................................... 6
2. Data Collection Methods............................................................................................................... 6
3. Data Indicators.............................................................................................................................. 7
4. Data Analysis Framework ............................................................................................................. 8
Conclusion ................................................................................................................................................. 9
1. Promising Practices ....................................................................................................................... 9
2. Concluding Remarks ...................................................................................................................... 9
Data Collection and Analysis: Annotated Bibliography .............................................................................. 10

1
©Urban Strategies Council, October 2012

A Resource Guide for Understanding Community Schools
INTRODUCTION

Urban Strategies Council has collected and reviewed more than 175 evaluations, case studies,
briefs and reports for use by those considering or planning a community school or community
school district. Our intention is to provide interested individuals and stakeholders the
resources they need to better understand the unique structure and core components of
community schools. The promising practices, recommendations, tools and information shared
in this document have been culled from documents representing the last 20 years of research
and documentation of community schools across the United States.

We highlighted 11 content areas that we believe to be the most foundational for understanding
community schools. Within each of the content areas, you will find:

1. A literature review: The literature reviews for each content area are organized
around core questions and provide a synthesis of the most commonly identified
solutions and responses to each question, as well as highlights, promising practices,
challenges and recommendations.

2. An annotated bibliography: We gathered and annotated literature in each of the
content areas to underscore key themes, some of which include: best practices,
exemplary sites, models and tools. The annotations vary by content area in order to
draw attention to the most pertinent information. For example, the Evaluations content
area includes annotations of the evaluation methodology and indicators of success.

The 11 content areas include the following:

1. Community School Characteristics
Provides a general overview of the structure, function, core elements, programs and
services of a community school.

2. Planning and Design
Explores the general planning and design structures for community schools, and
discusses the initial steps and central components of the planning and design process, as
well as strategies for scaling up community schools.

3. Equity Frameworks and Tools
Examines literature and tools that can be adapted to an equity framework for
community schools. We included equity frameworks and tools that explore
disproportionality and the monitoring of disparities and demographic shifts.

2

4. Collaborative Leadership
Addresses how to build, strengthen and expand the collaborative leadership structure at
community schools. Collaborative leadership is a unique governance structure that
brings together community partners and stakeholders to coordinate a range of services
and opportunities for youth, families and the community.

5. Family and Community Engagement
Explores how community and family engagement operates as well as the challenges for
actualizing it at the site level. Family and community engagement is a unique
component of community schools in which the school, families, and community actively
work together to create networks of shared responsibility for student success.

6. Data Collection and Analysis
Addresses the outcomes measured at community schools, methods for collecting data
at community schools, and short and long term indicators.

7. Assessment Tools
Includes tools used to measure outcomes at community schools.

8. Community School Evaluations
Provides evaluations of community school initiatives with special attention paid to
methodology, indicators of success, findings and challenges.

9. Community School Funding
Explores how to leverage revenue streams and allocate resources at community schools.

10. Budget Tools
Includes tools that support the process of budgeting and fiscal mapping.

11. Community School Sustainability
Explores promising practices for creating sustainability plans, partnership development
and leveraging resources for the future.

3

UPDATING THE RESOURCE GUIDE

Urban Strategies Council will continue its efforts to update the Resource Guide with the most
current information as it becomes available. If you know of topics or resources that are not
currently included in this guide, please contact Alison Feldman, Education Excellence Program,
at alisonf@urbanstrategies.org. We welcome your ideas and feedback for A Resource Guide for
Understanding Community Schools.

ADDITIONAL COMMUNITY SCHOOL RESOURCES

National:

The Coalition for Community Schools
http://www.communityschools.org/

The National Center for Community Schools (Children’s Aid Society)
http://nationalcenterforcommunityschools.childrensaidsociety.org/

Yale University Center in Child Development and Social Policy
http://www.yale.edu/21c/training.html

Regional:

The Center for Community School Partnerships, UC Davis
http://education.ucdavis.edu/community-school-partnerships

Center for Strategic Community Innovation
http://cscinnovation.org/community-schools-project/about-cscis-community-schools-
project/community-school-initiative-services-coaching-and-ta/’

4

Our Community School work with
Oakland Unified School District
Urban Strategies Council has a long history of working with the Oakland Unified School District
(OUSD) to support planning for improved academic achievement. Most recently, we helped
develop and support the implementation of OUSD’s five-year strategic plan, Community
Schools, Thriving Students. Adopted by the Board of Education in June 2011, the plan calls for
building community schools across the district that ensure high-quality instruction; develop
social, emotional and physical health; and create equitable opportunities for learning. Urban
Strategies Council has worked with the school district, community members and other
stakeholders to support this system reform in several ways:

Community Schools Strategic Planning: Urban Strategies Council facilitated six School
Board retreats over a 14-month period to help develop the strategic plan. As part of that
process, the District created 14 task forces to produce recommendations for the plan, with
Urban Strategies Council facilitating one task force and sitting on several others.
Full Service Community Schools Task Force: Urban Strategies Council convened and co-
facilitated the Full Service Community Schools and District Task Force, which created a
structural framework and tools for planning and implementation, and produced a report
with a set of recommendations that formed the foundation of the strategic plan.
Community Engagement in Planning: Urban Strategies Council partnered with the district
to educate and engage more than 900 school and community stakeholders on how
community schools could best serve them.
Planning for Community Schools Leadership Council: Urban Strategies Council has been
working with OUSD’s Department of Family, School and Community Partnerships to lay the
groundwork for building an interagency, cross-sector partnership body that will provide
high-level system oversight and support, and ensure shared responsibility and coordination
of resources towards the vision of healthy, thriving children supported through community
schools.
Convening Workgroups: Urban Strategies Council continues to partner with the District to
convene and facilitate several workgroups developing specific structures, processes, and
practices supporting community school implementation, as well as informing the eventual
work of the Community Schools Leadership Council.
African American Male Achievement Initiative: Urban Strategies Council is a partner in
OUSD’s African American Male Achievement Initiative (AAMAI), a collaboration supporting
efforts to close the achievement gap and improve other key outcomes for African American
males in OUSD. Urban Strategies Council has developed data-based research; explored
promising practices, programs and policies inside and outside the school district; analyzed
the impact of existing system-wide policies; and developed policy recommendations to
improve outcomes in various areas identified by the AAMAI Task Force.
Boys and Men of Color: Urban Strategies Council is the Regional Convener for the Oakland
Boys and Men of Color site, which adopted community schools as a vehicle to improve
health, education and employment outcomes for boys and men of color.
5

Data Collection and Analysis: Literature Review
Introduction

Accurate data collection and analysis is integral to understanding and responding to the impact
of the community school strategy. Data collection and analysis serves a multifunctional purpose
for community schools by tracking outcomes and informing the direction of programs, services,
operational elements and the overall structure of the schools. Furthermore, data can be
leveraged to create visibility and garner support for the expansion of community school
initiatives and its adoption at new sites. We used four central research questions to guide the
literature review of collecting and analyzing data for community schools:

1. What outcomes are community schools addressing?
2. What methods or systems are used to collect the data?
3. What indicators are used to measure outcomes in the short and long term?
4. Is there a suggested data analysis framework?

Published research on data collection and analysis at community school and related education
initiatives from 2009 to 2011 has been included as part of this literature review. While there is
some research that discusses individualized data collection and analysis, what seems to be
lacking in most research and scholarship is an explicit focus on how to disaggregate and analyze
data by targeted indicators.

Review

1. Outcomes Addressed by Community Schools

The community school strategy generally aims to address outcomes in academic, health, social
and emotional success. Short term results were most often identified as the following
outcomes: children’s readiness to enter school; consistent student attendance; the active
involvement of students in learning and in their community; and increased family and
community engagement. The long term results generally included the following outcomes:
academic improvement; the improved physical, social and emotional health of students; safe,
supportive, and stable learning environments; and improvement within the community.

2. Data Collection Methods

The most commonly identified data collection methods include the use of data systems, surveys
and focus groups. For schools and school districts, existing data systems usually collected
information regarding student and family demographics, student academic achievement,

6

attendance, suspensions and expulsions among others. Under this system, it is typical for the
school district to maintain the data while individual school access is variable1.

Surveys were identified as complementary to existing data systems because they help obtain
information about qualitative issues that are often excluded from existing database systems.
This includes qualitative information such as youth motivation, family satisfaction, and partner
and stakeholder perspectives. Utilizing focus groups was also highlighted as an excellent way to
ask process-type questions, such as participants’ sentiments about a program or service and
feedback on how it could be improved2.

3. Data Indicators

The Coalition for Community Schools provided the most comprehensive list of common data
indicators, in which short-term indicators3 were identified (in no rank order) as:

1. immunizations;
2. more children with health insurance;
3. children in expected height and weight range for their age;
4. availability of and attendance at early childhood education programs;
5. parents read to children;
6. vision, hearing, and dental status;
7. daily attendance;
8. early chronic absenteeism;
9. tardiness;
10. truancy;
11. trust between faculty and families;
12. teacher attendance and turnover;
13. faculty believe they are an effective and competent team;
14. community-school partnerships;
15. families support students’ education at home;
16. family attendance at school-wide events and parent-teacher conferences; and
17. Family experiences with school-wide events and classes; and family participation in
school decision-making.

Long term indicators4 were also identified as:

1
Systems Improvement: Training and Technical Assistance Project. “Using Data Effectively: A Toolkit of Practical
Strategies.” Institute for Educational Leadership. Web. 19 December
2011.<http://www.iel.org/pubs/sittap/toolkit_04.pdf>.
2
Shah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. “Community Schools Evaluation
Toolkit.” Coalition for Community Schools, 2009. Web. 19 December 2011.
<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>.
3
Toolkit.” Coalition for Community Schools, 2009. Web. 19 December
2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>.
7

1. families support students’ education at home;
2. family attendance at school-wide events and parent-teacher conferences;
3. family experiences with school-wide events and classes;
4. family participation in school decision-making;
5. asthma control; vision, hearing, and dental status;
6. physical fitness;
7. nutritional habits;
8. positive adult relationships;
9. positive peer relationships;
10. students, staff, and families feel safe;
11. schools are clean; families provide basic needs;
12. incidents of bullying;
13. reports of violence or weapons;
14. employment and employability of residents and families served by the school;
15. student and families with health insurance;
16. community mobility and stability; and
17. juvenile crime.

4. Data Analysis Framework

The Coalition for Community Schools also provided the most comprehensive data analysis
framework, which is a four-part, nine-step results framework for planning and conducting a
community school evaluation. The nine steps include: (1) developing a community schools logic
model; (2) ensuring you have what you need to conduct a successful evaluation; (3) knowing
what you want to evaluate; (4) aligning your evaluation to the logic model; (5) developing the
questions you want your evaluation to answer; (6) deciding what data to collect; (7) collecting
data; (8) making sense of data; and (9) using your findings5.

4
5
8

Conclusion

1. Promising Practices

The “Learning Partner Dashboard” used by the Cincinnati Community Learning Centers6 stood
out as the most promising tool for collecting and analyzing individual student data. Data was
disaggregated by multiple “priority factors” (such as race, non-proficiency on standardized
tests, five or more behavior referrals, etc.) and individual student data was then measured in
relation to rates of participation in specific programs. This allowed for the centers to not only
evaluate individual student success but the success of specific programs.

Another promising data collection method is the Early Warning Indicator and Intervention
System (EWS). EWS is a collaborative approach among educators, administrators, parents, and
communities to use data effectively to keep students on the pathway to graduation. The system
enables rapid identification of students who are in trouble; rapid interventions that are
targeted to students’ immediate and longer-term need for support; the frequent monitoring of
the success of interventions; a rapid modification of interventions that are not working; and
shared learning from outcomes7. It is based on the premise that students gradually show
identifiable signs of disengagement and increased risk of drop-out. Data can thus be used to
identify trends among students and enable educators and parents to intervene strategically and
provide supports.

2. Concluding Remarks

The ability to thoroughly collect, disaggregate and analyze data has a significant impact on the
accuracy and overall success of a community school initiative. The intentional collection and
analysis of data is integral to understanding the types of services and programs needed and the
specific groups that need them most. Data informs the changes that will be made to program,
service and operational elements and measures a school’s alignment with its vision. Moreover,
well leveraged data has the power to build positive public relations and community support
through demonstrating outcomes and trends.

6
Mitchell, Dr. Monica. “Community Learning Centers: Year in Review 2010-2011.” Cincinnati Public Schools,
INNOVATIONS in Community Research, 2011. Web. 19 December 2011.
<http://news.cincinnati.com/assets/AB1820921121.PDF>.
7
Bruce, Mary, John M. Bridgeland, Joanna Hornig Fox, and Robert Balfanz. “On Track for Success: The Use of Early
Warning Indicator and Intervention Systems to Build a Grad Nation.” Civic Enterprises, November 2011. Web. 13
February 2012. <http://www.civicenterprises.net/reports/on_track_for_success.pdf>.

9

Data Collection and Analysis: Annotated Bibliography

Community Schools Evaluation Toolkit
Shah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. Coalition for
Community Schools, 2009. Web. 19 December 2011.
<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.p
df>.

The toolkit is designed to help community schools evaluate their efforts in order to learn from
their successes, identify current challenges, and plan future efforts. It provides a nine-step
process for planning and conducting an evaluation at a community school site(s). The toolkit
serves as a guide to improve community schools’ effectiveness while also telling a school’s
individual story. Additionally, it offers a menu of data collection tools (i.e. surveys, public
databases) for evaluating whether and how your school is achieving results.

Best practices:
1. Use the Community Schools Logic Model
2. Make sure you have what you need to conduct a successful evaluation
a. Consider your readiness
b. Plan for success
3. Know what you want to evaluate
a. Identify your results and decide what activities will help you achieve them
b. Know who you want to evaluate
c. Prioritize your Results
4. Align your evaluation to the Community Schools Logic Model
a. Examine your activities and results in the context of the Community Schools
Logic Model
b. Decide which results will be your focus
5. Develop the questions you want your evaluation to answer
a. Two types of evaluation questions
b. Forming your questions
6. Decide what data to collect
7. Collect data
a. Create a detailed data collection plan
8. Make sense of your data
a. Organize your data in a format that is easy for you to use
b. Focus on what is important about your data
9. Use your findings
a. Select your audience and decide what to report
b. Present your data to change day-to-day practice and results-based planning
c. Use data to change policy
d. Use data for funders
e. Share data beyond the stakeholder group
10

Exemplary sites:
1. Kent School Services Network, Grand Rapids, MI
2. Mark Twain Elementary School, Tulsa, OK
3. Community Learning Centers, Lincoln, NE
4. Carlin Springs Elementary School, Arlington, VA
Models:
1. Table A: Community Schools Logic Model (pg. 8)
2. Table B: Results and Corresponding Indicators (pgs. 10-11)
3. Organizing and Conducting your Evaluation (pg. 12)
4. Continuum of Results (pg. 17)
Tools:
1. Sample Evaluation Questions Related to the Result (pg. 23)
2. Table E: Recommended Results, Indicators, and Data Collection Strategies for
Students, Families, Schools, and Communities (pg. 26-29)
3. Data Collection Plan Template (pg. 34)
4. Appendix C: School Funding Source-Data Collection Matrix (pg. 42)

Data Collection Tools Guide
Coalition for Community Schools. Coalition for Community Schools, 2011. Web. 19 December
2011.
<http://www.communityschools.org/resources/data_collection_instrument_guide.aspx>.

The web page provides a matrix outlining the result area, indicators, target group, questions,
and reference citations for each of the data collection tools included in the Community Schools
Evaluation Toolkit.

On Track for Success: The Use of Early Warning Indicator and Intervention Systems to Build a
Grad Nation
Bruce, Mary, John M. Bridgeland, Joanna Hornig Fox, and Robert Balfanz. Civic Enterprises,
November 2011. Web. 13 February 2012.
<http://www.civicenterprises.net/reports/on_track_for_success.pdf>.

The report provides an overview of the Early Warning Indicator and Intervention Systems (EWS)
research informed by conversations with teachers, district and state officials, nonprofits
working with school systems to implement EWS, and leading researchers. It also outlines
emerging best practices and policy recommendations, so that advocates for children can apply
the best in data innovation to their work. EWS serves as an evaluation tool to aid the process of
accelerating high school graduation rates, improving college and work readiness, and ultimately
strengthening American competitiveness. The instrument uses “real time” or “near real time”
data to identify students who are off track, so that educators can appropriately support them in
advancing from grade to grade, and eventually in graduating from high school with their class.
Emerging best practices are identified for the planning and implementation of EWS. They are:
11

1. Put the student first. Data helps to identify students and craft interventions, but the
success of the student is the ultimate goal;
2. Use research-based indicators and thresholds and respond to student behavior well
before triggers for more intensive interventions are reached;
3. EWS can be implemented as early as the later elementary school years and should cover
key transitions (i.e. sixth and ninth grade);
4. Record data from the simplest and most direct source possible;
5. Ensure data are entered by appropriately trained personnel following well-designed
protocols. The quality and utility of a data system depends on the accuracy of the data
stored within the system. Data must be consistently coded and coding protocols
followed daily;
6. Use the advantages of technology to compile information into easy-to-understand data
presentations. Transparency and usability should be the goals for these reports;
7. Explore issues of privacy. Ensure that children’s privacy is protected while also
leveraging data to effectively promote their success;
8. Teach people how to understand and use data and provide follow-up coaching for data
use. Provide training and professional development to help educators and
administrators learn how to leverage the power of data effectively. Compose a “support
list” of students, revise it every few weeks, and act on that data;
9. Provide local leadership for EWS. Every early warning indicator and intervention system
needs a champion who will advocate for it constantly at the school, district or higher
level;
10. Have a development and implementation plan and timeline;
11. Listen to the end-users and find out what they want before going too far. Convene focus
groups and build up from a pilot; and
12. Integrate EWS into instructional improvement efforts and other student support
services. High performing EWS link efforts to keep students on the graduation path with
school-wide efforts to improve instruction.

Best practices for advancing the field of EWS are identified as well as recommendations for
policymakers trying to advance the use of EWS. Further resources for using EWS are provided in
the Appendices.

Best practices: See 12 best practices above
Exemplary sites:
1. Big Brothers Big Sisters of Eastern Missouri, MO
2. Metropolitan Nashville Public Schools, TN
3. Chicago Public Schools, IL
4. Knox County Schools, TN
5. Philadelphia Education Fund, PA
6. Diplomas Now (U.S.)
7. Dropout Early Warning System, LA
Models: The Civic Marshall Plan to Build a Grad Nation (Appendix I)
12

Developing Early Warning Systems to Identify Potential High School Dropouts
Heppen, Jessica B. and Susan Bowles Therriault. American Institutes for Research, 2009. Web.
13 February 2012.
<http://www.betterhighschools.org/pubs/documents/IssueBrief_EarlyWarningSystemsGuide.p
df>.

The brief provides information on early warning systems along with a tool developed by the
National High School Center that systematically collects early warning indicator data in order to
identify students with the highest risk of dropout. It provides information about: (1) factors
contributing to a student’s dropping out; (2) research on early warning indicators; (3) school-
level early warning systems; (4) district-level early warning systems; and (5) states’ roles in
supporting the development and use of early warning systems.

Tools: Building your Early Warning System (pg. 6)

Community Schools: Promoting Student Success. A Rationale and Results Framework
Coalition for Community Schools. Coalition for Community Schools. Pages 7-11. Web. 19
December 2011.
<http://www.communityschools.org/assets/1/AssetManager/CS_Results_Framework.pdf>.

The purpose of the document is to: (1) outline a rationale for the community school as a vehicle
for increasing student success and strengthening families and community; and (2) define
specific results that community schools seek both in terms of how they function and in
relationship to the well being of students, families, and communities. The second half of the
document, the “Results Framework”, outlines results and indicators of student success as well
as how schools function as community hubs.

Models:
1. Community Schools Logic Model (pg. 9)
2. Community Schools Framework for Student Success (pg. 10)
Tools:
1. Indicators of Capacity (pg. 11)

Using Data Effectively: A Toolkit of Practical Strategies
Systems Improvement: Training and Technical Assistance Project. Institute for Educational
Leadership. Web. 19 December 2011.
<http://www.iel.org/pubs/sittap/toolkit_04.pdf>.

The toolkit is designed to provide ideas and linkages to other resources that will increase the
capacity of projects to use data to accurately assess their needs, design and implement
appropriate interventions, and monitor their progress and outcomes. It offers case study
examples and a variety of tools communities may want to use as part of their strategic planning
13

process. The toolkit is broken down into four main sections: (1) Decision-making; (2) Data
Collection and Analysis; (3) Data Reporting; and (4) Theory of Change.

Exemplary sites (case studies): Multiagency Integrated System of Care, Santa Barbara
County, CA
Tools:
1. Using Data Self-Assessment Guide (Appendix A, pgs. 19-20)
2. Using Data Planning Guide (Appendix A, pgs. 21-23)

14

Data Collection & Analysis

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (11)

Destacado

Destacado (20)

Similar a Data Collection & Analysis

Similar a Data Collection & Analysis (20)

Más de Urban Strategies Council

Más de Urban Strategies Council (14)

Último

Último (20)

Data Collection & Analysis