SlideShare una empresa de Scribd logo
1 de 9
Descargar para leer sin conexión
NURTURES Evaluation Report: Year 1
Completed by:
Acumen Research and Evaluation, LLC
April 10, 2012
On the behalf of:
1
Table of Contents
I. Executive Summary 2
II. Introduction 3
a. Evaluation Logic Model 3
III. Observations of the Planning Process 3
a. Tangible Outcomes 3
b. Group Progression 4
c. Group Dynamics 4
IV. Interviews with Planning Team 5
a. Interview Responses 5
b. Interview Summary 7
V. Master Teacher Recruiting Materials and Results 7
VI. Training Phase Assessment Tools 8
VII. Conclusion and Recommendations 8
VIII. References 8
This report summarizes the activities and finding of the evaluators of the NSF MSP project at The University of
Toledo entitled NURTURES from July 2011 through March 2012.
2
I. Executive Summary
This report examines the Planning Phase (Phase I) of the NURTURES project. The
project focus is to develop a complementary science learning environment for grades pre-K -3.
There are three phases to this project: Planning Phase (Year 1), Demonstration/Training Phase
(Year 2), and Implementation Phase (Years 3-5). This report examines the Planning Phase.
Evaluation of the Planning Phase consisted of comparing project outcome attainment with
proposal timeline and observing and reviewing the Planning Team’s group process procedures
and outcomes. Data were collected through review of project documents such as hiring
paperwork, direct observation of the retreat, examination of Summer Institutes, personal
interviews with members of Planning Team, and examination of recruiting materials and efforts.
Project documentation showed attainment of all Year 1 outcomes within a timely manner.
All employees have been hired and trained (Master Teachers are still in training but that is within
the timeframe). A detailed timeline of activities for this summer and Year 2 has been developed.
Evaluators made direct observations of several of the planning team’s weekly meetings.
Tangible outcomes, group progress, and group dynamics were examined. To date, the team has
achieved the tangible outcomes expected at this point. Interactions between members continually
improved over the course of the year as group members became more familiar with one another.
As far as group dynamics, the group works as a cohesive entity towards mutual goals.
The evaluators conducted personal interviews with members of the planning team to
further examine the group process—particularly from each member’s perspective. Again, strong
evidence of mutual respect and an atmosphere conducive to positive interaction and progress was
provided. While a few felt the meetings to be a bit long, all appreciated that they were well-
organized and little time was wasted. Most members of the team identified more closely with
their own specific role within the project as opposed to identifying with the project as a whole.
Master Teachers were recruited and hired. While the goal was to recruit teachers from the
Toledo Review and Alternative Compensation System (TRACS) within the Toledo Public
Schools, only two of the Master Teachers are TRACS teachers. However, the group represents
teachers from grades pre-K through 3 with experience teaching science and coaching or
providing other teachers with professional development.
The evaluators have developed interview protocols for Master Teachers to be used during
the Summer Institute, have worked with content faculty in the development of pretest/posttest
instruments to measure content gain, and have developed a Teacher Inquiry Behavior Protocol
and Parent Science Activity Survey to be piloted in Master Teacher classrooms during Year 2.
Overall, Phase I concludes having met its objectives. The groundwork for a successful
MSP has been laid and will be piloted during Year 2.
3
II. Introduction
The evaluation plan for the project was created by Acumen Research and Evaluation, LLC
to determine the impact of NURTURES on children’s interest in and curiosity about science. The
NURTURES grant evaluation logic model includes three phases of the evaluation plan matched
to the three phases of the project: planning, training, and implementation. The grant is currently
in year 1 and Phase 1 (Year 1 ends June 30, 2012). Evaluation of the Planning Phase consisted of
observing and reviewing the Planning Team’s group process procedures and outcomes. Data
were collected through direct observation of the retreat, examination of Summer Institutes,
personal interviews with members of Planning Team, and examination of recruiting materials
and efforts. In addition, the evaluators determined existing instruments suitable for measuring
Phase 2 (Training/Demonstration Phase) of the project and developed instruments when
appropriate measures did not exist.
A. Planning Phase Evaluation Model
The Planning Phase Evaluation Model was designed to map out the assessment of the
grant activities as it pertains to the goals and outcomes of the first year of the project (Table 1).
During Year 1, the Planning Phase, the evaluation team had five major responsibilities—(1)
observe the planning process and resulting group dynamics of the Planning Team, (2) review
schedules for Summer Institute and Demonstration Phase, (3) interview Planning Team
members, (4) observe the planning retreat, and (5) review recruiting materials for the Master
Teacher positions. Not listed in the model but of equal importance was the compilation and
development of assessments to use during Phase 2. To date, the evaluators have completed
observations of the planning process, interviewed the planning team, examined recruiting
materials for the Master Teachers, and compiled existing and developed assessments for Year 2.
Observation of the Retreat and review of the Summer Institute schedule will occur in May/June
2012 and reported in Year 2 Annual Report.
III. Observations of the Planning Process
A member of the evaluation team frequently attended planning meetings during the past
year. The purpose was not to provide input but rather to observe the group dynamics and
planning process. A teamwork protocol was used to ascertain the level of collaboration and
teamwork that is present within the group (Schultz, Israel, and Lance, 2002). Several themes
emerged from the meetings: Tangible outcomes, group progression, and group dynamics.
A. Tangible outcomes
The planning team, project staff, and graduate assistants have been developing a
framework for the 2012 Summer Institute. The framework will be used to educate Master
Teachers as to how to teach their peers science inquiry based strategies in the classroom (in
preparation for Summer Institute 2013). To assist with the creation of the tool, graduate students
have been researching and structuring how science inquiry content is currently being taught in
the classroom. Other strategies are being utilized to aid with the development of lesson content.
Reverse engineering is a well-known engineering strategy that is being used to develop what the
Summer Institute content will be for 2013. The group has been working towards the creation of
example lessons to illustrate and better understand how adults learn. In addition to researching
4
Table 1: NURTURES Planning Phase Evaluation Model
Planning Phase—The foundation for all outcomes and overall project goal
Method Variable Measured Purpose Progress to date
Observation of
planning process
Group process and
achievement of stated
outcomes
To determine whether the
Planning Team is
working together towards
project goals
Completed
Observation of
retreat
Group dynamics To determine whether
format is conducive to
collaboration
Will be
completed in June
2012
Interview with
planning team
members
Participants’
perception of success
of retreat
To determine degree to
which teamwork goals
are being met
Completed
Review of
Summer institute
schedule
Planning for summer
completed
To determine whether
summer program is
thoroughly planned
Will be
completed by
June 2012
Examination of
recruiting
materials and
strategies
Planning for
recruitment
completed
To determine whether
project is prepared for
recruitment
Completed
teacher behavior development (with regards to adult learning), the group has been addressing
other aspects of the grant such as what exemplar informal activities should be included for the
parents and children.
In addition to science content, the pedagogy methods and literacy strategies that will be
implemented into the new science content have been discussed. The group has decided to
develop a month long training session to develop inquiry based science content in science
lessons. The team has also been researching ways in which children think and ways to chart what
inquiry behaviors need to occur to illustrate learning is occurring.
A smaller group within the group has been developing a coding system for teacher
science inquiry behaviors. To ensure that the tool is inclusive, both early childhood and
engineering graduate students are developing the behavior codes for the assessment.
B. Group Progression
Over the past year, the planning team has been meeting weekly. The group has been able
to brainstorm and share ideas freely. Initially, the group spent quite a bit of time determining the
types of behaviors good science teachers exhibit when teaching science. During the second half
of the year, specific tasks were assigned to some members and smaller groups were formed.
With the groundwork for the first Summer Institute well underway, sights have turned to the
examination the new national science standards to determine what science learning consists of as
well as the hallmarks of a Master Teacher.
C. Group Dynamics
The group worked cohesively and group members were comfortable sharing perspectives
and eliciting feedback. While members of the group did not always agree, opposing perspectives
were presented cordially and no one member of the group dominated the conversation. Also,
5
team leaders were able to successfully guide their workgroups independent of the larger group.
The weekly meetings were conducive to removing barriers as they allowed group members to
work through their problems and, when problems did occur, group members were able to redirect
behavior to achieve desired outcomes. The diversity of disciplines provided a rich foundation for
group members to voice their opinions and work together.
IV. Interviews with Planning Team
In March 2012 the evaluators conducted personal interviews with members of the
Leadership team to examine group processes—to assess the level of teamwork and collaboration
present within the team. The Leadership team is comprised of six members including the PI, the
co-PIs, and some senior personnel. The interviews consisted of questions concerning their main
role on the project, group process, satisfaction with progress, and content development. Each
question asked is provided below along with a summary of the responses of the six interviewees.
A. Interview Responses
1. First tell me what you been doing for the NURTURES project over the past year?
The majority of the team described personal responsibilities concerning the project such
as providing advice, acting as a resource, developing a coding scheme, and hiring and training
personnel. Only two of the members provided a rich description of the group work as a whole
and the progress that has been made. The difference is in the interpretation of the word “you”.
The majority of respondents interpreted “you” to mean themselves and the other two identified
with the group to the extent that “you” has taken on the collective definition of “your group” or
“you all”.
2. With whom do you work with to plan?
Four of the six mentioned the entire planning team. Two indicated specific people—one
mentioned a member of the staff and the other worked mostly with graduate assistants.
3. How well does the group work together?
They all agreed that the group works well together and a few noted that the membership
consists of some very different backgrounds. In fact, several were surprised at how well the
members of the group get along. Only one respondent provided examples of their harmony. This
respondent felt comfortable speaking up when unsure and indicated that no matter how little a
group member knew about a subject, he or she could express an opinion and the group respected
it.
4. When problems arise, how well does the group work together to solve them? Can you
provide a specific example?
The definition of a “problem” seemed to vary among the respondents. In most cases, a
problem was equated with either a misunderstanding (a member not understanding what
someone else is trying to say) or a disagreement among members of the group. Responses to this
item were vague. Most said that disagreements were discussed and resolved. One member felt
that the group had not experienced any problems to date. Only one member was willing to
provide an example of a problem and indicated that the objection one member had to a situation
was not relevant to their discussion.
5. What are the major issues you are involved in?
6
All but one member had a clear idea of what they contribute to the group and which
responsibilities and to which issues they contribute. One member from the five who had a clear
idea felt that only recently has that contribution been realized and is now happy to have a place
within the group.
6. How would you describe the rate of progress in which the group is making in dealing
with major issues identified?
Three of the respondents felt progress to be satisfactory and one of the three mentioned
the value of taking their time at the beginning to set the groundwork. Two others felt progress
during the first six months to be slow but are now satisfied with the speed at which the group is
moving towards meeting its timetable. These two did not, however, object openly to their
perceived slow rate of progress during the early months of the project. One respondent did not
answer this question.
7. Where is your group as far as achieving its goals?
Three members provided specific goals achieved that related to their role in the project.
Two indicated progress towards project level goals and one was not sure “what the current goals
are.” While the remaining five felt progress to be satisfactory, only the two who identified
project goals, responded to the question accurately (your group rather than you).
8. How does the group work through times when they are bogged down?
One respondent was not aware of any roadblocks. The other five agreed that when the
group gets stymied they either table the topic till a future meeting or try to work through it. One
member felt that early on the group was not cohesive enough to work through rough spots but
now it is at a place where there is mutual respect in spite of the fact that the group has grown in
size.
9. What were your initial expectations of the planning meetings?
Only the PI had a clear idea as to what the planning meetings would be like. Others either
had no expectations, vague expectations or a very different idea of what was to occur.
10. Have your initial expectations changed?
Not surprisingly, all but the PI has changed his/her view of the meeting expectations. Of
course those who had not made any assumptions about the meeting now have a clear
understanding of their function within the project. Those with vague ideas have a more precise
understanding and one member was surprised that the meetings were as democratic as they are
and that everyone seems to have input as to the direction the meetings take.
11. Can you tell me effective attributes that form the planning meetings?
The respondents mentioned several positive attributes. Most frequently they noted the
diverse group, mutual respect, and the open atmosphere. Also mentioned was that the meetings
were well organized. One member felt the meetings run too long.
12. What do you think the end results of the planning meetings will be?
7
All felt the Planning Phase will end with a strong Summer Institute design. One member
felt that the Planning Phase will not be completed at the end of the first year but will continue
into the Pilot Phase.
13. Do you know what to do next and are you comfortable with carrying these
responsibilities?
Five of the six articulated future responsibilities and felt comfortable with their
responsibilities for the summer and Year 2. One was not sure of what was expected of him/her.
B. Summary of Interviews:
Responses from the six members of the planning team indicated mutual respect and an
atmosphere conducive to positive interaction and progress. While some felt the initial meetings
to be drawn out, all agreed that recently they have been more productive and most (5) have a
clear vision of their role in the project for Phase II. All felt the meetings have been well
organized and appreciated the time taken to review agendas and provide background materials in
advance. While all members see themselves as a contributing piece to the project, most do not
associate themselves with the planning group as such. In other words, when answering questions,
the majority of comments centered upon the individual (I will/have done this) as opposed to
work the group as a whole has accomplished. Only two of the members spoke from the
perspective of the group rather than their individual contribution and experience.
V. Master Teacher Recruiting Materials and Results
Prior to recruiting Master Teachers, the NURTURES planning team held a meeting in
which the expectations and commitments of the Master Teacher position were detailed. Based
upon previous experience with Toledo Public Schools, it was determined that recruitment efforts
would focus on selecting teachers who are in the Toledo Review and Alternative Compensation
System (TRACS). The goal of TRACS is to promote teacher quality while improving student
academic performance. TRACS focuses on ongoing teacher professional development that
targets specific student academic and school improvement needs including effective teaching and
learning and the retention of the most accomplished teachers through acknowledging and
rewarding teaching excellence. TRACS maximizes the talents of recognized teacher leaders by
assigning additional assignments or leadership roles and/or by placing them in high needs
schools or in challenging teaching assignments. If the project was unable to recruit teachers from
TRACS, then STEM teachers with strong recommendations from principals and/or other school
leadership would be considered.
The interview questions for Master Teacher were intended to identify the underlying
philosophy of the potential Master Teachers. The interview team consisted of the PI/co-PI’s and
project staff. Questions focused on the interviewee’s ideology as it relates to his/her teaching
philosophy. Other questions focused on teaching practices. For example, “Describe the best
science lesson you ever taught; what are some ways you integrate STEM into your everyday
curriculum?” The questions were designed to probe for information about the teacher’s
flexibility in dealing with the constant changes that occur in the classroom when teaching using
inquiry-based strategies. To date, all six of the Master Teachers have been selected. They
represent preschool through grade 3 and range in years of teaching from 6 to 29 with an average
of 16.5 years. While only two are TRACS teachers, a third has been a science support teacher in
8
Toledo Public Schools for several years. The two TRACS teachers, both teachers of first grade,
have served as a Peer Math Coach and a Peer Literacy Coach for their district.
VI. Training Phase Assessment Tools
The original evaluation plan called for the use of the Science Teacher Self Efficacy
Instrument (STEBI) (Riggs and Enochs, 1990) the Science Teacher Ideological Preference Scale
(STIPS) (Jones and Harty, 1978; Gado, 2005), focus group interviews, content assessments, and
classroom observations. In addition, a leadership survey used in the NSF MSP LEADERS will
be adapted to assess Master Teacher leadership skills development and growth. Data will be
collected from Master Teachers during Year 2 to establish a baseline to measure Master Teacher
growth over time. Rather than implementing the Horizon Inside the Classroom Observation
Protocol, a classroom observation rubric that examines teacher inquiry behaviors has been
developed and will be discussed with advisor Shymansky when he visits the project this summer.
The Teacher Inquiry Behavior Protocol was devised to assess the level of science inquiry in
teaching behaviors. The protocol rates teachers on each of the following: questioning, defining
problems, engagement in evidence-based discourse, and developing and using STEM concepts.
The decision to change the observation tool was based upon a desire to more closely match the
instrument with the intentions of the project. It will be piloted in the Master Teacher classrooms
during Year 2 to establish evidence of reliability and validity.
Another tool developed this year and piloted next year is the Parent Science Activity
Survey. This instrument is designed to examine parents’ opinions of change in science interest in
their children as well as their own proclivity towards helping their child learn more about
science.
VII. Conclusions
In summary, the Planning Phase has met its stated objectives. Personnel have been hired;
detailed planning for the first Summer Institute is well on its way; and members of the project
team work together towards a common goal. Substantial groundwork has been laid for Phase II
of the project and project personnel appear eager to enter into this next phase.
VIII. References
Gado, I. (2005). Determinants of k-2 school teachers' orientation towards inquiry-based science
activities: A mixed method study. International Journal of Science and Mathematics
Education, 3, 511-539.
Riggs, I., & Enochs, L. (1990). Towards the development of an elementary teacher’s science
teaching efficacy belief instrument. Science Education, 74, 625-637.
Schulz, A.J., Israel, B.A., & Lantz, P. (2002). Instrument for evaluating dimensions of group
dynamics within community-based participatory research partnerships. Evaluation and
Program Planning, 26, 249-262.

Más contenido relacionado

La actualidad más candente

Dlp template 2017
Dlp template 2017Dlp template 2017
Dlp template 2017WencyPigon
 
Whole School Planning For Student Learning
Whole School Planning For Student LearningWhole School Planning For Student Learning
Whole School Planning For Student LearningDavid Tyler
 
Curriculum Mapping Intro 1 13 10
Curriculum Mapping Intro 1 13 10Curriculum Mapping Intro 1 13 10
Curriculum Mapping Intro 1 13 10fergui
 
Curriculum evaluation
Curriculum evaluationCurriculum evaluation
Curriculum evaluationHennaAnsari
 
School Improvement Plan
School Improvement PlanSchool Improvement Plan
School Improvement PlanMedia Center
 
010 COT-RPMS protocols
010 COT-RPMS protocols010 COT-RPMS protocols
010 COT-RPMS protocolsedmond84
 
Curriculum evaluation
Curriculum evaluationCurriculum evaluation
Curriculum evaluationbibashenry
 
Curriculum leadership chapter 12 powerpoint
Curriculum leadership chapter 12 powerpointCurriculum leadership chapter 12 powerpoint
Curriculum leadership chapter 12 powerpointlbrannan84
 
enhanced teacher education curruculum anchored on obe (mr.balana)
enhanced teacher education curruculum anchored on obe (mr.balana)enhanced teacher education curruculum anchored on obe (mr.balana)
enhanced teacher education curruculum anchored on obe (mr.balana)lemuel lucaban
 
Curriculum development
Curriculum developmentCurriculum development
Curriculum developmentArish Hares
 
Course evaluation summary report
Course evaluation summary reportCourse evaluation summary report
Course evaluation summary reportScottish TNE
 
Curriculum implementation
Curriculum implementationCurriculum implementation
Curriculum implementationkishamarie1
 
Assessing the curriculum
Assessing the curriculumAssessing the curriculum
Assessing the curriculummilcrez
 
Proposed Pre-departure Training for Peru Practicum
Proposed Pre-departure Training for Peru PracticumProposed Pre-departure Training for Peru Practicum
Proposed Pre-departure Training for Peru PracticumCortney Copeland
 
Curriculum Management for Senior Management Teams in South Africa
Curriculum Management for Senior Management Teams in South AfricaCurriculum Management for Senior Management Teams in South Africa
Curriculum Management for Senior Management Teams in South AfricaEducation Moving Up Cc.
 

La actualidad más candente (20)

Dlp template 2017
Dlp template 2017Dlp template 2017
Dlp template 2017
 
Whole School Planning For Student Learning
Whole School Planning For Student LearningWhole School Planning For Student Learning
Whole School Planning For Student Learning
 
Curriculum Mapping Intro 1 13 10
Curriculum Mapping Intro 1 13 10Curriculum Mapping Intro 1 13 10
Curriculum Mapping Intro 1 13 10
 
Curriculum evaluation
Curriculum evaluationCurriculum evaluation
Curriculum evaluation
 
Cd assignment 1111
Cd assignment 1111Cd assignment 1111
Cd assignment 1111
 
School Improvement Plan
School Improvement PlanSchool Improvement Plan
School Improvement Plan
 
010 COT-RPMS protocols
010 COT-RPMS protocols010 COT-RPMS protocols
010 COT-RPMS protocols
 
Year plan
Year planYear plan
Year plan
 
Curriculum evaluation
Curriculum evaluationCurriculum evaluation
Curriculum evaluation
 
Curriculum leadership chapter 12 powerpoint
Curriculum leadership chapter 12 powerpointCurriculum leadership chapter 12 powerpoint
Curriculum leadership chapter 12 powerpoint
 
enhanced teacher education curruculum anchored on obe (mr.balana)
enhanced teacher education curruculum anchored on obe (mr.balana)enhanced teacher education curruculum anchored on obe (mr.balana)
enhanced teacher education curruculum anchored on obe (mr.balana)
 
Planning 1
Planning 1Planning 1
Planning 1
 
Curriculum development
Curriculum developmentCurriculum development
Curriculum development
 
M4
M4M4
M4
 
Course evaluation summary report
Course evaluation summary reportCourse evaluation summary report
Course evaluation summary report
 
Curriculum implementation
Curriculum implementationCurriculum implementation
Curriculum implementation
 
Assessing the curriculum
Assessing the curriculumAssessing the curriculum
Assessing the curriculum
 
Proposed Pre-departure Training for Peru Practicum
Proposed Pre-departure Training for Peru PracticumProposed Pre-departure Training for Peru Practicum
Proposed Pre-departure Training for Peru Practicum
 
DEVELOPING LESSON PLAN - EDUC 5
DEVELOPING LESSON PLAN - EDUC 5DEVELOPING LESSON PLAN - EDUC 5
DEVELOPING LESSON PLAN - EDUC 5
 
Curriculum Management for Senior Management Teams in South Africa
Curriculum Management for Senior Management Teams in South AfricaCurriculum Management for Senior Management Teams in South Africa
Curriculum Management for Senior Management Teams in South Africa
 

Destacado

Impact 2012 annual report
Impact 2012 annual reportImpact 2012 annual report
Impact 2012 annual reportLouise Smyth
 
Gk 12 eval report 2011
Gk 12 eval report 2011Gk 12 eval report 2011
Gk 12 eval report 2011Louise Smyth
 
Impact what do urban student think about science
Impact what do urban student think about scienceImpact what do urban student think about science
Impact what do urban student think about scienceLouise Smyth
 
2012 2013 annual report part ii
2012 2013 annual report part ii2012 2013 annual report part ii
2012 2013 annual report part iiLouise Smyth
 
Gk 12 eval report 2010
Gk 12 eval report 2010Gk 12 eval report 2010
Gk 12 eval report 2010Louise Smyth
 
Annual nsf gk 12 report u toledo 4-7-09
Annual nsf gk 12 report u toledo 4-7-09Annual nsf gk 12 report u toledo 4-7-09
Annual nsf gk 12 report u toledo 4-7-09Louise Smyth
 
Gk 12 eval report 2012
Gk 12 eval report 2012Gk 12 eval report 2012
Gk 12 eval report 2012Louise Smyth
 
14 years of evaluation
14 years of evaluation14 years of evaluation
14 years of evaluationLouise Smyth
 
Impact 2009 annual report
Impact 2009 annual reportImpact 2009 annual report
Impact 2009 annual reportLouise Smyth
 

Destacado (9)

Impact 2012 annual report
Impact 2012 annual reportImpact 2012 annual report
Impact 2012 annual report
 
Gk 12 eval report 2011
Gk 12 eval report 2011Gk 12 eval report 2011
Gk 12 eval report 2011
 
Impact what do urban student think about science
Impact what do urban student think about scienceImpact what do urban student think about science
Impact what do urban student think about science
 
2012 2013 annual report part ii
2012 2013 annual report part ii2012 2013 annual report part ii
2012 2013 annual report part ii
 
Gk 12 eval report 2010
Gk 12 eval report 2010Gk 12 eval report 2010
Gk 12 eval report 2010
 
Annual nsf gk 12 report u toledo 4-7-09
Annual nsf gk 12 report u toledo 4-7-09Annual nsf gk 12 report u toledo 4-7-09
Annual nsf gk 12 report u toledo 4-7-09
 
Gk 12 eval report 2012
Gk 12 eval report 2012Gk 12 eval report 2012
Gk 12 eval report 2012
 
14 years of evaluation
14 years of evaluation14 years of evaluation
14 years of evaluation
 
Impact 2009 annual report
Impact 2009 annual reportImpact 2009 annual report
Impact 2009 annual report
 

Similar a Nurtures evaluation report 2012

curriculum development process-1.pptx
curriculum development process-1.pptxcurriculum development process-1.pptx
curriculum development process-1.pptxSani191640
 
Ped 104 curriculum improvement final (1)
Ped 104 curriculum improvement final   (1)Ped 104 curriculum improvement final   (1)
Ped 104 curriculum improvement final (1)JOHNDAVEANIONBARTE
 
Lesson 4 Curriculum Development 2023.pptx
Lesson 4 Curriculum  Development 2023.pptxLesson 4 Curriculum  Development 2023.pptx
Lesson 4 Curriculum Development 2023.pptxFenembarMekonnen
 
Characteristics of a Good Curriculum
Characteristics of a Good Curriculum Characteristics of a Good Curriculum
Characteristics of a Good Curriculum Jill Frances Salinas
 
Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)
Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)
Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)Angie Magdasoc
 
Developing and validating an instrument for student evaluation of
Developing and validating an instrument for student evaluation ofDeveloping and validating an instrument for student evaluation of
Developing and validating an instrument for student evaluation ofCiv Nortub
 
Curriculum and Course Planning_BINALET.pptx
Curriculum and Course Planning_BINALET.pptxCurriculum and Course Planning_BINALET.pptx
Curriculum and Course Planning_BINALET.pptxCedraBinalet1
 
continous assessment (LH) for Jinela Teachers.pdf
continous assessment (LH)   for Jinela Teachers.pdfcontinous assessment (LH)   for Jinela Teachers.pdf
continous assessment (LH) for Jinela Teachers.pdfbeyeneyewondwossenDi
 
Managemenyt of instructional planning
Managemenyt of instructional planningManagemenyt of instructional planning
Managemenyt of instructional planningjermskie
 
Learning experiences
Learning experiences Learning experiences
Learning experiences Marry Adina
 
WhatWhy-and-How-to-Evaluate-a-Curriculum.pptx
WhatWhy-and-How-to-Evaluate-a-Curriculum.pptxWhatWhy-and-How-to-Evaluate-a-Curriculum.pptx
WhatWhy-and-How-to-Evaluate-a-Curriculum.pptxEckelleGraceMagalay
 

Similar a Nurtures evaluation report 2012 (20)

838-2.docx
838-2.docx838-2.docx
838-2.docx
 
Curriculum Development-Aroona Hashmi
Curriculum Development-Aroona HashmiCurriculum Development-Aroona Hashmi
Curriculum Development-Aroona Hashmi
 
curriculum development process-1.pptx
curriculum development process-1.pptxcurriculum development process-1.pptx
curriculum development process-1.pptx
 
Ped 104 curriculum improvement final (1)
Ped 104 curriculum improvement final   (1)Ped 104 curriculum improvement final   (1)
Ped 104 curriculum improvement final (1)
 
Lesson 4 Curriculum Development 2023.pptx
Lesson 4 Curriculum  Development 2023.pptxLesson 4 Curriculum  Development 2023.pptx
Lesson 4 Curriculum Development 2023.pptx
 
Siraj .dotx;body of research
Siraj .dotx;body of researchSiraj .dotx;body of research
Siraj .dotx;body of research
 
Educational leadership
Educational leadershipEducational leadership
Educational leadership
 
Characteristics of a Good Curriculum
Characteristics of a Good Curriculum Characteristics of a Good Curriculum
Characteristics of a Good Curriculum
 
Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)
Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)
Curriculum Approaches (Systems-managerial and Intellectual-academic Approach)
 
Developing and validating an instrument for student evaluation of
Developing and validating an instrument for student evaluation ofDeveloping and validating an instrument for student evaluation of
Developing and validating an instrument for student evaluation of
 
Curriculum and Course Planning_BINALET.pptx
Curriculum and Course Planning_BINALET.pptxCurriculum and Course Planning_BINALET.pptx
Curriculum and Course Planning_BINALET.pptx
 
Curriculum
CurriculumCurriculum
Curriculum
 
Program evaluation plan
Program evaluation planProgram evaluation plan
Program evaluation plan
 
Assessment_in_Practice_Maryville
Assessment_in_Practice_MaryvilleAssessment_in_Practice_Maryville
Assessment_in_Practice_Maryville
 
continous assessment (LH) for Jinela Teachers.pdf
continous assessment (LH)   for Jinela Teachers.pdfcontinous assessment (LH)   for Jinela Teachers.pdf
continous assessment (LH) for Jinela Teachers.pdf
 
Managemenyt of instructional planning
Managemenyt of instructional planningManagemenyt of instructional planning
Managemenyt of instructional planning
 
Learning experiences
Learning experiences Learning experiences
Learning experiences
 
WhatWhy-and-How-to-Evaluate-a-Curriculum.pptx
WhatWhy-and-How-to-Evaluate-a-Curriculum.pptxWhatWhy-and-How-to-Evaluate-a-Curriculum.pptx
WhatWhy-and-How-to-Evaluate-a-Curriculum.pptx
 
Assessment-Poster
Assessment-PosterAssessment-Poster
Assessment-Poster
 
Lt meeting 3
Lt meeting 3Lt meeting 3
Lt meeting 3
 

Más de Louise Smyth

Duckett asee presentation 2019.6.16
Duckett asee presentation 2019.6.16Duckett asee presentation 2019.6.16
Duckett asee presentation 2019.6.16Louise Smyth
 
Earthday Flyer pdf
Earthday Flyer pdfEarthday Flyer pdf
Earthday Flyer pdfLouise Smyth
 
Mixed methods technology analysis
Mixed methods technology analysisMixed methods technology analysis
Mixed methods technology analysisLouise Smyth
 
Leaders evaluation report year 1 2010
Leaders evaluation report year 1 2010Leaders evaluation report year 1 2010
Leaders evaluation report year 1 2010Louise Smyth
 
Leaders evaluation report year 2 2011
Leaders evaluation report year 2 2011Leaders evaluation report year 2 2011
Leaders evaluation report year 2 2011Louise Smyth
 
Year 3 leaders eval report year 3 2012
Year 3 leaders eval report  year 3 2012Year 3 leaders eval report  year 3 2012
Year 3 leaders eval report year 3 2012Louise Smyth
 
2013, year 4 leaders eval report
2013, year 4 leaders eval report2013, year 4 leaders eval report
2013, year 4 leaders eval reportLouise Smyth
 
Impact 2011 annual report
Impact 2011 annual reportImpact 2011 annual report
Impact 2011 annual reportLouise Smyth
 
Paragraph about mentzer
Paragraph about mentzerParagraph about mentzer
Paragraph about mentzerLouise Smyth
 
Measuring effectiveness of introduction to teaching
Measuring effectiveness of introduction to teachingMeasuring effectiveness of introduction to teaching
Measuring effectiveness of introduction to teachingLouise Smyth
 
Long and winding road
Long and winding roadLong and winding road
Long and winding roadLouise Smyth
 
Exploring effectiveness of teacher leader support
Exploring effectiveness of teacher leader supportExploring effectiveness of teacher leader support
Exploring effectiveness of teacher leader supportLouise Smyth
 
Developing science teacher leaders
Developing  science teacher leadersDeveloping  science teacher leaders
Developing science teacher leadersLouise Smyth
 

Más de Louise Smyth (14)

Duckett asee presentation 2019.6.16
Duckett asee presentation 2019.6.16Duckett asee presentation 2019.6.16
Duckett asee presentation 2019.6.16
 
Earthday Flyer pdf
Earthday Flyer pdfEarthday Flyer pdf
Earthday Flyer pdf
 
Peas in a pod
Peas in a podPeas in a pod
Peas in a pod
 
Mixed methods technology analysis
Mixed methods technology analysisMixed methods technology analysis
Mixed methods technology analysis
 
Leaders evaluation report year 1 2010
Leaders evaluation report year 1 2010Leaders evaluation report year 1 2010
Leaders evaluation report year 1 2010
 
Leaders evaluation report year 2 2011
Leaders evaluation report year 2 2011Leaders evaluation report year 2 2011
Leaders evaluation report year 2 2011
 
Year 3 leaders eval report year 3 2012
Year 3 leaders eval report  year 3 2012Year 3 leaders eval report  year 3 2012
Year 3 leaders eval report year 3 2012
 
2013, year 4 leaders eval report
2013, year 4 leaders eval report2013, year 4 leaders eval report
2013, year 4 leaders eval report
 
Impact 2011 annual report
Impact 2011 annual reportImpact 2011 annual report
Impact 2011 annual report
 
Paragraph about mentzer
Paragraph about mentzerParagraph about mentzer
Paragraph about mentzer
 
Measuring effectiveness of introduction to teaching
Measuring effectiveness of introduction to teachingMeasuring effectiveness of introduction to teaching
Measuring effectiveness of introduction to teaching
 
Long and winding road
Long and winding roadLong and winding road
Long and winding road
 
Exploring effectiveness of teacher leader support
Exploring effectiveness of teacher leader supportExploring effectiveness of teacher leader support
Exploring effectiveness of teacher leader support
 
Developing science teacher leaders
Developing  science teacher leadersDeveloping  science teacher leaders
Developing science teacher leaders
 

Último

A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.mcshagufta46
 
7movierulz.uk
7movierulz.uk7movierulz.uk
7movierulz.ukaroemirsr
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView
 
PDT 89 - $1.4M - Seed - Plantee Innovations.pdf
PDT 89 - $1.4M - Seed - Plantee Innovations.pdfPDT 89 - $1.4M - Seed - Plantee Innovations.pdf
PDT 89 - $1.4M - Seed - Plantee Innovations.pdfHajeJanKamps
 
Developing Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, OursDeveloping Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, OursKaiNexus
 
To Create Your Own Wig Online To Create Your Own Wig Online
To Create Your Own Wig Online  To Create Your Own Wig OnlineTo Create Your Own Wig Online  To Create Your Own Wig Online
To Create Your Own Wig Online To Create Your Own Wig Onlinelng ths
 
NewBase 25 March 2024 Energy News issue - 1710 by Khaled Al Awadi_compress...
NewBase  25 March  2024  Energy News issue - 1710 by Khaled Al Awadi_compress...NewBase  25 March  2024  Energy News issue - 1710 by Khaled Al Awadi_compress...
NewBase 25 March 2024 Energy News issue - 1710 by Khaled Al Awadi_compress...Khaled Al Awadi
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxJemalSeid25
 
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxCracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxWorkforce Group
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsIntellect Design Arena Ltd
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfAnhNguyen97152
 
Intellectual Property Licensing Examples
Intellectual Property Licensing ExamplesIntellectual Property Licensing Examples
Intellectual Property Licensing Examplesamberjiles31
 
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 Building Your Personal Brand on LinkedIn - Expert Planet-  2024 Building Your Personal Brand on LinkedIn - Expert Planet-  2024
Building Your Personal Brand on LinkedIn - Expert Planet- 2024Stephan Koning
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Winbusinessin
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toumarfarooquejamali32
 
Tata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakTata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakEditores1
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access
 
Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024Borderless Access
 

Último (20)

A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.
 
7movierulz.uk
7movierulz.uk7movierulz.uk
7movierulz.uk
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
 
PDT 89 - $1.4M - Seed - Plantee Innovations.pdf
PDT 89 - $1.4M - Seed - Plantee Innovations.pdfPDT 89 - $1.4M - Seed - Plantee Innovations.pdf
PDT 89 - $1.4M - Seed - Plantee Innovations.pdf
 
Developing Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, OursDeveloping Coaching Skills: Mine, Yours, Ours
Developing Coaching Skills: Mine, Yours, Ours
 
To Create Your Own Wig Online To Create Your Own Wig Online
To Create Your Own Wig Online  To Create Your Own Wig OnlineTo Create Your Own Wig Online  To Create Your Own Wig Online
To Create Your Own Wig Online To Create Your Own Wig Online
 
NewBase 25 March 2024 Energy News issue - 1710 by Khaled Al Awadi_compress...
NewBase  25 March  2024  Energy News issue - 1710 by Khaled Al Awadi_compress...NewBase  25 March  2024  Energy News issue - 1710 by Khaled Al Awadi_compress...
NewBase 25 March 2024 Energy News issue - 1710 by Khaled Al Awadi_compress...
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptx
 
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxCracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking Applications
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
 
Intellectual Property Licensing Examples
Intellectual Property Licensing ExamplesIntellectual Property Licensing Examples
Intellectual Property Licensing Examples
 
WAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdfWAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdf
 
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 Building Your Personal Brand on LinkedIn - Expert Planet-  2024 Building Your Personal Brand on LinkedIn - Expert Planet-  2024
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb to
 
Tata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakTata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerak
 
Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024Borderless Access - Global B2B Panel book-unlock 2024
Borderless Access - Global B2B Panel book-unlock 2024
 
Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024Borderless Access - Global Panel book-unlock 2024
Borderless Access - Global Panel book-unlock 2024
 

Nurtures evaluation report 2012

  • 1. NURTURES Evaluation Report: Year 1 Completed by: Acumen Research and Evaluation, LLC April 10, 2012 On the behalf of:
  • 2. 1 Table of Contents I. Executive Summary 2 II. Introduction 3 a. Evaluation Logic Model 3 III. Observations of the Planning Process 3 a. Tangible Outcomes 3 b. Group Progression 4 c. Group Dynamics 4 IV. Interviews with Planning Team 5 a. Interview Responses 5 b. Interview Summary 7 V. Master Teacher Recruiting Materials and Results 7 VI. Training Phase Assessment Tools 8 VII. Conclusion and Recommendations 8 VIII. References 8 This report summarizes the activities and finding of the evaluators of the NSF MSP project at The University of Toledo entitled NURTURES from July 2011 through March 2012.
  • 3. 2 I. Executive Summary This report examines the Planning Phase (Phase I) of the NURTURES project. The project focus is to develop a complementary science learning environment for grades pre-K -3. There are three phases to this project: Planning Phase (Year 1), Demonstration/Training Phase (Year 2), and Implementation Phase (Years 3-5). This report examines the Planning Phase. Evaluation of the Planning Phase consisted of comparing project outcome attainment with proposal timeline and observing and reviewing the Planning Team’s group process procedures and outcomes. Data were collected through review of project documents such as hiring paperwork, direct observation of the retreat, examination of Summer Institutes, personal interviews with members of Planning Team, and examination of recruiting materials and efforts. Project documentation showed attainment of all Year 1 outcomes within a timely manner. All employees have been hired and trained (Master Teachers are still in training but that is within the timeframe). A detailed timeline of activities for this summer and Year 2 has been developed. Evaluators made direct observations of several of the planning team’s weekly meetings. Tangible outcomes, group progress, and group dynamics were examined. To date, the team has achieved the tangible outcomes expected at this point. Interactions between members continually improved over the course of the year as group members became more familiar with one another. As far as group dynamics, the group works as a cohesive entity towards mutual goals. The evaluators conducted personal interviews with members of the planning team to further examine the group process—particularly from each member’s perspective. Again, strong evidence of mutual respect and an atmosphere conducive to positive interaction and progress was provided. While a few felt the meetings to be a bit long, all appreciated that they were well- organized and little time was wasted. Most members of the team identified more closely with their own specific role within the project as opposed to identifying with the project as a whole. Master Teachers were recruited and hired. While the goal was to recruit teachers from the Toledo Review and Alternative Compensation System (TRACS) within the Toledo Public Schools, only two of the Master Teachers are TRACS teachers. However, the group represents teachers from grades pre-K through 3 with experience teaching science and coaching or providing other teachers with professional development. The evaluators have developed interview protocols for Master Teachers to be used during the Summer Institute, have worked with content faculty in the development of pretest/posttest instruments to measure content gain, and have developed a Teacher Inquiry Behavior Protocol and Parent Science Activity Survey to be piloted in Master Teacher classrooms during Year 2. Overall, Phase I concludes having met its objectives. The groundwork for a successful MSP has been laid and will be piloted during Year 2.
  • 4. 3 II. Introduction The evaluation plan for the project was created by Acumen Research and Evaluation, LLC to determine the impact of NURTURES on children’s interest in and curiosity about science. The NURTURES grant evaluation logic model includes three phases of the evaluation plan matched to the three phases of the project: planning, training, and implementation. The grant is currently in year 1 and Phase 1 (Year 1 ends June 30, 2012). Evaluation of the Planning Phase consisted of observing and reviewing the Planning Team’s group process procedures and outcomes. Data were collected through direct observation of the retreat, examination of Summer Institutes, personal interviews with members of Planning Team, and examination of recruiting materials and efforts. In addition, the evaluators determined existing instruments suitable for measuring Phase 2 (Training/Demonstration Phase) of the project and developed instruments when appropriate measures did not exist. A. Planning Phase Evaluation Model The Planning Phase Evaluation Model was designed to map out the assessment of the grant activities as it pertains to the goals and outcomes of the first year of the project (Table 1). During Year 1, the Planning Phase, the evaluation team had five major responsibilities—(1) observe the planning process and resulting group dynamics of the Planning Team, (2) review schedules for Summer Institute and Demonstration Phase, (3) interview Planning Team members, (4) observe the planning retreat, and (5) review recruiting materials for the Master Teacher positions. Not listed in the model but of equal importance was the compilation and development of assessments to use during Phase 2. To date, the evaluators have completed observations of the planning process, interviewed the planning team, examined recruiting materials for the Master Teachers, and compiled existing and developed assessments for Year 2. Observation of the Retreat and review of the Summer Institute schedule will occur in May/June 2012 and reported in Year 2 Annual Report. III. Observations of the Planning Process A member of the evaluation team frequently attended planning meetings during the past year. The purpose was not to provide input but rather to observe the group dynamics and planning process. A teamwork protocol was used to ascertain the level of collaboration and teamwork that is present within the group (Schultz, Israel, and Lance, 2002). Several themes emerged from the meetings: Tangible outcomes, group progression, and group dynamics. A. Tangible outcomes The planning team, project staff, and graduate assistants have been developing a framework for the 2012 Summer Institute. The framework will be used to educate Master Teachers as to how to teach their peers science inquiry based strategies in the classroom (in preparation for Summer Institute 2013). To assist with the creation of the tool, graduate students have been researching and structuring how science inquiry content is currently being taught in the classroom. Other strategies are being utilized to aid with the development of lesson content. Reverse engineering is a well-known engineering strategy that is being used to develop what the Summer Institute content will be for 2013. The group has been working towards the creation of example lessons to illustrate and better understand how adults learn. In addition to researching
  • 5. 4 Table 1: NURTURES Planning Phase Evaluation Model Planning Phase—The foundation for all outcomes and overall project goal Method Variable Measured Purpose Progress to date Observation of planning process Group process and achievement of stated outcomes To determine whether the Planning Team is working together towards project goals Completed Observation of retreat Group dynamics To determine whether format is conducive to collaboration Will be completed in June 2012 Interview with planning team members Participants’ perception of success of retreat To determine degree to which teamwork goals are being met Completed Review of Summer institute schedule Planning for summer completed To determine whether summer program is thoroughly planned Will be completed by June 2012 Examination of recruiting materials and strategies Planning for recruitment completed To determine whether project is prepared for recruitment Completed teacher behavior development (with regards to adult learning), the group has been addressing other aspects of the grant such as what exemplar informal activities should be included for the parents and children. In addition to science content, the pedagogy methods and literacy strategies that will be implemented into the new science content have been discussed. The group has decided to develop a month long training session to develop inquiry based science content in science lessons. The team has also been researching ways in which children think and ways to chart what inquiry behaviors need to occur to illustrate learning is occurring. A smaller group within the group has been developing a coding system for teacher science inquiry behaviors. To ensure that the tool is inclusive, both early childhood and engineering graduate students are developing the behavior codes for the assessment. B. Group Progression Over the past year, the planning team has been meeting weekly. The group has been able to brainstorm and share ideas freely. Initially, the group spent quite a bit of time determining the types of behaviors good science teachers exhibit when teaching science. During the second half of the year, specific tasks were assigned to some members and smaller groups were formed. With the groundwork for the first Summer Institute well underway, sights have turned to the examination the new national science standards to determine what science learning consists of as well as the hallmarks of a Master Teacher. C. Group Dynamics The group worked cohesively and group members were comfortable sharing perspectives and eliciting feedback. While members of the group did not always agree, opposing perspectives were presented cordially and no one member of the group dominated the conversation. Also,
  • 6. 5 team leaders were able to successfully guide their workgroups independent of the larger group. The weekly meetings were conducive to removing barriers as they allowed group members to work through their problems and, when problems did occur, group members were able to redirect behavior to achieve desired outcomes. The diversity of disciplines provided a rich foundation for group members to voice their opinions and work together. IV. Interviews with Planning Team In March 2012 the evaluators conducted personal interviews with members of the Leadership team to examine group processes—to assess the level of teamwork and collaboration present within the team. The Leadership team is comprised of six members including the PI, the co-PIs, and some senior personnel. The interviews consisted of questions concerning their main role on the project, group process, satisfaction with progress, and content development. Each question asked is provided below along with a summary of the responses of the six interviewees. A. Interview Responses 1. First tell me what you been doing for the NURTURES project over the past year? The majority of the team described personal responsibilities concerning the project such as providing advice, acting as a resource, developing a coding scheme, and hiring and training personnel. Only two of the members provided a rich description of the group work as a whole and the progress that has been made. The difference is in the interpretation of the word “you”. The majority of respondents interpreted “you” to mean themselves and the other two identified with the group to the extent that “you” has taken on the collective definition of “your group” or “you all”. 2. With whom do you work with to plan? Four of the six mentioned the entire planning team. Two indicated specific people—one mentioned a member of the staff and the other worked mostly with graduate assistants. 3. How well does the group work together? They all agreed that the group works well together and a few noted that the membership consists of some very different backgrounds. In fact, several were surprised at how well the members of the group get along. Only one respondent provided examples of their harmony. This respondent felt comfortable speaking up when unsure and indicated that no matter how little a group member knew about a subject, he or she could express an opinion and the group respected it. 4. When problems arise, how well does the group work together to solve them? Can you provide a specific example? The definition of a “problem” seemed to vary among the respondents. In most cases, a problem was equated with either a misunderstanding (a member not understanding what someone else is trying to say) or a disagreement among members of the group. Responses to this item were vague. Most said that disagreements were discussed and resolved. One member felt that the group had not experienced any problems to date. Only one member was willing to provide an example of a problem and indicated that the objection one member had to a situation was not relevant to their discussion. 5. What are the major issues you are involved in?
  • 7. 6 All but one member had a clear idea of what they contribute to the group and which responsibilities and to which issues they contribute. One member from the five who had a clear idea felt that only recently has that contribution been realized and is now happy to have a place within the group. 6. How would you describe the rate of progress in which the group is making in dealing with major issues identified? Three of the respondents felt progress to be satisfactory and one of the three mentioned the value of taking their time at the beginning to set the groundwork. Two others felt progress during the first six months to be slow but are now satisfied with the speed at which the group is moving towards meeting its timetable. These two did not, however, object openly to their perceived slow rate of progress during the early months of the project. One respondent did not answer this question. 7. Where is your group as far as achieving its goals? Three members provided specific goals achieved that related to their role in the project. Two indicated progress towards project level goals and one was not sure “what the current goals are.” While the remaining five felt progress to be satisfactory, only the two who identified project goals, responded to the question accurately (your group rather than you). 8. How does the group work through times when they are bogged down? One respondent was not aware of any roadblocks. The other five agreed that when the group gets stymied they either table the topic till a future meeting or try to work through it. One member felt that early on the group was not cohesive enough to work through rough spots but now it is at a place where there is mutual respect in spite of the fact that the group has grown in size. 9. What were your initial expectations of the planning meetings? Only the PI had a clear idea as to what the planning meetings would be like. Others either had no expectations, vague expectations or a very different idea of what was to occur. 10. Have your initial expectations changed? Not surprisingly, all but the PI has changed his/her view of the meeting expectations. Of course those who had not made any assumptions about the meeting now have a clear understanding of their function within the project. Those with vague ideas have a more precise understanding and one member was surprised that the meetings were as democratic as they are and that everyone seems to have input as to the direction the meetings take. 11. Can you tell me effective attributes that form the planning meetings? The respondents mentioned several positive attributes. Most frequently they noted the diverse group, mutual respect, and the open atmosphere. Also mentioned was that the meetings were well organized. One member felt the meetings run too long. 12. What do you think the end results of the planning meetings will be?
  • 8. 7 All felt the Planning Phase will end with a strong Summer Institute design. One member felt that the Planning Phase will not be completed at the end of the first year but will continue into the Pilot Phase. 13. Do you know what to do next and are you comfortable with carrying these responsibilities? Five of the six articulated future responsibilities and felt comfortable with their responsibilities for the summer and Year 2. One was not sure of what was expected of him/her. B. Summary of Interviews: Responses from the six members of the planning team indicated mutual respect and an atmosphere conducive to positive interaction and progress. While some felt the initial meetings to be drawn out, all agreed that recently they have been more productive and most (5) have a clear vision of their role in the project for Phase II. All felt the meetings have been well organized and appreciated the time taken to review agendas and provide background materials in advance. While all members see themselves as a contributing piece to the project, most do not associate themselves with the planning group as such. In other words, when answering questions, the majority of comments centered upon the individual (I will/have done this) as opposed to work the group as a whole has accomplished. Only two of the members spoke from the perspective of the group rather than their individual contribution and experience. V. Master Teacher Recruiting Materials and Results Prior to recruiting Master Teachers, the NURTURES planning team held a meeting in which the expectations and commitments of the Master Teacher position were detailed. Based upon previous experience with Toledo Public Schools, it was determined that recruitment efforts would focus on selecting teachers who are in the Toledo Review and Alternative Compensation System (TRACS). The goal of TRACS is to promote teacher quality while improving student academic performance. TRACS focuses on ongoing teacher professional development that targets specific student academic and school improvement needs including effective teaching and learning and the retention of the most accomplished teachers through acknowledging and rewarding teaching excellence. TRACS maximizes the talents of recognized teacher leaders by assigning additional assignments or leadership roles and/or by placing them in high needs schools or in challenging teaching assignments. If the project was unable to recruit teachers from TRACS, then STEM teachers with strong recommendations from principals and/or other school leadership would be considered. The interview questions for Master Teacher were intended to identify the underlying philosophy of the potential Master Teachers. The interview team consisted of the PI/co-PI’s and project staff. Questions focused on the interviewee’s ideology as it relates to his/her teaching philosophy. Other questions focused on teaching practices. For example, “Describe the best science lesson you ever taught; what are some ways you integrate STEM into your everyday curriculum?” The questions were designed to probe for information about the teacher’s flexibility in dealing with the constant changes that occur in the classroom when teaching using inquiry-based strategies. To date, all six of the Master Teachers have been selected. They represent preschool through grade 3 and range in years of teaching from 6 to 29 with an average of 16.5 years. While only two are TRACS teachers, a third has been a science support teacher in
  • 9. 8 Toledo Public Schools for several years. The two TRACS teachers, both teachers of first grade, have served as a Peer Math Coach and a Peer Literacy Coach for their district. VI. Training Phase Assessment Tools The original evaluation plan called for the use of the Science Teacher Self Efficacy Instrument (STEBI) (Riggs and Enochs, 1990) the Science Teacher Ideological Preference Scale (STIPS) (Jones and Harty, 1978; Gado, 2005), focus group interviews, content assessments, and classroom observations. In addition, a leadership survey used in the NSF MSP LEADERS will be adapted to assess Master Teacher leadership skills development and growth. Data will be collected from Master Teachers during Year 2 to establish a baseline to measure Master Teacher growth over time. Rather than implementing the Horizon Inside the Classroom Observation Protocol, a classroom observation rubric that examines teacher inquiry behaviors has been developed and will be discussed with advisor Shymansky when he visits the project this summer. The Teacher Inquiry Behavior Protocol was devised to assess the level of science inquiry in teaching behaviors. The protocol rates teachers on each of the following: questioning, defining problems, engagement in evidence-based discourse, and developing and using STEM concepts. The decision to change the observation tool was based upon a desire to more closely match the instrument with the intentions of the project. It will be piloted in the Master Teacher classrooms during Year 2 to establish evidence of reliability and validity. Another tool developed this year and piloted next year is the Parent Science Activity Survey. This instrument is designed to examine parents’ opinions of change in science interest in their children as well as their own proclivity towards helping their child learn more about science. VII. Conclusions In summary, the Planning Phase has met its stated objectives. Personnel have been hired; detailed planning for the first Summer Institute is well on its way; and members of the project team work together towards a common goal. Substantial groundwork has been laid for Phase II of the project and project personnel appear eager to enter into this next phase. VIII. References Gado, I. (2005). Determinants of k-2 school teachers' orientation towards inquiry-based science activities: A mixed method study. International Journal of Science and Mathematics Education, 3, 511-539. Riggs, I., & Enochs, L. (1990). Towards the development of an elementary teacher’s science teaching efficacy belief instrument. Science Education, 74, 625-637. Schulz, A.J., Israel, B.A., & Lantz, P. (2002). Instrument for evaluating dimensions of group dynamics within community-based participatory research partnerships. Evaluation and Program Planning, 26, 249-262.