Program Evaluation Plan for Expanding Possibilities Program

1. Expanding Possibilities Program Evaluation Plan for 2009- 2011 Presented by Rob Rice, Ph.D., LMHC September 24th, 2009

2. Rob Rice, Ph.D., LMHC Ph.D., University of Rochester, April 2009 Director for Easter Seals New York, 2004-2009 Currently a full time faculty member in the St. John Fisher College Masters Program in Mental Health Counseling Excited about the Expanding Possibilities Project!

3. Program Evaluation Model for Expanding Possibilities Martin, L. & Kettner, P. (2009). Measuring the Performance of Human Service Programs, 2nd Ed. SAGE Publishing. Basic Systems Model Expanded Systems Model Inputs Process (treatment plan or service delivery methodology) Output Quality (e.g., responsiveness, strength-based, etc.) Outcomes (results) Feedback

4. Output Performance Measures Based on the following 2 grant priorities: The percentage of relevant school personnel, community based organizations, and families served by the grant trained to make appropriate referrals. Provide technical assistance and consultation to each school, community based organizations, and families participating in the program.

5. Output Measure #1 Units of Service: One completed training of a school’s personnel Completion of Service: 1 year Mid-grant objective: 50% of the relevant school mental health staff is trained to make appropriate referrals to mental health services End of grant objective: 95% of the relevant school mental health staff is trained to make appropriate referrals to mental health services. Method of Evaluation: The director will collect and report data to the program evaluator on a monthly basis

6. Output Measure #2 Units of Service: # of schools/community based organizations that receive technical assistance Completion of Service: 1 year Mid-grant objective: 18 of the 27 schools and five community based organizations have received technical assistance and consultation from the program. End of grant objective: all 27 schools and at least 8 community based organizations Method of Evaluation: The director will collect and report data to the program evaluator on a monthly basis

7. Quality Performance Measures Based on the following grant priority: All services will be provided in a linguistically appropriate and culturally competent method.

8. Quality Measures Quality Dimension: Cultural Competence Mid grant objective: The cultural competence of the program will increase by 8% End of grant objective: The cultural competence of the program will increase by 15% Method of Evaluation: Likert based survey will be provided to families, schools, and mental health agencies This survey will also endeavor to capture valuable data related to other quality dimensions important to the program (e.g., strength-based, solution-focused)

9. Outcome Performance Measures Based on the following 3 grant priorities: The percentage of schools served by the grant that have comprehensive, detailed linkage protocols in place. Improve linkages with all mental health community based organizations and government entities in Seneca and Tompkins County. The program’s effectiveness will be evaluated as to the effectiveness of the program in increasing student access to mental health services and recommendations will be made to the Secretary of Education about the sustainability of the program.

10. Outcome Measure #1 Linkage Protocols Objective: All 27 TST BOCES and component schools will have linkage protocols in place. Method of Evaluation: The existence of linkage protocols will be verified with the principle of each school. Relevant school staff will receive a Likert based survey intended to measure dimensions such as collaboration, accessibility , ease, responsiveness, effectiveness, sustainability, leadership, policies, resources, political climate, connectedness, community understanding, stigma, and other indicators. The increase in reported results will indicate the improvement in the system.

11. Outcome Measure #2 Student Access Method of Evaluation: referrals for mental health services will be tracked by the school staff appointed to serve as the point person for the Expanding Possibilities Program. This data will be communicated to the Program Evaluator monthly. The independent evaluator will provide a baseline, first year, and second year evaluation of the program.

12. Exactly What We Need From You Schools must identify appropriate staff to complete Likertbased survey intended to measure dimensions such as collaboration, accessibility , ease, responsiveness, effectiveness, sustainability, leadership, policies, resources, political climate, connectedness, community understanding, stigma, and other indicators. At a recent meeting, the Expanding Possibilities Team suggested using the newly formed linkage committees for this purpose. (Outcome Measure #1) TIMEFRAME: 12/09, 9/10, 9/11 School point people must begin tracking referrals for mental health services and communicate this data to the Program Evaluator monthly (Outcome Measure #2) TIMEFRAME: begin reporting 11/09

13. How to Reach Me Cell (585) 727-4811 Office (585) 385-7318 E-mail: rrice@sjfc.edu