SlideShare una empresa de Scribd logo
1 de 48
Keep It Clean Partnership




           Measuring Embraced
           Behavior To Evaluate
           Effectiveness of MS4
           Education and Outreach
           StormCon 2012
           Session P-35
Keep It Clean Partners
   Boulder County - 22,469 /44,000
   City of Boulder – 97,385
   Town of Erie – 18,135
   City of Lafayette – 24,453
   City of Longmont – 86,270
   City of Louisville – 18,376
   Town of Superior – 12,483
Keep It Clean Partnership
The Watershed
Audiences and Behaviors

   Adults
   School Children
   Construction
   Businesses
   Municipal employees and contractors
Partners for a Clean
             Environment
   Restaurants
   Vehicle Service
   Gas Stations
   Retail
   Facility Managers
   Landscapers
   Pressure Washers
   Municipal
Questions about Effectiveness

   Questions for the Consultant
       Are KICP shared programs effective?
       Is KICP program administration efficient?
       How can KICP improve water quality and
        measure it?
Recommendations for Effectiveness
   Update IGA and Bylaws
   Reinforce watershed approach
   Leverage brand awareness
       “Keep it Clean”
   Define priority pollutants and the behaviors
    that generate them
   Measure “Embraced Behavior”
Embraced Behavior Proposal

   “Be the expert about the best practices.” ~
    Greg Davis – EPA Region 8
   “Benefits can be monetized for MEP.” ~ John
    Sorensen, AMEC
   “Put the “Point” back into Non-Point Source
    pollution education.” ~ Jesse Poore, Felsburg
    Holt & Ullevig
The Trouble with
              “Effectiveness”
   “I don’t care if my program is effective. I just
    want to be in compliance.” ~ MS4 Operator
   If MS4 Education and Outreach Programs are
    supposed to accomplish “NPDES” (Pollutant
    Discharge Elimination) …
   Can we correlate Education to Water Quality?
The Trouble with
             “Effectiveness”
   “We are ‘flying blind’ or, probably, by the
    seat of our pants, when it comes to making
    decisions about how best to address water
    quality problems and allocate our limited
    resources for cleanup, pollution prevention
    and restoration.” ~ G. Tracy Mehan, former
    EPA Assistant Administrator
The Trouble with
             “Effectiveness”
   “Without assessing the effectiveness of the
    stormwater management program the
    permittee will not know which parts of the
    program need to be modified to protect and/or
    improve water quality and instead will
    essentially be operating blindly.” EPA 2010
    Permit Improvement Guide
The Trouble with
“Effectiveness”




          FR Vol 65 No. 235 Part II(H)(3)(a)(iii)
The Trouble with
            “Effectiveness”
   Peer Research



           KPCRC, Washington                 LCSC, South Carolina
                  Mindy Fohn                    Synithia Williams




                               MRSWMP, California
                                 Heidi Niggemeyer
The Trouble with
              “Effectiveness”
   MRSWMP - Co-permittees, evaluation
    structure provided, benchmarks defined

   KPCRC - State campaign, local economics,
    focus on observing behavior

   LCSC - University facilitated education, local
    input, targets for audience and pollutants
The Trouble with
                   “Effectiveness”
   Literature Research
   2001 Study of Six California MS4 Costs
   2010 Tracked KICP Expenditures
          Category              Average         KICP
      Budget                   $2.5 Million   $1.7 Million
      Management                $375,000       $362,000
      Education                 $120,000       $48,500
               Proportion         4.0%           2.9%
                  Per Person      $0.46          $0.18
                  Per House       $1.24          $0.48
Effective Education and
        Outreach
Effective Education and
             Outreach




Dorthea Lange – 1938   Arthur Rothstein – 1936
Effective Education and
             Outreach




Source: CNN 8/16/2012
Effective Education and
            Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
           Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
Effective Education and
                   Outreach




Source: Jesse Poore - Felsburg Holt & Ullevig
What Gets Measured Matters

   Typical Permit Language:
   “Within [insert deadline, e.g., within the permit
    term], the permittee must assess changes in
    public awareness and behavior resulting
    from the implementation of the program such
    as using a statistically valid survey and modify
    the education/outreach program accordingly.”
What Gets Measured Matters
   Permits and guidance typically promote
    measuring percent change in behavior.
                         BUT
   Does not correlate behavior to a loading rate
   Rarely identifies a valid start point or a
    satisfactory end point
   Ability to prioritize limited dollars is minimized
What Gets Measured Matters
   Permit Effectiveness ≠ Behavior Effectiveness

   Measuring evaluations and surveys against
    learning objectives ≠ Behavior Effectiveness
Evaluate Cost Effectiveness
   Recent webcasts provide helpful techniques
   WEF: Evaluating Stormwater Outreach
   EPA: Conducting Effective Stormwater Outreach
   Neither promoted direct measurements to
    supplement surveys and raised awareness
   Neither discussed pollutant loading associated
    with behaviors
Community Based Social Marketing
   Approach by McKenzie-Mohr and Smith
       Identify barriers and benefits
       Define tools to change behavior
       Pilot the approach
       Evaluate results
   Emphasize direct measurements
   Supported by surveys and focus groups
Evaluate Cost Effectiveness
Evaluate Cost Effectiveness
Evaluate Cost Effectiveness
Important Question #1

   “If you can’t measure embraced
    behavior, how are you going to measure
    percent behavior change?”
   Hint: You can’t!
Measuring Embraced Behavior

   Identify behaviors for target pollutants
   Define target behavior or indicator
   Establish target for Embraced Behavior
   Commit to observing behavior
   Compare observed to recommended
   Communicate effectiveness
Measuring Embraced Behavior
Important Question #2

   “If you can’t establish a load associated
    with a behavior, how are you going to
    measure improvement in water quality?”
   Hint: Urban runoff loading based on
    behavior is uncharted territory
Behaviors Impact TMDLs

   # of Restaurants Screened: 38
   Number Grease Bins Open: 5      (13.2%)
   Non-Conforming Bins:            1     (2.6%)
   No Bin Seen:                4   (10.5%)
   Surface Area Stain
    None/Covered:          20   (52.6%)
    Slight:                10   (26.3%)
    Significant:           4    (10.5%)
Behaviors Impact TMDLs



None/Covered = X
                      Slight = X lbs./Yr.
    lbs./Yr   .




   Significant = X   Non-Conforming = X
      lbs./Yr.            lbs./Yr.
Embraced Behavior Proposal

   Establish desired social, institutional behaviors
   Centralize “Embraced Behavior” conversation
   Discuss best methods for direct measurement
   Research pollutant loading associated with
    undesired behaviors
   International Stormwater Behavior Database
Presenters
      KeepitCleanPartnership.org
     Janice Lopitz 303-441-1439

        Felsburg Holt & Ullevig
  Jesse Poore - jesse.poore@fhueng.com
            @Jesse_W_Poore

             CP-Compliance
Carrie Powers - cpowers@cp-compliance.com
Outcome of Permit Language
Measuring Embraced Behavior
Outcome of Permit Language

Más contenido relacionado

La actualidad más candente

Measuring gender transformative change: Frameworks, Characteristics, Methodol...
Measuring gender transformative change: Frameworks, Characteristics, Methodol...Measuring gender transformative change: Frameworks, Characteristics, Methodol...
Measuring gender transformative change: Frameworks, Characteristics, Methodol...Cynthia McDougall
 
Rise pp ace 2015 1
Rise pp ace 2015 1Rise pp ace 2015 1
Rise pp ace 2015 1Mike Mel
 
Richard Crespin - CollaborateUP
Richard Crespin - CollaborateUPRichard Crespin - CollaborateUP
Richard Crespin - CollaborateUPitjkg
 
Weekend in Boca VII - Richard Crespin - "CollaborateUP"
Weekend in Boca VII - Richard Crespin - "CollaborateUP"Weekend in Boca VII - Richard Crespin - "CollaborateUP"
Weekend in Boca VII - Richard Crespin - "CollaborateUP"OfficeDepotFoundation
 
The Quest for Impact: The Transformation of Research from a Traditional to a ...
The Quest for Impact: The Transformation of Research from a Traditional to a ...The Quest for Impact: The Transformation of Research from a Traditional to a ...
The Quest for Impact: The Transformation of Research from a Traditional to a ...Global Livestock CRSP
 
Patients as agents of change: bridging the gap between "them" and "us"
Patients as agents of change: bridging the gap between "them" and "us"Patients as agents of change: bridging the gap between "them" and "us"
Patients as agents of change: bridging the gap between "them" and "us"Helen Bevan
 
Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...
Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...
Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...Sustainable Summits Initiative
 
Raising Awareness and Learning Practices of Citizens for Energy Savings
Raising Awareness and Learning Practices of Citizens for Energy SavingsRaising Awareness and Learning Practices of Citizens for Energy Savings
Raising Awareness and Learning Practices of Citizens for Energy SavingsAndreas Kamilaris
 
Stories from the Field: Building a Transformative Partnership with Families a...
Stories from the Field: Building a Transformative Partnership with Families a...Stories from the Field: Building a Transformative Partnership with Families a...
Stories from the Field: Building a Transformative Partnership with Families a...LucilePackardFoundation
 
Gender transformative change along the capture fishery value chain: Panel evi...
Gender transformative change along the capture fishery value chain: Panel evi...Gender transformative change along the capture fishery value chain: Panel evi...
Gender transformative change along the capture fishery value chain: Panel evi...WorldFish
 
How to create change that sticks and spreads
How to create change that sticks and spreadsHow to create change that sticks and spreads
How to create change that sticks and spreadsHelen Bevan
 
2013 murray presentation
2013 murray presentation2013 murray presentation
2013 murray presentationLisa Dickson
 

La actualidad más candente (18)

Measuring gender transformative change: Frameworks, Characteristics, Methodol...
Measuring gender transformative change: Frameworks, Characteristics, Methodol...Measuring gender transformative change: Frameworks, Characteristics, Methodol...
Measuring gender transformative change: Frameworks, Characteristics, Methodol...
 
2.1: Basics of Rapid Re-Housing
2.1: Basics of Rapid Re-Housing2.1: Basics of Rapid Re-Housing
2.1: Basics of Rapid Re-Housing
 
Rise pp ace 2015 1
Rise pp ace 2015 1Rise pp ace 2015 1
Rise pp ace 2015 1
 
Richard Crespin - CollaborateUP
Richard Crespin - CollaborateUPRichard Crespin - CollaborateUP
Richard Crespin - CollaborateUP
 
Weekend in Boca VII - Richard Crespin - "CollaborateUP"
Weekend in Boca VII - Richard Crespin - "CollaborateUP"Weekend in Boca VII - Richard Crespin - "CollaborateUP"
Weekend in Boca VII - Richard Crespin - "CollaborateUP"
 
The Quest for Impact: The Transformation of Research from a Traditional to a ...
The Quest for Impact: The Transformation of Research from a Traditional to a ...The Quest for Impact: The Transformation of Research from a Traditional to a ...
The Quest for Impact: The Transformation of Research from a Traditional to a ...
 
Frontline Practice within Housing First Programs
Frontline Practice within Housing First ProgramsFrontline Practice within Housing First Programs
Frontline Practice within Housing First Programs
 
Patients as agents of change: bridging the gap between "them" and "us"
Patients as agents of change: bridging the gap between "them" and "us"Patients as agents of change: bridging the gap between "them" and "us"
Patients as agents of change: bridging the gap between "them" and "us"
 
Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...
Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...
Wade Vagias - Identifying and Testing Predictors of Compliance with Recommend...
 
DSRA
DSRADSRA
DSRA
 
Do we have a climate for change insights about adaptation planning actions in...
Do we have a climate for change insights about adaptation planning actions in...Do we have a climate for change insights about adaptation planning actions in...
Do we have a climate for change insights about adaptation planning actions in...
 
Raising Awareness and Learning Practices of Citizens for Energy Savings
Raising Awareness and Learning Practices of Citizens for Energy SavingsRaising Awareness and Learning Practices of Citizens for Energy Savings
Raising Awareness and Learning Practices of Citizens for Energy Savings
 
Stories from the Field: Building a Transformative Partnership with Families a...
Stories from the Field: Building a Transformative Partnership with Families a...Stories from the Field: Building a Transformative Partnership with Families a...
Stories from the Field: Building a Transformative Partnership with Families a...
 
Gender transformative change along the capture fishery value chain: Panel evi...
Gender transformative change along the capture fishery value chain: Panel evi...Gender transformative change along the capture fishery value chain: Panel evi...
Gender transformative change along the capture fishery value chain: Panel evi...
 
Coalition Orientation for SACADA Board Members
Coalition Orientation for SACADA Board MembersCoalition Orientation for SACADA Board Members
Coalition Orientation for SACADA Board Members
 
The Sweet spot
The Sweet spotThe Sweet spot
The Sweet spot
 
How to create change that sticks and spreads
How to create change that sticks and spreadsHow to create change that sticks and spreads
How to create change that sticks and spreads
 
2013 murray presentation
2013 murray presentation2013 murray presentation
2013 murray presentation
 

Destacado

Implementing Technology for “Effective” Environmental Compliance
Implementing Technology for “Effective” Environmental ComplianceImplementing Technology for “Effective” Environmental Compliance
Implementing Technology for “Effective” Environmental ComplianceJPoore
 
Past Present and Future Trends
Past Present and Future TrendsPast Present and Future Trends
Past Present and Future TrendsJPoore
 
Past Present And Future Trend
Past Present And Future TrendPast Present And Future Trend
Past Present And Future TrendJPoore
 
Digital Stormwater Data
Digital Stormwater DataDigital Stormwater Data
Digital Stormwater DataJPoore
 
Bioassessment Approach to MS4 Evaluation and Assessment
Bioassessment Approach to MS4 Evaluation and AssessmentBioassessment Approach to MS4 Evaluation and Assessment
Bioassessment Approach to MS4 Evaluation and AssessmentJPoore
 
1.2 developing a national river health program in australia en
1.2 developing a national river health program in australia en1.2 developing a national river health program in australia en
1.2 developing a national river health program in australia enInternational WaterCentre
 
Bioindicators ppt
Bioindicators pptBioindicators ppt
Bioindicators pptChitra Nair
 

Destacado (7)

Implementing Technology for “Effective” Environmental Compliance
Implementing Technology for “Effective” Environmental ComplianceImplementing Technology for “Effective” Environmental Compliance
Implementing Technology for “Effective” Environmental Compliance
 
Past Present and Future Trends
Past Present and Future TrendsPast Present and Future Trends
Past Present and Future Trends
 
Past Present And Future Trend
Past Present And Future TrendPast Present And Future Trend
Past Present And Future Trend
 
Digital Stormwater Data
Digital Stormwater DataDigital Stormwater Data
Digital Stormwater Data
 
Bioassessment Approach to MS4 Evaluation and Assessment
Bioassessment Approach to MS4 Evaluation and AssessmentBioassessment Approach to MS4 Evaluation and Assessment
Bioassessment Approach to MS4 Evaluation and Assessment
 
1.2 developing a national river health program in australia en
1.2 developing a national river health program in australia en1.2 developing a national river health program in australia en
1.2 developing a national river health program in australia en
 
Bioindicators ppt
Bioindicators pptBioindicators ppt
Bioindicators ppt
 

Similar a Storm Con2012

Agency Effectiveness - Essential Guidelines and Planning
Agency Effectiveness - Essential Guidelines and PlanningAgency Effectiveness - Essential Guidelines and Planning
Agency Effectiveness - Essential Guidelines and PlanningJodi Rudick
 
Results based accountability 101 20 min Australia, NSW (2007)
Results based accountability 101 20 min Australia, NSW (2007)Results based accountability 101 20 min Australia, NSW (2007)
Results based accountability 101 20 min Australia, NSW (2007)Results Leadership Group, LLC
 
Task Force Reviewand%20 Charge[1]
Task Force Reviewand%20 Charge[1]Task Force Reviewand%20 Charge[1]
Task Force Reviewand%20 Charge[1]Timothy M. Ervolina
 
Achieving Measurable Collective Impact with Results-Based Accountability - Co...
Achieving Measurable Collective Impact with Results-Based Accountability - Co...Achieving Measurable Collective Impact with Results-Based Accountability - Co...
Achieving Measurable Collective Impact with Results-Based Accountability - Co...Clear Impact
 
Tsipporah Top of FormResults of the EvaluationThe results of.docx
Tsipporah Top of FormResults of the EvaluationThe results of.docxTsipporah Top of FormResults of the EvaluationThe results of.docx
Tsipporah Top of FormResults of the EvaluationThe results of.docxwillcoxjanay
 
CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15
CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15
CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15CSV_UK
 
Pre-Conference: Evidence-based Practices for Serving Runaway and Homeless Youth
Pre-Conference: Evidence-based Practices for Serving Runaway and Homeless YouthPre-Conference: Evidence-based Practices for Serving Runaway and Homeless Youth
Pre-Conference: Evidence-based Practices for Serving Runaway and Homeless YouthNational Alliance to End Homelessness
 
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...Clear Impact
 
Ventura counseling presentation 12 1 3
Ventura counseling presentation 12 1 3Ventura counseling presentation 12 1 3
Ventura counseling presentation 12 1 3mdalgarn
 
Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020Jodi Rudick
 
Ccdo Feb25 Evaluating Cc Is
Ccdo Feb25 Evaluating Cc IsCcdo Feb25 Evaluating Cc Is
Ccdo Feb25 Evaluating Cc IsTom Kelly
 

Similar a Storm Con2012 (20)

Improving quality in the early years report
Improving quality in the early years reportImproving quality in the early years report
Improving quality in the early years report
 
Agency Effectiveness - Essential Guidelines and Planning
Agency Effectiveness - Essential Guidelines and PlanningAgency Effectiveness - Essential Guidelines and Planning
Agency Effectiveness - Essential Guidelines and Planning
 
Results based accountability 101 20 min Australia, NSW (2007)
Results based accountability 101 20 min Australia, NSW (2007)Results based accountability 101 20 min Australia, NSW (2007)
Results based accountability 101 20 min Australia, NSW (2007)
 
Task Force Reviewand%20 Charge[1]
Task Force Reviewand%20 Charge[1]Task Force Reviewand%20 Charge[1]
Task Force Reviewand%20 Charge[1]
 
Achieving Measurable Collective Impact with Results-Based Accountability - Co...
Achieving Measurable Collective Impact with Results-Based Accountability - Co...Achieving Measurable Collective Impact with Results-Based Accountability - Co...
Achieving Measurable Collective Impact with Results-Based Accountability - Co...
 
Makin' chli
Makin' chliMakin' chli
Makin' chli
 
Community Engagement Approaches for Active Transportation and Equity
Community Engagement Approaches for Active Transportation and EquityCommunity Engagement Approaches for Active Transportation and Equity
Community Engagement Approaches for Active Transportation and Equity
 
Tsipporah Top of FormResults of the EvaluationThe results of.docx
Tsipporah Top of FormResults of the EvaluationThe results of.docxTsipporah Top of FormResults of the EvaluationThe results of.docx
Tsipporah Top of FormResults of the EvaluationThe results of.docx
 
CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15
CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15
CSV NNVIA Measuring Impact of Volunteering event - Iona Joy - 27.03.15
 
An Innovative Planning Framework: Building Collective Impact
An Innovative Planning Framework: Building Collective ImpactAn Innovative Planning Framework: Building Collective Impact
An Innovative Planning Framework: Building Collective Impact
 
Agriculture and water quality in the midwestern USA
Agriculture and water quality in the midwestern USAAgriculture and water quality in the midwestern USA
Agriculture and water quality in the midwestern USA
 
Pre-Conference: Evidence-based Practices for Serving Runaway and Homeless Youth
Pre-Conference: Evidence-based Practices for Serving Runaway and Homeless YouthPre-Conference: Evidence-based Practices for Serving Runaway and Homeless Youth
Pre-Conference: Evidence-based Practices for Serving Runaway and Homeless Youth
 
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
The Power of ABCD and Results-Based Accountability for Greater Impact and Res...
 
Ventura counseling presentation 12 1 3
Ventura counseling presentation 12 1 3Ventura counseling presentation 12 1 3
Ventura counseling presentation 12 1 3
 
Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020
 
Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020
 
Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020Innovations in Integrating Quality of Life Elements - 2020
Innovations in Integrating Quality of Life Elements - 2020
 
Ccdo Feb25 Evaluating Cc Is
Ccdo Feb25 Evaluating Cc IsCcdo Feb25 Evaluating Cc Is
Ccdo Feb25 Evaluating Cc Is
 
Addressing Obesity In The Latino Community Through Community-based Advocacy I...
Addressing Obesity In The Latino Community Through Community-based Advocacy I...Addressing Obesity In The Latino Community Through Community-based Advocacy I...
Addressing Obesity In The Latino Community Through Community-based Advocacy I...
 
Policy Toolkit
Policy ToolkitPolicy Toolkit
Policy Toolkit
 

Storm Con2012

  • 1. Keep It Clean Partnership Measuring Embraced Behavior To Evaluate Effectiveness of MS4 Education and Outreach StormCon 2012 Session P-35
  • 2. Keep It Clean Partners  Boulder County - 22,469 /44,000  City of Boulder – 97,385  Town of Erie – 18,135  City of Lafayette – 24,453  City of Longmont – 86,270  City of Louisville – 18,376  Town of Superior – 12,483
  • 3. Keep It Clean Partnership
  • 5. Audiences and Behaviors  Adults  School Children  Construction  Businesses  Municipal employees and contractors
  • 6. Partners for a Clean Environment  Restaurants  Vehicle Service  Gas Stations  Retail  Facility Managers  Landscapers  Pressure Washers  Municipal
  • 7. Questions about Effectiveness  Questions for the Consultant  Are KICP shared programs effective?  Is KICP program administration efficient?  How can KICP improve water quality and measure it?
  • 8. Recommendations for Effectiveness  Update IGA and Bylaws  Reinforce watershed approach  Leverage brand awareness  “Keep it Clean”  Define priority pollutants and the behaviors that generate them  Measure “Embraced Behavior”
  • 9. Embraced Behavior Proposal  “Be the expert about the best practices.” ~ Greg Davis – EPA Region 8  “Benefits can be monetized for MEP.” ~ John Sorensen, AMEC  “Put the “Point” back into Non-Point Source pollution education.” ~ Jesse Poore, Felsburg Holt & Ullevig
  • 10. The Trouble with “Effectiveness”  “I don’t care if my program is effective. I just want to be in compliance.” ~ MS4 Operator  If MS4 Education and Outreach Programs are supposed to accomplish “NPDES” (Pollutant Discharge Elimination) …  Can we correlate Education to Water Quality?
  • 11. The Trouble with “Effectiveness”  “We are ‘flying blind’ or, probably, by the seat of our pants, when it comes to making decisions about how best to address water quality problems and allocate our limited resources for cleanup, pollution prevention and restoration.” ~ G. Tracy Mehan, former EPA Assistant Administrator
  • 12. The Trouble with “Effectiveness”  “Without assessing the effectiveness of the stormwater management program the permittee will not know which parts of the program need to be modified to protect and/or improve water quality and instead will essentially be operating blindly.” EPA 2010 Permit Improvement Guide
  • 13. The Trouble with “Effectiveness” FR Vol 65 No. 235 Part II(H)(3)(a)(iii)
  • 14. The Trouble with “Effectiveness”  Peer Research KPCRC, Washington LCSC, South Carolina Mindy Fohn Synithia Williams MRSWMP, California Heidi Niggemeyer
  • 15. The Trouble with “Effectiveness”  MRSWMP - Co-permittees, evaluation structure provided, benchmarks defined  KPCRC - State campaign, local economics, focus on observing behavior  LCSC - University facilitated education, local input, targets for audience and pollutants
  • 16. The Trouble with “Effectiveness”  Literature Research  2001 Study of Six California MS4 Costs  2010 Tracked KICP Expenditures Category Average KICP Budget $2.5 Million $1.7 Million Management $375,000 $362,000 Education $120,000 $48,500 Proportion 4.0% 2.9% Per Person $0.46 $0.18 Per House $1.24 $0.48
  • 18. Effective Education and Outreach Dorthea Lange – 1938 Arthur Rothstein – 1936
  • 19. Effective Education and Outreach Source: CNN 8/16/2012
  • 20. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 21. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 22. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 23. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 24. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 25. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 26. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 27. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 28. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 29. Effective Education and Outreach Source: Jesse Poore - Felsburg Holt & Ullevig
  • 30. What Gets Measured Matters  Typical Permit Language:  “Within [insert deadline, e.g., within the permit term], the permittee must assess changes in public awareness and behavior resulting from the implementation of the program such as using a statistically valid survey and modify the education/outreach program accordingly.”
  • 31. What Gets Measured Matters  Permits and guidance typically promote measuring percent change in behavior. BUT  Does not correlate behavior to a loading rate  Rarely identifies a valid start point or a satisfactory end point  Ability to prioritize limited dollars is minimized
  • 32. What Gets Measured Matters  Permit Effectiveness ≠ Behavior Effectiveness  Measuring evaluations and surveys against learning objectives ≠ Behavior Effectiveness
  • 33. Evaluate Cost Effectiveness  Recent webcasts provide helpful techniques  WEF: Evaluating Stormwater Outreach  EPA: Conducting Effective Stormwater Outreach  Neither promoted direct measurements to supplement surveys and raised awareness  Neither discussed pollutant loading associated with behaviors
  • 34. Community Based Social Marketing  Approach by McKenzie-Mohr and Smith  Identify barriers and benefits  Define tools to change behavior  Pilot the approach  Evaluate results  Emphasize direct measurements  Supported by surveys and focus groups
  • 38. Important Question #1  “If you can’t measure embraced behavior, how are you going to measure percent behavior change?”  Hint: You can’t!
  • 39. Measuring Embraced Behavior  Identify behaviors for target pollutants  Define target behavior or indicator  Establish target for Embraced Behavior  Commit to observing behavior  Compare observed to recommended  Communicate effectiveness
  • 41. Important Question #2  “If you can’t establish a load associated with a behavior, how are you going to measure improvement in water quality?”  Hint: Urban runoff loading based on behavior is uncharted territory
  • 42. Behaviors Impact TMDLs  # of Restaurants Screened: 38  Number Grease Bins Open: 5 (13.2%)  Non-Conforming Bins: 1 (2.6%)  No Bin Seen: 4 (10.5%)  Surface Area Stain None/Covered: 20 (52.6%) Slight: 10 (26.3%) Significant: 4 (10.5%)
  • 43. Behaviors Impact TMDLs None/Covered = X Slight = X lbs./Yr. lbs./Yr . Significant = X Non-Conforming = X lbs./Yr. lbs./Yr.
  • 44. Embraced Behavior Proposal  Establish desired social, institutional behaviors  Centralize “Embraced Behavior” conversation  Discuss best methods for direct measurement  Research pollutant loading associated with undesired behaviors  International Stormwater Behavior Database
  • 45. Presenters KeepitCleanPartnership.org Janice Lopitz 303-441-1439 Felsburg Holt & Ullevig Jesse Poore - jesse.poore@fhueng.com @Jesse_W_Poore CP-Compliance Carrie Powers - cpowers@cp-compliance.com
  • 46. Outcome of Permit Language
  • 48. Outcome of Permit Language

Notas del editor

  1. Revisit IGA and bylaws to ensure reflection of current goals Revisit priority pollutant focus and associated behaviors to determine strategies - methodically Employ tools to measure embraced behavior - tracking and evaluate resource expenditures ?
  2. When I think about education and outreach, I consider it across all Minimum Control Measures. IDDE, CSW, PCSW, GHPP
  3. I also keep in my mind the 1999 text in the Federal Register regarding Maximum Extent Practicable. EPA envisioned an iterative process that would take 10-15 years to eliminate water quality impairments caused by MS4s In this frame of mind, education and outreach was never expected to be a permanent effort if MCMs were effective Illicit discharges were reported when found and citizens disposed of all waste properly Construction sites were all maintained properly and inspections were self-conducted to identify corrective actions Maintenance facilities were managed in pristine condition with pollution prevention integrated into all operations Watersheds were planned wisely, all developments included water quality treatment and permanent maintenance
  4. The many unique differences between one program and another were fascinating, but frustrating at the same time Each partnership had its own approach and localized resources to approach education and outreach with One unifying theme – credit Mindy Fohn for illustrating this point for me – effectiveness is best defined by direct observations
  5. If we aren’t clear where the end point might be, perhaps we could trend against categorical expenses incurred by other MS4s Trouble comparing due to expense categories and not tracking all things. Did start to get me thinking about budget amounts and the “knee of the curve” argument for CSO LTCPs exponential output of cost to generate an incremental benefit result put that idea into the parking lot for later use
  6. One of these things is not like the other…stormwater is the only mandated effort required to change social and institutional behaviors.
  7. Man’s impact upon the environment is nothing new. We have always been slow to acknowledge our need to embrace best practices in order to get the most out of the land. Hugh Bennett was the first to definitively communicate to the nation that the Dust Bowl was the result of man’s behavior The impact was wide, personal, and deadly – a consequence we may be to far from to appreciate the correlation to stormwater runoff
  8. source: CNN 8/16/2012 http://www.cnn.com/2012/08/16/business/tobacco-health-warnings/index.html?hpt=hp_c2 Even measuring raised awareness isn’t a very good indicator of good behavior. America has one of the highest levels of national awareness about the health effects of smoking. And yet it continues to be the habit of choice for so many. Awareness does not equal embraced behavior.
  9. Guns kill people and people create pollution The Causes of Pollution Conceptually, pollution is NOT the affect of pollutants. Pollution is the result of the ACT of introducing a pollutant into the environment. Therefore, MS4s should focus education on the pollutant generator. With proper behavior, pollution doesn’t occur and permit is not necessary.
  10. Changing behavior is the underlying principle of any public education and outreach effort. Use a variety of assessment methods to evaluate the effectiveness. Are public education and outreach programs changing public awareness and behaviors?
  11. US Stormwater education focuses on surveys and evaluations, but leave direct measurments off hoping that raised awareness will equate changed behaviors McKenzie-Mhor understands that sustainable behaviors are the result of a targeted outreach effort the starts with direct measurements and is supported by indirect assessments.