![Overview ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
Viewers also liked
Viewers also liked (8)
Similar to Evaluating Social Marketing Campaigns
Similar to Evaluating Social Marketing Campaigns (20)
Recently uploaded
Recently uploaded (20)
Evaluating Social Marketing Campaigns
- 1. Evaluating Social Marketing Campaigns Jorge A. Montoya, Ph.D. Los Angeles County Department of Public Health
- 15. Evaluation of Check Yourself: A Social Marketing Campaign for MSM
- 17. Appeal, Motivation, and Relevance of Campaign Ads
- 18. Average Number of Syphilis Tests Performed Per Month Among MSM at LAGLC Clinics Jan 06 Jul 06 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Check Yourself Campaign
- 19. Evaluation of I Know: A Social Marketing Campaign for Young Women of Color
- 20. Campaign Awareness (n=599) General HIV/AIDS Reference BET – Rap It Up HPV/Cervical Cancer/Gardasil Ref. Safer Sex/Protection/Use Condom General Advertising Reference Blair Underwood – Man Up I Know/Don’t Think Know Get Tested/Get Checked Other (Less than 2% mention) General STD Reference 63
- 21. Mode of Exposure to Campaign Among Those Aware (n=599) Billboards Television Posters Bus Signage Radio Other (Less than 2% mentions) Magazine *Multiple mentions allowed – May not add to 100%.
- 23. Knowledge – Heard of Chlamydia? By Baseline & Endline (Unaware vs. Aware of Campaign) Baseline (n=301) Endline Aware (n=375) Endline Unaware (n=224) Endline Total (n=599) Desired Answer = Yes p<.001 p<.001
- 24. CT & GC Testing In Last 6 Months By Campaign Awareness (Endline = 599) Aware (n=375) Not Aware (n=224) * Chi-Square test significant at p=0.011
- 25. Predictors of CT & GC Testing In Last 6 Months (n=591) 95% CI Variable Sig. OR Lower Upper Campaign Awareness (Not aware vs. Aware) .041 1.56 1.02 2.38 Language (Spanish vs. English) .001 3.00 1.56 5.75 Age ns 1.05 0.96 1.16 Relationship (No vs. Yes) .015 1.65 1.10 2.48 Children (No vs. Yes) .010 2.04 1.18 3.51 Ethnicity (Black vs. Latino) ns 0.90 0.77 1.05 Medical Insurance (No vs. Yes) ns 0.89 0.57 1.38 Previous STD Diagnosis (No vs. Yes) .000 3.83 1.85 7.94 None REF ---- ---- ---- One .000 12.87 4.45 37.18 Multiple .000 21.54 7.47 62.09 Some High School or Less REF ---- ---- ---- Completed High School ns 1.33 0.70 2.52 Some College/Trade .042 2.00 1.02 3.91 College Graduate + ns 1.95 0.7 4.95 Number of Partners Level of Education
- 26. Home Test Kit Orders by Week from June 8, 2009 through February 9, 2010 Week Number of Orders Ad Campaign 71% of total orders Average Weekly = 153 Average Weekly = 30 PR
- 27. donthinkknow.org Visits by Week from June 8, 2009 through February 9, 2010 Week Number of Orders Ad Campaign 58% of total visits Average Weekly = 1,126 Average Weekly = 398 PR awareness measure
- 28. *How did you hear about DTK? (n=576) Family or friends Online Ad TV Commercial Poster Magazine Article/ Marie Claire No Response Bus Ad – Inside Bus Browsing online/ Search/ Google Other (Less than 2% mentions) Info. Card Bus Ad – Outside Bus *Multiple mentions allowed – May not add to 100%, from dontthinkknow.org web site users. School/ Teacher/ Class Presentation
- 29. Contact Info: Jorge A. Montoya, Ph.D. (p)213.864.8959 (f) 866.614.6075 [email_address] Sponsor: www.sentientresearch.net
- 33. Properties of Measurement Levels: Permissible Statistics Increase With Each Level A is equal to (not equal to) B = (≠) √ √ √ √ A is greater than (less than) B > (<) √ √ √ A is three more than (less than) B + (-) √ √ A is twice (half) as large as B x ( ÷) √ Appropriate math operations Examples of comparison statements Relevant Level of Measurement Nominal Ordinal Interval Ratio
- 34. Evaluation of Stop the Sores : A Social Marketing Campaign for MSM
- 35. Results – Campaign Awareness (1) Safer Sex Reference * Not equal to 100%, multiple mentions allowed. Syphilis Reference Get Tested HIV Stops with Me HIV/AIDS Reference Know Your Partner Healthy Penis 71% Stop the Sore/Phil the Sore AIDS Walk Smoking Crystal Meth Other STD/General reference HIV Testing 3
Editor's Notes
- Formative research typically involves Qualitative methods : focus groups, one-on-one interviews, participant observation, etc. Qualitative research in terms of focus groups will be covered in more detail by VIVIAN MARLES. Formative research can also include quantitative methods `Market Segmentation (i.e., Hierarchical Cluster Analysis) `Trade-off Analysis (i.e., Choice-Based Conjoint Analysis) I will focus on Quantitative Evaluation Research .
- Cross-Sectional One Shot Case Study Longitudinal Designs… Repeated Cross-Sectional – Usually manifested as trend studies (e.g., global changes in public attitudes such as racial tolerance, abortion, gun control; political polling such as favorability ratings of the president). Fixed-Sample Panel – This design follows the same individuals over time so that it allows us to observe changes in the DV in each individual who acts is his/her own control. Considered better than Repeated Cross-Sectional for testing causal hypotheses. Like Before-and-After or One-Group Pretest Posttest Design. Cohort study : Where you select those with exposure and those without exposure and follow forward (or backwards in time) for the development/occurrence of disease/outcome. A cohort is a group of people who share a common characteristic or experience within a defined period (e.g., are born, leave school, lose their job, are exposed to a drug or a vaccine, etc.). Thus a group of people who were born on a day or in a particular period, say 1948, form a birth cohort. The comparison group may be the general population from which the cohort is drawn, or it may be another cohort of persons thought to have had little or no exposure to the substance under investigation, but otherwise similar. Alternatively, subgroups within the cohort may be compared with each other. A research study that compares a particular outcome (such as lung cancer) in groups of individuals who are alike in many ways but differ by a certain characteristic (for example, female nurses who smoke compared with those who do not smoke). The other problem with cohort studies is that they can end up taking a very long time, since the researchers have to wait for the conditions of interest to develop. Researchers are, of course, anxious to have meaningful results as soon as possible, but another disadvantage with long studies is that things tend to change over the course of the study. So research turn to the case-control study. Case-Control Study: case-control, you identify your cases (have outcome) and controls (don't have outcome) and then you can look look back in time or at current exposure, so can have longitudinal component that is retrospective. could also have prospective, eg., measures of biomarkers forward in time. A study that compares two groups of people: those with the disease or condition under study (cases) and a very similar group of people who do not have the disease or condition (controls). Researchers study the medical and lifestyle histories of the people in each group to learn what factors may be associated with the disease or condition. For example, one group may have been exposed to a particular substance that the other was not. Also called retrospective study
- -Usually the best alternative to an experimental design in maximizing internal validity is quasi-experimental design. - No consistent definition of quasi-experimental design in the literature, but generally it is one that is like an experimental design with the exception that groups are not randomly created. Statistical Control: Conduct multivariate statistics and control for -Demographics -Other relevant variables to the DV Use of pre-test and post-test so that subjects serve as their own control.
- Refusal rates Screen outs Funnelled…Broad general questions in the beginning, warm-ups…more specific questions later..sensitive questions much later too. Example with does syphilis have any effect on HIV? What kind of effect?
- Examples of bad questions: Double barreled: How relevant and motivational do you find the advertisements?
- WOC Unaided: There are many announcements and ads to keep people informed about issues. Can you recall any ads or public events during the last year that provided information about sexually transmitted diseases? Unaided awareness question from STS: Can you recall any campaigns or advertisements in Los Angeles in the past year that were about sexual health in your community? Aided: Do you recall hearing or seeing one of these ads (SHOW ADS)? Modes of Exposure: Where do you recall seeing or hearing this advertising? Exposure frequency: About how many times did you see/hears this/these ads? _____ # (best guess) Key Message(s): What is/are the main message(s) in this advertising? (DO NOT READ OPTIONS)
- A Co-Factor (or a potential confounder) is anything that can have an influence in your outcome besides you campaign. Now this can be a lot of things, so we have to be practical and use a lot of common sense to keep the list of questions at a reasonable length. FOR example, for WOC potential co-factors was covered with about 15 questions.
- Process measures that measure implementation vs. process measures that help assess outcome
- Aware=269, 30=Unaware There are many announcements and publicity in the media that are directed at the public to keep them informed about issues in their community. Can you recall any ads or public events that provided information about sexual health during the last year here in Los Angeles? If yes, please describe the campaign or key messages as best as you can. (WRITE IN NAME OR DESCRIPTION VERBATIM GIVEN BY RESPONDENT IN ORDER ANSWERED. DO NOT PROMPT OR PROBE
- 10% is a watermark – there were no billboards in bus, so this is a false listing. We estimate that anything above 10% should be a valid percentage. Multiple answers were allowed for last question. Magazines was most commonly given answer. (people listed frontiers, adelante, and odyssey.) Television + TV commercial were combined for this graph to equal 12 %. Only 7% mentioned the STS commercial unaided. Internet + website were also combined to equal 7%. Only 4% mentioned the STS website unaided. Other: consisted of mostly places, where people would have seen various outreach materials. These places could have included places where the mobile testing van was located. In a few cases, they actually mentioned outreach materials (posters, stress grip, palm cards, comic book.) We can’t say specifically what these outreach materials are, however. Other – unique mentions of less than 3%. Median is only 2 because there were a lot of 0’s for seeing campaign in last 3 months.
- Baseline vs. Endline Total significant at p=.005 Baseline vs. Endline Aware significant at p<.001 Baseline vs. Endline Unaware NS Endline Unaware vs. Endline Aware significant at <.001
- 80% drop in average weekly orders after the campaign ended. 71% of orders occurred during the campaign or 1,885/2,572
- 65% drop in average weekly visits after marketing campaign Visits during campaign = 13,508 / After = 9,939
- This is how you avoid GIGO (Garbage in, Garbage out). This is also where sampling, instrument development, and design come into play. Measurement Validity is the first concern in establishing the validity of research results. Usually remedied by using proven scales (i.e., self-efficacy, etc.) Quality of instrument design will determine measurement validity. Causal Validity considerably much more important in evaluative research. Remedied with a sound research design and clearly defined dependent and independent variables. Generalizability Sampling strategies can remedy generalizability. Two aspects of Generalizability: - Sample Generalizability : Exists when a conclusion based on a sample , or subset, of a larger population holds to for that population - Cross-population generalizability : Exists when findings about one group or population or setting hold true for other groups or populations or settings (Also called External Validity ) External validity is absolutely meaningless if you do not have internal validity.
- Additional manifest variables with subsequent questions may include Sexual Addiction, Self-Esteem, Self-Efficacy with regards to Safer Sex Practices, etc. Self-Efficacy: Person’s belief about his/her ability or capacity to succeed in a particular situation (accomplish a task or to deal with the challenges of life). Most often you can use available data, theories or scales that have already defined constructs and have a specific set of questions associated with them, or scales. The use of scales in research will be covered in greater detail by LAUREN FRANK How are manifest variables identified?
- You can elaborate on how there is a loss of information as you move from ratio to nominal, so whenever possible you try to collect ratio info, which you can recategorize to become nominal, ordinal, etc. My favorite example of this is when someone gave me age, which was collected in categories, and asked me to compute the mean age!! perfect example of loss of info.
- Respondents were asked if they could recall any ads or public events that provided information about sexual health issues since June of last year in Los Angeles. All of these responses are unaided awareness, except that aided questions were asked regarding STS and Healthy Penis. Top bars represent mentions of specific health campaigns. Healthy Penis awareness also went up from the 2002-2003 evaluation from 5% awareness to 21%. 27% unaided awareness is larger even than general HIV/AIDS reference.
- Here you can see that syphilis testing was significantly associated with campaign awareness. We are comparing those who were exposed to the campaign versus those who weren’t. You can see that >>>>>>>> And this difference is stat. sig.
- Here you can see that campaign awareness is the only factor that is significantly related to syphilis testing, with an odds ratio of 1.83. We did a multivariate analysis. This allows us to control for other variables which might be assoc. with syphilis testing behavior. After controlling for XXXX variables, we found that those exposed to our campaign were nearly twice as likely to be tested for syphilis. Campaign awareness is the only sig. variable.