Recomendados
Recomendados
Más contenido relacionado
Similar a Using Small Area Estimates for ACA Outreach
Similar a Using Small Area Estimates for ACA Outreach (20)
Más de soder145
Más de soder145 (20)
Último
Último (20)
Using Small Area Estimates for ACA Outreach
- 1. USING SMALL AREA ESTIMATES FOR ACA OUTREACH 2014 AcademyHealth San Diego, California June 10
- 2. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Acknowledgements 2 • Supported by a grant from the Robert Wood Johnson Foundation to the State Health Access Data Assistance Center (SHADAC) at the University of Minnesota • Co-Authors • Michel Boudreaux, Lead Author, SHADAC • Peter Graven, Oregon Health & Science University • Special Thanks to: • Elizabeth Lukanen – Deputy Director, SHADAC • Karen Turner—Senior Programmer Analyst, SHADAC • Joanna Turner – Senior Research Fellow, SHADAC • Lynn Blewett – SHADAC Director
- 3. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Outline 3 • Research Objective • Background • ACS • ZCTAs • Reliability • Methods • CAR model • Composite model • Results • Interactive Maps • Findings
- 4. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Motivation • The public’s knowledge of the ACA is poor • As of January 2014, 46% of the uninsured did not know about the availability of financial help for coverage • Overall, the first open enrollment season was successful • But lots of variation across the states and a long way to go • Success during the 2nd season will depend on outreach 4
- 5. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Outreach • Blanket media campaigns might not be enough • Need to target the uninsured • To do that efficiently • Need to know where the uninsured are • What kind of communities they live in • What institutions are present in the local community that can serve as access points 5
- 6. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Research Objective 6 • PROBLEMS: • Small Area Health Insurance Estimates (SAHIE) are not granular enough • Direct zip code level estimates (ACS) can be unreliable • Accessing the data can be difficult • GOALS: • Improve access to ZIP Code level estimates • Improve reliability of ZIP Code level estimates
- 7. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level BACKGROUND 7
- 8. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level American Community Survey (ACS) • General household survey conducted by the U.S. Census Bureau • Mandatory survey in 4 modes (mail, internet, phone, in person) • Collects sample in all counties or county equivalents in the U.S. every year • Replacement for the “long form” of the decennial census • Collects detailed economic, social, demographic, and housing information annually instead of once every ten years • Collects information on health insurance status information at time of survey (produces point in time insurance estimate) 8
- 9. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Identify Location of Potentially Eligible 9 Nation, States, & DC Congressional Districts Counties School Districts Public Use Microdata Area (PUMA) Metro & Micro Statistical Areas ZIP-Code Tabulation Areas Census Tracts
- 10. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Counties - Reliability 10 Minnesota Percent Uninsured Estimates by County • ACS 2008-2012 • Highest RSE is 18.4% • Average RSE is 8.0% • Average RSE top ten (sample size) counties 3.7% • Average RSE bottom ten (sample size) counties 12.2% • SAHIE 2011 • Highest RSE is 7.9% • Average RSE is 6.0% • Average RSE top ten (sample size) counties 5.1% • Average RSE bottom ten (sample size) counties 6.3% Note: RSE is relative standard error (standard error/estimate)
- 11. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level ZCTAs – Reliability 11 Non-Zero Percent Uninsured Estimates in Minnesota • Highest RSE is 174% • Average RSE is 27.6% • Thirty percent of RSEs>28% • Seven percent of RSEs>50% • N ≈ 890 Non-Zero Percent Uninsured Estimates in U.S. • Highest RSE is 509% • Average RSE is 27.6% • Thirty percent of RSEs>31% • Eleven percent of RSEs>50% • N ≈ 33,000
- 12. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level 95% confidence intervals for ZCTA estimates with RSEs >50%: MN 12 0 .2.4.6.8 1
- 13. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level METHODS 13
- 14. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Two Methods for improving precision 14 • Conditional Auto-Regressive Model (CAR) • Advantages: Established method in the statistics literature • Disadvantages: High level of complexity, difficult to scale and apply to other types of estimates • Modified Composite Method • Advantages: Easy to scale and apply to different types of estimates • Disadvantages: New approach so not peer reviewed
- 15. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level CAR Model • Auxiliary data (covariates) improves prediction • 𝑟𝑧 = 𝛼 + 𝛽𝑋𝑧 + 𝑣𝑧 • Borrows strength from neighbors • Creates term for average value of adjacent neighbors • 𝑣𝑧|𝑣−𝑧, 𝜎𝑣 2~𝑁 𝑗∈𝛿 𝑧 𝑣 𝑗 𝛿 𝑧 , 𝜎 𝑣 2 𝛿 𝑧 15
- 16. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Composite Model • Rough approximation of a composite estimator 𝐶𝑜𝑚𝑝 𝑧𝑐 = 𝑤𝑡 𝑧𝑐 ∗ 𝑍𝐼𝑃𝑅𝐴𝑇𝐸𝑧𝑐 + 1 − 𝑤𝑡 𝑧𝑐 ∗ 𝐶𝑂𝑈𝑁𝑇𝑌𝑐 where wt=weight 𝑤𝑡 𝑧𝑐 = (𝐶𝑜𝑢𝑛𝑡𝑦𝑐 − 𝑍𝐼𝑃𝑅𝐴𝑇𝐸𝑧𝑐)2 /𝑇𝑜𝑡𝑎𝑙 𝐸𝑟𝑟𝑜𝑟𝑧𝑐 𝑇𝑜𝑡𝑎𝑙 𝐸𝑟𝑟𝑜𝑟𝑧𝑐 = (𝐶𝑜𝑢𝑛𝑡𝑦𝑐 − 𝑍𝐼𝑃𝑅𝐴𝑇𝐸𝑧𝑐)2 + 𝑆𝐸2 𝑧𝑐 See Rao (2003) 16
- 17. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Composite Model Intuition 17 Leans toward ZCTA Leans toward county Difference County and ZCTA SEofZCTA Low High HighLow
- 18. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Model Results CAR Model Results Coef. 95% Credible Interval % White -0.02 (-0.02,-0.017) % Living w/Kids -0.01 (-0.016,-0.009) SD of Spatial Effects 0.32 Composite Model Mean SD Weight .53 .32 18
- 19. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Comparison of Methods Direct CAR Composite Rate, % 9.3 9.5 9.1 SE 2.6 1.1 1.6 RSE, % 27.6 11.1 20.9 RSE>30, % 28.7 0.1 11.2 RSE>50, % 8.5 0.1 3.4 19 Average across estimates: Minnesota
- 20. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Distributions by Sample Size 20 0 1020304050 Rate Direct CAR 0 1020304050 Rate 0 1000 2000 3000 4000 5000 Sample Size Direct Composite
- 21. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Which method is better? Direct CAR Composite Complexity Low High Low Scalability Easy Hard Easy Reliability Not very reliable Very reliable More reliable but still not great Bias ? ? ? 21
- 22. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Maps! 22
- 23. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Minnesota: Comparing Estimates 23
- 24. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Start with the County 24
- 25. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Then look at ZCTA estimates 25
- 26. Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level Findings • Providing uninsurance estimates at the ZCTA level is problematic both from the standpoint of reliability and accessibility • Possible solutions to the problem of reliability is to use small area methods such as CAR or a moderated composite estimator • CAR is the more established method and provides more reliable estimates but is complex and difficult to scale • A potential compromise is to use the modified composite estimator but more testing is needed • Interactive mapping can make these estimates available at the ZCTA level 26
Notas del editor
- In this presentation I am going to discuss research where our goal is to improve the access to and reliability of uninsurance estimates at the ZIP Code level.
- The lead author of this work is my colleague Michel Boudreaux. Peter Graven from Oregon Health Science University is also a contributing author. However, we have also received help and support from the following other individuals:
- I will start out with some background information on the American Community Survey Then discuss the reliability of estimates at the ZIP Code level and look at ZIP code level percent uninsured estimates in three ways: Direct estimates Estimates from an area level conditional autoregressive model Estimates from a modified composite model. I will also show some of these estimates along with other demographics on an interactive map Discuss our findings
- As Kaiser has shown in their tracking polls the publics knowledge of the provisions of the Affordable Care Act is not great. However, 8 million are enrolled in the ACA health insurance exchanges. So do we still need outreach. Yes we think outreach is particularly important now because a lot of the individuals who were relatively easy to enroll have already done so. Remember that the CBO estimates that by 2016 enrollment in the exchanges will be 24 million
- Particularly now after the first open enrollment period, we think that a more targeted strategy of outreach makes sense. Purpose is to find out Where the uninsured are What kind of communities they live in What institutions are close by that can serve as access points
- The census bureau does have a small area health insurance estimates program that produces estimates at the county level. However, states have been telling us they are also interested in these estimates at the ZIP Code level. Currently there are estimates available at the zip code level but it is difficult to access them and many of them are unreliable.
- Now we will talk a little bit about the 5-year pooled ACS data that we will be using and the reliability of the zip code level uninsurance estimates .
- The only publicly available source for uninsurance estimates at the ZIP Code level is the five year pooled data from the American Community Survey that is available through the Census Bureau’s American Factfinder. Although 5-year ACS pooled data has been released by the Census since 2010, it was not until December 2013 when the 2008-2012 file became available that health insurance estimates became available
- This figure is not to scale but does give an idea of the geographic hierarchy in terms of size of the availability of health insurance data from the Census Bureau. Our focus for this presentation is the next to lowest geography shown in this figure: ZIP Code Tabulation Areas or ZCTAs. ZIP Code Tabulation Areas are like ZIP codes except they conform to census geography.
- In the rest of this presentation we will focus on ZIP Code Tabulation Areas but in this slide we look at the reliability of uninsurance estimates at the county level from two sources to give some perspective on reliability when we then move to ZCTAs. This data is for Minnesota. In both cases (the SAHIE and the ACS) the average relative standard error is less than 10% and the highest relative standard error is 18 percent. However, states also want more granular estimates because the percent uninsured can vary quite a bit for ZCTAs within the same county. For example, in Ramsey County, Minnesota where I live the percent uninsured by ZCTAs varies from 4.3% to 17.6% uninsured.
- In contrast to data for counties in Minnesota where the highest RSE for percent uninsured estimates is 18%, for ZCTAs it is 174%. The average is also much higher at 27.6% rather than 6 to 8 %. It is not that the data for MN is particularly unreliable since the average RSE in the U.S. is the same and in both cases about 30% of the estimates have RSEs greater than 30%. For some perspective on what this means, the National Center for Health Statistics suppresses estimates if the RSE is greater than 30%. If we used this rule for Minnesota or used it for the U.S. this would result in the suppression of about 30% of the percent of uninsured estimates by ZCTA.
- This is just another perspective on reliability using the lower and upper bounds of the confidence intervals for the percent uninsured estimates with RSEs greater than 50% for Minnesota.
- Now we will look at the two methods we used to try to improve the reliability of the estimates: the area level conditional autoregressive model and the modified composite model.
- The advantages of the CAR model is that it is more vetted approach, but resource intensive and not as intuitive. The other is more intuitive and easy to apply, but is new and has not been peer reviewed.
- The area-level conditional autoregressive model fits the survey’s direct estimate of the uninsurance rate with covariates with larger sample sizes to provide a more reliable prediction of the uninsurance in each ZCTA. The spatial correlation allows for borrowing of strength by using the average uninsurance rate of adjacent ZCTAs, for each ZCTA. The performance of area-level models has been tested in several simulations (Gomez-Rubio et al. 2008) and provides distinct improvements over using direct estimates. Covariates included in the small area model came from the 2010 Census Gomez-Rubio, V., N. Best, S. Richardson, and G. Li. 2008. Bayesian statistics for small area estimation. United Kingdom: Office for National Statistics.
- Intuition: A weighted average of the direct ZCTA rate and the direct county rate. We make a compromise between the uncertainty of the direct ZCTA estimate and the uncertainty that the direct county rate is a good proxy for the ZCTA estimate. The standard error estimate of the composite estimator uses the same weighted average approach as used to produce the composite estimates,
- For example, if the direct ZCTA estimate had a low standard error and the difference between the county estimate and the ZCTA estimate was high then we would lean towards the ZCTA estimate..
- We were limited in the choice of covariates by what was available in the 2010 census and ended up choosing two covariates: percent white and percent of households that included an own child under 19. These are preliminary and we are still looking at other covariates but as you will see in the next slide these covariates combined with the spatial component do result in a model that has the best results in terms of reliability. The weight of the composite model was close to 50% which means we were borrowing just a little more from the direct ZCTA estimates than from counties.
- CAR: 60% reduction in SE; Virtually no ZCTA is unreliable (strange result for one ZCTA); Composite: 40% reduction in SE; 20% Reduction in RSE; 60% reduction in the number of unusable ZIPS. Clearly in terms of improving reliability the CAR method is best.
- The first figure shows the modeled CAR estimates with green circles and the direct estimates as blue circles. At smaller sample sizes the distribution of CAR estimates is much tighter than those of direct estimates and as the sample size increases the variation between the CAR and direct estimates becomes smaller. This is also true of the composite estimates but to a much less degree.
- Our hope at the beginning of this process was that the composite estimator would look more like the CAR estimates than the direct estimates. However, these ended up looking more like the direct estimates than the composite estimates. We also discovered that implementing the CAR model was difficult because we couldn’t run large amounts of data using the baysian analysis software called open bugs. We are looking into seeing if there is a better way to do this as well as refine our composite model. However, when we provide this data to states we will choose only one set of estimates of the percent uninsured.
- While I have spent most of my time talking about how we have tried to address the reliability issue, probably the most important development in terms of disseminating these estimates to the people who need them is our work on developing interactive maps.
- Here we have three maps of Minnesota ZCTAs. The map on the left is of direct estimates, the one in the middle is of composite estimates and the one on the right is for the CAR estimates. The little black triangles indicate if the RSE of the percent uninsured estimate >30% and it just shows in picture form that the composite estimates fall in the middle in terms of reliability.
- All of the interactive maps start out with the counties and clicking on a county gives information on the percent uninsured, number of uninsured and population of the county.
- Then when you zoom in and click on a ZCTA. the box that comes up includes, percent unininsured, whether the ZCTA estimates are reliable, the RSE, the standard error, the ZCTA population, poverty level categories, race and ethnicity and language. This is true for all three layers: direct estimates, composite estimates and CAR estimates. We also currently include overlays of hospitals and churches and plan to include other overlays such as schools and federally qualified health centers.