Health Services Research © Health Research and Educational Trust DOI: 10.1111/1475-6773.12125 EDITORIAL
Editorial
Evaluating the Massachusetts Health Care Reform There are a number of reasons the Affordable Care Act (ACA) is unprecedented in its scope and reach. The law aims to introduce the largest expansion in insurance coverage in the nation’s history, to redistribute billions of dollars to make health insurance more affordable, to overhaul the regulation of insurance markets, and to introduce system reforms that transform the delivery of health care in the United States. Yet another unprecedented feature of the ACA is that it is perhaps the only massive expansion of our nation’s safety net which was directly piloted before passage. The state of Massachusetts passed a health care reform in 2006 which was the explicit model for the coverage and insurance market reform portions of the ACA. By the time that the ACA was passed in March 2010, there were nearly 4 years of experience with a similar law that could be studied to predict the ACA’s impacts. The Massachusetts law is not identical to the ACA for a number of reasons. First and most important, it applies only in Massachusetts, which is different from the nation as a whole in a variety of ways. Second, the subsidies provided to many low-income families in Massachusetts were significantly more generous than those offered under the ACA. Third, employers play a much larger role in financing the ACA than they did with the Massachusetts law. However, in the main, the basic structure of the ACA was pioneered in the Bay State. As a result, researchers have flocked to Massachusetts to try to gather a picture of what life might look like under the ACA. Such assessments have been carried out on a wide variety of measures with differing degrees of rigor. In this commentary I would like to touch on the broader methodological issues involved in evaluating the Massachusetts “experiment” before 1819
1820
HSR: Health Services Research 48:6, Part I (December 2013)
returning to the interpretation of the article by Bond and White in this issue of Health Services Research. A number of different methodological approaches have been applied to considering the evidence from Massachusetts. Least convincing are studies that simply compare outcomes in Massachusetts after reform to outcomes elsewhere in the nation. This naïve approach completely ignores the multitude of differences between Massachusetts and other states, such as higher rates of insurance coverage, a larger medical sector, and a higher income level. These differences dwarf any effects of health care reform when comparing Massachusetts to other states, so that this comparison cannot be credibly used to infer the impacts of reform. This approach has been used frequently in the press but fortunately not in academic studies of note. Somewhat more convincing are studies that examine how outcomes changed in Massachusetts from before to after reform (e.g., pre 2006 vs. post 2006). This approach controls for steady state differences in the state such as its generally higher rate of insurance coverage. However, it has the weakness that many other things changed around 2007—in particular, the Great Recession. So to simply compare outcomes before and after that time would miss trends that are completely independent of Massachusetts health reform.1 More convincing still are studies that compare the change in outcomes in Massachusetts to changes in other states over the same period. Such a “difference-in-difference” (DD) approach allows researchers to consider how reform impacted Massachusetts relative to other states which were subject to similar time series shocks in outcomes. That is, this approach controls for both long-standing differences across states and time series trends in outcomes. A wide variety of studies have used this approach. Some examples include Kolstad and Kowalski’s (2012) findings that reform led to reduced rates of preventable hospitalization; Courtemanche and Zapata’s (2012) findings that reform led to improved health outcomes along a number of dimensions; Miller’s (2012, in press) findings that reform led to more preventive care use and better reported health outcomes for children, as well as to an 8% overall decrease in emergency room visits; and Graves and Gruber’s (2011) finding that reform led to enormous declines in nongroup insurance premiums but no meaningful change in employer insurance premiums. However, such an approach is not perfect: it requires the key assumption that nothing else changed differentially in Massachusetts relative to other states other than health care reform. For example, it is possible that the Great Recession hit Massachusetts in a different way than other states, perhaps
Evaluating the Massachusetts Health Care Reform
1821
because of a different industrial composition across states. This could bias the estimates from a DD strategy. There are several approaches to dealing with this limitation, none of them perfect. One approach is to try to find states that are most comparable to Massachusetts in terms of the likely impacts of other time series factors, for example, regional neighbors or states with a similar industrial composition. A number of the studies cited above use this approach. Another is to find “within-state controls,” groups that are more or less impacted by the health care reform. For example, health care reform had much smaller impacts in areas of the state where private insurance rates were already very high, with larger effects on areas with more uninsurance. This brings us to the article by Bond and White. The question at hand is one of “spillover” effects of the Massachusetts reform: does covering more nonelderly residents with health insurance “crowd out” use of primary care by the elderly by overburdening primary care providers? This question is part of a larger concern expressed by critics of the ACA that expanding coverage to our nation’s uninsured residents will reduce access to care for those who already have insurance. The authors address this question through the difference-in-difference strategy discussed above, augmented with both of the approaches to dealing with the limitations of this strategy. The authors conclude that Massachusetts reform led to a significant decline in use of primary care by the elderly. I don’t agree that this conclusion is warranted by the results they present. Their basic difference-in-difference analysis does not support this conclusion, even when the analysis is restricted to geographically similar states. It is only when the authors dive in to consider heterogeneity by areas of the state that they find evidence of a reduction in primary care use by Medicare enrollees, and even in this context the evidence is somewhat confusing. The key facts are presented in Figure 1 (Graves and Gruber 2012), which shows the pattern of Medicare spending on physician and clinical services by state over time, for Massachusetts, the nation as a whole, and a set of Northeastern comparison states. The figure clearly shows that all three of these groups were on a comparable trend until 2005, suggesting that preexisting conditions were comparable. Starting in 2006, spending rises in Massachusetts relative to both the comparison states and the United States as a whole. By 2009, Massachusetts spending was slightly below the comparison states and equal to the nation as a whole. Certainly, this figure does not paint a picture of access problems for Massachusetts seniors.
1822
HSR: Health Services Research 48:6, Part I (December 2013)
Table 1 (Graves and Gruber 2012) further enriches the analysis by considering only primary care visits. This seems to strengthen the conclusion that health care reform increased utilization by seniors. Regardless of the weighting scheme used, primary care visits under Medicare rose in Massachusetts and fell in the set of comparison states. The authors’ conclusion regarding reduced utilization is based on Table 2, which takes the additional step of dividing the state’s zip codes by tercile of the uninsurance rates before reform; the natural presumption is that those zip codes with the highest rate of uninsurance saw the greatest rise in coverage. They find that in fact Medicare primary care visits fell by almost 7% in the set of zip codes with the highest rate of uninsurance before reform, and an equal or larger effect on those zip codes with the lowest rates of uninsurance (which are captured by the Massachusetts dummy variable). The authors focus on the former finding to conclude that reform led to reduced use of primary care by seniors. However, that conclusion is simply too strong. Indeed, the fact that Medicare primary care visits rise so significantly in the zips with the lowest rates of uninsurance is just as striking as the fact they fall in the zips with the highest rate of uninsurance. At best, I would characterize the overall results as mixed. The overall difference-in-difference comparison clearly shows an increase in use of physicians by Medicare patients. The zip-code level analysis just shows enormous heterogeneity in how this played out across Massachusetts zip codes. To conclude any more from the analysis would be premature. A more conservative reading of the same evidence would be that there is suggestive evidence that use rose among seniors in Massachusetts, but that evidence is not confirmed by the within-state analysis, leaving us uncertain as to the bottom-line result. This article does raise the interesting issue of how to trade off increasing access for the uninsured against potentially reduced access for the (elderly and nonelderly) insured. Existing evidence from health economics suggests that even if such a trade-off were to exist, making that trade would likely improve overall population health. For example, a large literature shows sizeable impacts of expanding insurance coverage on physical health (e.g., Currie and Gruber 1996a,b) and on mental health (Baicker et al. 2013). At the same time, there is little evidence that raising patient costsharing (e.g., Newhouse 1993) or limiting provider access through limited networks (e.g., Glied 2002) has any meaningful impact on health. Moving from no or very limited primary care to regularly available primary care (such as would happen for the uninsured) almost certainly improves health
Evaluating the Massachusetts Health Care Reform
1823
more than moving from more easily to somewhat less easily accessed primary care. Ideally, however, society need not face such a trade-off. The medical system could readily expand its capability to deliver the necessary primary care by lowering restrictions on who can deliver such care (e.g., practice standards for nurse practitioners) or expanding medical training (one feature of the ACA). The era of studying health insurance expansion is far from over. The ACA will have widely varying impacts across states, particularly with respect to the states’ decisions to expand Medicaid. The type of difference-in-difference methodology described above can be used to further examine the impacts of insurance expansion on a variety of outcomes, including access of the insured to primary care. Hopefully future studies will help clarify whether there is really a trade-off between covering the uninsured and access for the insured. Jonathan Gruber
NOTE 1. For example, a recent study by Himmelstein, Thorne, and Woolhandler (2011) concluded that Massachusetts health care reform had not provided financial protection because the number of medical bankruptcies rose in that state. However, this comparison ignores the fact that the recession likely caused a large rise in medical bankruptcies nationwide at the same time.
REFERENCES Baicker, K., S. L. Taubman, H. L. Allen, M. Bernstein, J. H. Gruber, J. P. Newhouse, E. C. Schneider, B. J. Wright, A. M. Zaslavsky, and A. N. Finkelstein, for the Oregon Health Study Group. 2013. “The Oregon Experiment --- Effect of Medicaid on Clinical Outcomes.” New England Journal of Medicine 368: 1713–22. Courtemanche, C., and D. Zapata. 2012. Does Universal Coverage Improve Health? The Massachusetts Experience. NBER Working Paper #17893, March 2012. Cambridge, MA: National Bureau of Economic Research. Currie, J., and J. Gruber. 1996a. “Saving Babies: The Efficacy and Cost of Recent Expansions of Medicaid Eligibility for Pregnant Women.” Journal of Political Economy 104 (6): 1263–96. ———————, and ———————. 1996b. “Health Insurance Eligibility, Utilization of Medical Care, and Child Health.” Quarterly Journal of Economics 111 (2): 431–66.
1824
HSR: Health Services Research 48:6, Part I (December 2013)
Glied, S. 2002. “Managed Care.” In Handbook of Health Economics, edited by J. Newhouse and M. Pauly. Amsterdam: Elsevier. Graves, J. A., and J. Gruber. 2012. “How Did Health Care Reform in Massachusetts Impact Insurance Premiums?.” American Economic Review 102 (3): 508–13. Himmelstein, D., D. Thorne, and S. Woolhandler. 2011. “Medical Bankruptcy in Massachusetts: Has Health Reform Made a Difference?” The American Journal of Medicine 124: 224–8. Kolstad, J., and A. Kowalski. 2012. “The Impact of Health Care Reform on Hospital and Preventive Care: Evidence from Massachusetts.” Journal of Public Economics 96: 909–29. Miller, S. 2012. “The Impact of Massachusetts Health Care Reform on Health Care Use among Children,.” American Economic Review 102 (3): 502–7. ——————— (in press). “The Effect of Insurance on Emergency Room Visits: An Analysis of the 2006 Massachusetts Health Reform.” Journal of Public Economics 96 (11–12): 893–908. Newhouse, J. and the Insurance Experiment Group. 1993. Free For All: Lessons from the RAND Health Insurance Experiment. Cambridge, MA: Harvard University Press.
Editorial
Evaluating the Massachusetts Health Care Reform There are a number of reasons the Affordable Care Act (ACA) is unprecedented in its scope and reach. The law aims to introduce the largest expansion in insurance coverage in the nation’s history, to redistribute billions of dollars to make health insurance more affordable, to overhaul the regulation of insurance markets, and to introduce system reforms that transform the delivery of health care in the United States. Yet another unprecedented feature of the ACA is that it is perhaps the only massive expansion of our nation’s safety net which was directly piloted before passage. The state of Massachusetts passed a health care reform in 2006 which was the explicit model for the coverage and insurance market reform portions of the ACA. By the time that the ACA was passed in March 2010, there were nearly 4 years of experience with a similar law that could be studied to predict the ACA’s impacts. The Massachusetts law is not identical to the ACA for a number of reasons. First and most important, it applies only in Massachusetts, which is different from the nation as a whole in a variety of ways. Second, the subsidies provided to many low-income families in Massachusetts were significantly more generous than those offered under the ACA. Third, employers play a much larger role in financing the ACA than they did with the Massachusetts law. However, in the main, the basic structure of the ACA was pioneered in the Bay State. As a result, researchers have flocked to Massachusetts to try to gather a picture of what life might look like under the ACA. Such assessments have been carried out on a wide variety of measures with differing degrees of rigor. In this commentary I would like to touch on the broader methodological issues involved in evaluating the Massachusetts “experiment” before 1819
1820
HSR: Health Services Research 48:6, Part I (December 2013)
returning to the interpretation of the article by Bond and White in this issue of Health Services Research. A number of different methodological approaches have been applied to considering the evidence from Massachusetts. Least convincing are studies that simply compare outcomes in Massachusetts after reform to outcomes elsewhere in the nation. This naïve approach completely ignores the multitude of differences between Massachusetts and other states, such as higher rates of insurance coverage, a larger medical sector, and a higher income level. These differences dwarf any effects of health care reform when comparing Massachusetts to other states, so that this comparison cannot be credibly used to infer the impacts of reform. This approach has been used frequently in the press but fortunately not in academic studies of note. Somewhat more convincing are studies that examine how outcomes changed in Massachusetts from before to after reform (e.g., pre 2006 vs. post 2006). This approach controls for steady state differences in the state such as its generally higher rate of insurance coverage. However, it has the weakness that many other things changed around 2007—in particular, the Great Recession. So to simply compare outcomes before and after that time would miss trends that are completely independent of Massachusetts health reform.1 More convincing still are studies that compare the change in outcomes in Massachusetts to changes in other states over the same period. Such a “difference-in-difference” (DD) approach allows researchers to consider how reform impacted Massachusetts relative to other states which were subject to similar time series shocks in outcomes. That is, this approach controls for both long-standing differences across states and time series trends in outcomes. A wide variety of studies have used this approach. Some examples include Kolstad and Kowalski’s (2012) findings that reform led to reduced rates of preventable hospitalization; Courtemanche and Zapata’s (2012) findings that reform led to improved health outcomes along a number of dimensions; Miller’s (2012, in press) findings that reform led to more preventive care use and better reported health outcomes for children, as well as to an 8% overall decrease in emergency room visits; and Graves and Gruber’s (2011) finding that reform led to enormous declines in nongroup insurance premiums but no meaningful change in employer insurance premiums. However, such an approach is not perfect: it requires the key assumption that nothing else changed differentially in Massachusetts relative to other states other than health care reform. For example, it is possible that the Great Recession hit Massachusetts in a different way than other states, perhaps
Evaluating the Massachusetts Health Care Reform
1821
because of a different industrial composition across states. This could bias the estimates from a DD strategy. There are several approaches to dealing with this limitation, none of them perfect. One approach is to try to find states that are most comparable to Massachusetts in terms of the likely impacts of other time series factors, for example, regional neighbors or states with a similar industrial composition. A number of the studies cited above use this approach. Another is to find “within-state controls,” groups that are more or less impacted by the health care reform. For example, health care reform had much smaller impacts in areas of the state where private insurance rates were already very high, with larger effects on areas with more uninsurance. This brings us to the article by Bond and White. The question at hand is one of “spillover” effects of the Massachusetts reform: does covering more nonelderly residents with health insurance “crowd out” use of primary care by the elderly by overburdening primary care providers? This question is part of a larger concern expressed by critics of the ACA that expanding coverage to our nation’s uninsured residents will reduce access to care for those who already have insurance. The authors address this question through the difference-in-difference strategy discussed above, augmented with both of the approaches to dealing with the limitations of this strategy. The authors conclude that Massachusetts reform led to a significant decline in use of primary care by the elderly. I don’t agree that this conclusion is warranted by the results they present. Their basic difference-in-difference analysis does not support this conclusion, even when the analysis is restricted to geographically similar states. It is only when the authors dive in to consider heterogeneity by areas of the state that they find evidence of a reduction in primary care use by Medicare enrollees, and even in this context the evidence is somewhat confusing. The key facts are presented in Figure 1 (Graves and Gruber 2012), which shows the pattern of Medicare spending on physician and clinical services by state over time, for Massachusetts, the nation as a whole, and a set of Northeastern comparison states. The figure clearly shows that all three of these groups were on a comparable trend until 2005, suggesting that preexisting conditions were comparable. Starting in 2006, spending rises in Massachusetts relative to both the comparison states and the United States as a whole. By 2009, Massachusetts spending was slightly below the comparison states and equal to the nation as a whole. Certainly, this figure does not paint a picture of access problems for Massachusetts seniors.
1822
HSR: Health Services Research 48:6, Part I (December 2013)
Table 1 (Graves and Gruber 2012) further enriches the analysis by considering only primary care visits. This seems to strengthen the conclusion that health care reform increased utilization by seniors. Regardless of the weighting scheme used, primary care visits under Medicare rose in Massachusetts and fell in the set of comparison states. The authors’ conclusion regarding reduced utilization is based on Table 2, which takes the additional step of dividing the state’s zip codes by tercile of the uninsurance rates before reform; the natural presumption is that those zip codes with the highest rate of uninsurance saw the greatest rise in coverage. They find that in fact Medicare primary care visits fell by almost 7% in the set of zip codes with the highest rate of uninsurance before reform, and an equal or larger effect on those zip codes with the lowest rates of uninsurance (which are captured by the Massachusetts dummy variable). The authors focus on the former finding to conclude that reform led to reduced use of primary care by seniors. However, that conclusion is simply too strong. Indeed, the fact that Medicare primary care visits rise so significantly in the zips with the lowest rates of uninsurance is just as striking as the fact they fall in the zips with the highest rate of uninsurance. At best, I would characterize the overall results as mixed. The overall difference-in-difference comparison clearly shows an increase in use of physicians by Medicare patients. The zip-code level analysis just shows enormous heterogeneity in how this played out across Massachusetts zip codes. To conclude any more from the analysis would be premature. A more conservative reading of the same evidence would be that there is suggestive evidence that use rose among seniors in Massachusetts, but that evidence is not confirmed by the within-state analysis, leaving us uncertain as to the bottom-line result. This article does raise the interesting issue of how to trade off increasing access for the uninsured against potentially reduced access for the (elderly and nonelderly) insured. Existing evidence from health economics suggests that even if such a trade-off were to exist, making that trade would likely improve overall population health. For example, a large literature shows sizeable impacts of expanding insurance coverage on physical health (e.g., Currie and Gruber 1996a,b) and on mental health (Baicker et al. 2013). At the same time, there is little evidence that raising patient costsharing (e.g., Newhouse 1993) or limiting provider access through limited networks (e.g., Glied 2002) has any meaningful impact on health. Moving from no or very limited primary care to regularly available primary care (such as would happen for the uninsured) almost certainly improves health
Evaluating the Massachusetts Health Care Reform
1823
more than moving from more easily to somewhat less easily accessed primary care. Ideally, however, society need not face such a trade-off. The medical system could readily expand its capability to deliver the necessary primary care by lowering restrictions on who can deliver such care (e.g., practice standards for nurse practitioners) or expanding medical training (one feature of the ACA). The era of studying health insurance expansion is far from over. The ACA will have widely varying impacts across states, particularly with respect to the states’ decisions to expand Medicaid. The type of difference-in-difference methodology described above can be used to further examine the impacts of insurance expansion on a variety of outcomes, including access of the insured to primary care. Hopefully future studies will help clarify whether there is really a trade-off between covering the uninsured and access for the insured. Jonathan Gruber
NOTE 1. For example, a recent study by Himmelstein, Thorne, and Woolhandler (2011) concluded that Massachusetts health care reform had not provided financial protection because the number of medical bankruptcies rose in that state. However, this comparison ignores the fact that the recession likely caused a large rise in medical bankruptcies nationwide at the same time.
REFERENCES Baicker, K., S. L. Taubman, H. L. Allen, M. Bernstein, J. H. Gruber, J. P. Newhouse, E. C. Schneider, B. J. Wright, A. M. Zaslavsky, and A. N. Finkelstein, for the Oregon Health Study Group. 2013. “The Oregon Experiment --- Effect of Medicaid on Clinical Outcomes.” New England Journal of Medicine 368: 1713–22. Courtemanche, C., and D. Zapata. 2012. Does Universal Coverage Improve Health? The Massachusetts Experience. NBER Working Paper #17893, March 2012. Cambridge, MA: National Bureau of Economic Research. Currie, J., and J. Gruber. 1996a. “Saving Babies: The Efficacy and Cost of Recent Expansions of Medicaid Eligibility for Pregnant Women.” Journal of Political Economy 104 (6): 1263–96. ———————, and ———————. 1996b. “Health Insurance Eligibility, Utilization of Medical Care, and Child Health.” Quarterly Journal of Economics 111 (2): 431–66.
1824
HSR: Health Services Research 48:6, Part I (December 2013)
Glied, S. 2002. “Managed Care.” In Handbook of Health Economics, edited by J. Newhouse and M. Pauly. Amsterdam: Elsevier. Graves, J. A., and J. Gruber. 2012. “How Did Health Care Reform in Massachusetts Impact Insurance Premiums?.” American Economic Review 102 (3): 508–13. Himmelstein, D., D. Thorne, and S. Woolhandler. 2011. “Medical Bankruptcy in Massachusetts: Has Health Reform Made a Difference?” The American Journal of Medicine 124: 224–8. Kolstad, J., and A. Kowalski. 2012. “The Impact of Health Care Reform on Hospital and Preventive Care: Evidence from Massachusetts.” Journal of Public Economics 96: 909–29. Miller, S. 2012. “The Impact of Massachusetts Health Care Reform on Health Care Use among Children,.” American Economic Review 102 (3): 502–7. ——————— (in press). “The Effect of Insurance on Emergency Room Visits: An Analysis of the 2006 Massachusetts Health Reform.” Journal of Public Economics 96 (11–12): 893–908. Newhouse, J. and the Insurance Experiment Group. 1993. Free For All: Lessons from the RAND Health Insurance Experiment. Cambridge, MA: Harvard University Press.
