Hospital executives and other stakeholders should be aware that a new research study1 challenges the idea that CMS hospital star ratings based solely on patient experiences can serve as a reliable indicator for clinical quality. This study follows a wake of industry criticism regarding a recent research letter published in JAMA Internal Medicine based on a
study approved by the institutional review board at the Harvard T.H. Chan School of Public Health. The highly controversial Harvard study suggests that consumers could choose five-star hospitals without a concern for their clinical quality, and a 5-star rating may in fact guide patients to better institutions.2 This conclusion is misleading and may actually steer patients to hospitals with poor clinical outcomes.3
The new study linked CMS hospital star ratings from Hospital Compare based on federal fiscal year 2014 Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) surveys with risk-adjusted outcomes data for the same period using the CMS Medicare Provider Analysis and Review File.4
The analysis reveals5 that hospitals with star ratings of two, three or four had more than three times the percentage of hospitals in the top 10% in the nation than five-star hospitals. Equally, if not more surprisingly, one-star hospitals actually outperformed five-star hospitals with 6% experiencing composite outcome scores in the top 10% of the nation (≥ 90th percentile) compared to only 4% of five-star hospitals.
These findings clearly confirm that consumers cannot safely assume that hospitals with a CMS five-star rating will provide better clinical quality than other star-rated hospitals. In fact, reliance on five-star ratings will place them at substantial risk6 of choosing a hospital that provides sub-standard care.
The study also investigates whether a positive correlation exists between actual patient experience scores and composite outcome scores across 3,456 star-rated hospitals based on their percentile ranking. The study indicates7 there wasn’t a positive correlation between patient experiences and composite outcomes. Instead, a slightly negative correlation was observed, but the results were not statistically significant.
After evaluating each individual outcome measure, the study found a significant negative correlation between mortality and patient experiences; although, the association was weak. A non-significant negative correlation existed between complications and patient experiences. Only patient safety had a positive correlation; however, the association was very weak.
Here are potential reasons that findings from the study performed at Harvard’s School of Public Health differs from the Quantros study:
- The Harvard study linked HCAHPS patient experience data from second quarter 2014 through first quarter 2015 with 2013 patient outcomes data from the 100% Medicare Inpatient File. This resulted in inconsistent data periods for determining associations (i.e., patient experience ratings were linked with outcomes associated with a completely different patient population).
- The Harvard study only assessed patient outcomes for mortality and readmissions across three conditions (acute myocardial infarction, pneumonia and heart failure) rather than evaluating a more extensive set of clinical outcomes across all conditions commonly treated by hospitals.
In July, CMS released another star rating system designed to measure the overall quality of hospital care using additional measures beyond patient experiences. The industry has already expressed concerns over the new methodology, indicating it puts certain hospitals, such as academic medical centers and safety-net hospitals, at a disadvantage because the data are not risk-adjusted and do not account for socioeconomic factors.8 More research will be needed to validate if these ratings provide a reasonable solution for measuring the quality of hospital care in an equitable manner.
The True Measure of Care: Safety and Quality
While it is commendable for hospitals to deliver positive patient experiences, the healthcare industry needs to recognize that CMS star ratings are a poor proxy for guiding patients to safe, high-quality care.
Despite flaws in the CMS star rating methodology,9 hospital executives should continue to seek ways to improve performance. They could look for areas where performance is most influenced by a rating, such as patient
complications, mortality or safety, using internal data to determine if there has been improvement in an area that is not evident because of outdated information. They should also focus on making improvements in a few key areas that will have the biggest impact on a rating.
That said, there is simply no substitute for measuring the actual safety and quality of care delivered. Quality and safety indicator measurements could be captured automatically and electronically to reflect true quality and safety using coding modifiers that identify when a medical event doesn’t meet specifications for a true failure or is present on admission.
This requires a unique blend of specific functionality that centers on three fundamental areas: safety risk management and surveillance, pay-for-value reporting and performance analytics. The combination of SaaS-based solutions and information services should be made available on a stand-alone or fully integrated basis to more effectively monitor and measure clinical and financial performance with precision and conviction.
When captured in a structured taxonomy, incidents of medical error could be aggregated and prioritized for performance improvement. Because errors and adverse events occur relatively frequently in healthcare, no organization can afford to maintain resources that target them all for improvement at one time. But organizations could efficiently and effectively learn
and improve through prioritization of their efforts following review of their data.
Information technology vendors are creating software solutions to automate outcomes data collection and aggregation, and embedding the standard sets in electronic medical records. A data platform to allow voluntary provider benchmarking and learning on a condition-by-condition basis is under development. Determining standard sets of outcomes for each medical
condition is a practical and necessary step for accelerating value improvement in healthcare.10
Furthermore, episode evaluation systems that span the entire continuum of patient care and have the capacity to capture all clinically related encounters and assign them to a single episode of illness regardless of care setting allow hospitals and providers to accurately compare benchmarks against peer groups, national norms and best practices. This gives them the power to measure what matters using meaningful and reliable information for assessing safety and quality care across the continuum of care.
1 “CMS Hospital Star Ratings Based on Patient Experiences Shown to Be a Poor Proxy for Clinical Quality.” Quantros. Aug. 15, 2016. 2 Wang D.E. Tsuawa Y, Figueroa J.F, Jha AK. “Association Between Centers for Medicare and Medicaid Services Hospital Star Rating and Patient Outcomes.” JAMA Internal Medicine. June 2016;176(6):848-850.
3 “CMS Hospital Star Ratings Based on Patient Experiences Shown to Be a Poor Proxy for Clinical Quality.” Quantros. Aug. 15, 2016.
4 Ibid.
5 Ibid.
6 Ibid.
7 Ibid.
8 Punke H. “CMS Releases Overall Hospital Star Ratings: 12 Things to Know.” Becker’s Infection Control and Clinical Quality. July 27, 2016.
9 “CMS Hospital Star Ratings Based on Patient Experiences Shown to Be a Poor Proxy for Clinical Quality.” Quantros. Aug. 15, 2016.
10 Porter ME, et al. “Standardizing Patient Outcomes Measurement.” New England Journal of Medicine. Feb. 11, 2016.
Originally posted on Accountable Care News
