RECENTLY we published an article here praising the introduction of the five-star rating system for nursing homes. We’re not stepping away from this, but we do think it’s important to point out a few of the weaknesses in and of this system.
The first weakness is one common to all rating systems which award a single overall rating. That weakness is simplification: the nursing home rating system tells you whether a nursing home is totally unacceptable (1-star), unacceptable (2-star), acceptable (3-star), good (4-star) or excellent (5-star).
A lot of information goes into arriving at such a rating. The rating will only be as sound as not only the information used in the calculations but also as the method of translating that information into a numerical score.
The new nursing home five-star rating system looks at four categories of performance: residents’ experience, compliance, staffing and quality measures. The MyAgedCare website shows the rating for each individual category and offers a link to how each category rating was calculated.
There are twelve questions, which are answered by a minimum sample of 10 per cent of residents. This means that in a typical nursing home of 100 beds, ten residents are surveyed.
About half of nursing home residents countrywide are cognitively impaired because of diseases like dementia. Many of them would be unable to answer the survey questions.
To get around that, questions can be answered by family, friends or visiting volunteers who are deemed to know the resident well. Clearly, the answers collected in this way are not always accurate, and in some cases, answers may be misleading.
For example, a family member may say the resident is happy in the nursing home because the family member doesn’t want to have the hassle or the additional expense of moving the resident to a better nursing home.
To take such wrong answers into account, a ‘data integrity’ measure is used when a nursing home’s overall star rating is calculated. This means that only 70 per cent of the score on residents’ experience is counted in the overall score.
The responses to residents’ experience survey questions make up one-third of the overall star rating.
Another issue is the question residents are asked about how safe they feel: “Do you feel safe here?” Like all questions in this category, the response can be one of four responses: never; some of the time; most of the time; and always.
Residents who say they feel safe ‘some of the time’ are really saying they feel unsafe most of the time.
Those who are saying they feel safe ‘most of the time’ are really saying they feel unsafe some of the time.
Yet both those responses attract points in the five-star rating system. Two points multiplied by the proportion of residents who say they feel unsafe most of the time and three points in the case of residents who feel unsafe some of the time.
Clearly, a nursing home should not be rewarded with points at all for resident safety unless all residents are safe.
For example, 69 per cent of residents surveyed in an eighty-bed nursing home said they felt safe ‘always’. The remaining 31 per cent said, ‘most of the time’, meaning almost one-third of residents in this nursing home said they felt unsafe some of the time.
According to the five-star system, this nursing home got 3.38 points for resident safety. Note that the survey on which this score was based indicated just under one-third of residents were not always feeling safe.
Take away those 3.38 points, and this nursing home’s residents’ three-star experience rating (‘acceptable’) falls from an overall 39.28 to a two-star rating (‘improvement needed’) at 35.59 points. ‘Improvement needed’ means bad.
As you can see, arriving at a rating for residents’ experience is not straightforward.
Perhaps, resident safety should be a stand-alone subcategory within the five-star rating system. In combination with data collected using the Serious Incident Reporting System, an assessment of resident safety based on how safe residents feel can produce a more reliable measure.
Obviously, the five-star nursing home rating system wants to look like Google Reviews, which allows people to comment freely on how a business or service provider performed for them. Google Reviews may not be appropriate and create legal problems for government, industry and care recipients, but the current five-star rating system for nursing homes leaves a lot to be desired.
Keep the five-star ratings system, by all means, but put a program in place to improve it progressively and continuously.
