The Surprising Things About User Reviews in Health Care

By Karl Ulfers | December 20, 2016 | Rally Health

User review illustration

For years, consumers have been sharing their experiences and opinions with each other through restaurant, product, and other online reviews — it’s now an essential part of our shopping experience. And now this same trend is beginning to be embraced by the health care system. In a couple of medical journal articles about doctor search earlier this year, the importance of patient reviews became clear.

“Yelp reviews actually cover more topics and may be more relevant to consumers,” wrote Health Affairs, which backed up an earlier report from the Journal of the American Medical Association. “Online portals provide instantaneous and public communication between patients, allowing more elaborate and timely data on health care professionals than traditional surveys,” said JAMA.

This syncs with our experience with our doctor search and cost transparency platform Rally ConnectSM and our new Find Care app — people are definitely embracing user reviews. But we’ve also learned some things about reviews that we didn’t expect.

At Rally®, we use reviews to optimize results both for the consumer and for the health plan. Say a health plan has a quality rating system for their doctors. We list the doctors with the highest quality first, and within the same quality rating we list doctors by how their patients have rated them. As a result, the top search results are doctors who have high quality as well as high patient ratings.

We made this update to our product after our data science team investigated which variables cause people to actually engage with a doctor. In other words, what causes someone to choose a particular doctor over another?

Like the JAMA and Health Affairs studies, we’ve found that the single biggest factor that drives somebody to click on a doctor, and then actually go and see that doctor, is a patient rating of four stars or above on a five-star scale. Our data shows that if a primary care doctor is rated four stars or above, there’s a 40 percent chance that a user will click through. For specialists like cardiologists, a four-star review still has an impact, but it’s lower — around 31 percent of users will click for a closer look.


We already knew that being on the first search result page increased the likelihood of a doctor being clicked on by 17%. Our new research is telling us that star ratings add to that effect. For example, in a search where 31 percent of doctors had a quality rating, over 51 percent of our users chose those doctors. The combination of the user rating stars and quality ratings gets people to prefer those doctors. And let me be clear that we are measuring this through actual data from the health care system; these are coming in from real health claims.

This makes sense if you think about it. Let’s say you’re shopping for something on Amazon. If you see two products side-by-side, and one is four stars and the other is three stars, assuming the price is about the same, you’re going to pick the four-star item every time. And since there really isn’t a cost savings associated with lower-reviewed providers, everyone’s going to pick the four- and five-star doctors.

But what really surprised us was that the user review had to be four stars or better. And if it wasn’t, the review basically had a negative impact. We were shocked, quite honestly. The negative impact of a one-star review makes sense, because it’s just a bad review. But an average review has essentially the same result. When it comes to reviews, basically a doctor either gets an A or an F. There really isn’t a lot of middle ground from the consumer’s perspective.

When we first launched Rally Connect, some of our clients were hesitant to include reviews, at least at first. It took some convincing, but by using industry research on reviews and showing how we could map to their own quality rating system we were able to convince them that this would increase engagement. Consumer interest in reviews has validated our approach. Since we launched reviews in March 2016, 5.5 million people have used the platform, with 3.5 million doctor reviews viewed. Granted, an individual may look at more than one review, but this still shows strong interest.

Such heavy use makes sense. As a consumer, when I use any service online, the first thing I do is look for reviews. I trust them more than I trust anything the company itself is telling me, because it’s coming from real people like me.

Understandably, some doctors are wary of reviews, because they worry about being unfairly rated, perhaps by patients who might be disappointed with their treatment for reasons beyond the doctor’s control. As a result, some payers are hesitant to put a review system in place for fear of angering their provider network. The good news is, as the Health Affairs article notes, is that doctors can relax about ratings, because for the most part, people rate their doctors highly. Ultimately, reviews are to everyone’s benefit. If you’re a good doctor, people are going to recognize that.

At Rally, we are convinced of the need for reviews, and so are our clients. And of course, consumers love being able to leave reviews that reflect their experiences. The only holdouts at this point are some doctors who are nervous about being reviewed. But in our experience, most physicians are coming around, and far more of them are open to reviews than even two years ago. They realize that in the end, user reviews are good for everyone. Even doctors.

Karl Ulfers is Vice President of Product at Rally Health, Inc.


Rally Health


Would you like to see more? Explore


Articles on Rally Health’s website are provided for informational purposes only, as a free resource for the public. They are not a substitute for medical advice, diagnosis, or treatment. Rally Health does not accept solicitations or compensation from any parties mentioned in the articles, and the articles are not an endorsement of any providers, experts, websites, tools, or financial consultants, services, and organizations.