The study, conducted by researchers at New York City's Hospital for Special Surgery, found it was "debatable whether these websites in their current form truly capture patient satisfaction and objectively evaluate the delivery of care," said Benedict Nwachukwu, M.D., who presented the findings at the American Academy of Orthopaedic Surgeons Annual Meeting in San Diego today.
The researchers studied ratings on Healthgrades.com, Vitals.com, and Ratemds.com and found "a low degree of correlation in ratings for individual surgeons on the different websites." That, said Nwachukwu, is an important finding that had not been previously demonstrated.
The inconsistency in ratings from one site to the next may be explained by the relatively low number of reviews submitted per doctor. In fact, the researchers noted, the number of reviews needed for a reliable rating is unknown and is a subject for more study. Also problematic is the unverified nature of the reviews.
The sites, said Anil Ranawat, MD, senior investigator for the study, may be good businesses but whether they're truly useful for consumers is questionable.
"Online rating websites are for-profit business enterprises, which at this point demonstrate significant growth potential," Ranawat said. "However, the low degree of correlation between these websites is concerning. It also questions the collective utility of these sites and potentially demonstrates the individually capricious nature of online physician reviews."
The study
To identify surgeons for the study, researchers accessed the online member directory of The American Orthopaedic Society for Sports Medicine. Their query in May 2015 yielded 2,813 entries. Investigators selected every tenth surgeon profile on a continuous basis and came up with 275 sports medicine surgeons to include in their study.
The researchers compiled data on years in practice, location, academic affiliation, and ratings for each surgeon on the three websites. Patients' written comments were categorized as relating to surgeon competence, affability, and the process of care delivery.
With respect to factors that appeared to influence ratings, being female was a significant predictor of higher ratings on Healthgrades. Surgeons with an academic affiliation were also more likely to receive higher overall ratings.
A physician's online and social media presence, including Facebook, Twitter, and possession of a personalized website, did not influence the strength of ratings on Healthgrades, Vitals, or Ratemds. Surprisingly, across all three websites, an increased number of years in practice generally led to lower ratings.