main page books and articles schaller health creed facebook testimonies search
menu main page what's new second opinion new patient meet doctor schaller location, travel

James Schaller, MD on Physician Rating Sites

Never Send a Non-Physician to Do Real Medicine—Science Flaws are Obvious in Rating Sites

In the past five years, non-physicians have been asked increasingly to define what is a good physician. While they might be good at determining if a physician listens to them, and is empathetic, even if we accept that patients are perfect evaluators of profoundly trained physicians, the sites currently in existence are filled with defects that would have their "ratings" tossed out of a High School science newspaper.

Below is some research emerging on the science flaws in rating sites, which include massive sampling bias with physicians who treat an average of 2,000 people a year. But our research and that of others has uncovered a few sample things to add to the paper below.

  • Well-known doctors are often rated by people who they have never met. The reasons do not matter. And this includes people who have no friend or relative who has met the physician.

  • Criminals, severely abusive parents or police reported for felonies by the physician's staff or the physician have rated or entered the reputation arena.

  • Physicians who do not take insurance are attacked for fees published and explicitly clear.

  • Patients who live outside the USA are deleted. Some of the best physicians in the world are in the USA, in some areas of medicine. Patients leaving their FULL NAMES, COUNTRY AND ADDRESS are even deleted. Apparently, the bias is to assume all physicians can be good, but not so good as to be internationally good.

  • Reporting has anger bias. Happy patients are rarely or never asked to post, and angry patients have a drive to destroy. So if 99% of patients are pleased, the one most agitated to post is the one angry.

  • Doctors know they cannot serve all the people who want to be patients so ignore ratings sites. But this may lead to trouble over time.

  • "Patients" can range from cleaners who were fired for not doing a good job, to past people the physician dated, to relatives who dislike the physician, to neighbors who were not given free advice or prescription call ins on a needy weekend medical problem.

  • Some physicians report negative rater patients terminated for violation of controlled drug contracts, are character disordered such as those with borderline personality disorder or have other medical or psychiatric rage, and while they can often be the profound exception in a medical practice. I suspect until 2012, they now control rating sites.

  • Some physicians are clearly masochistic. They will do anything to be liked. In human relationships this is impossible. Even the "great mother" or Mother Teresa was not pleased at times, because sometimes all she could do with a dying impoverished Untouchable was simply hold them while they died. My father had a bit of this excessive self sacrifice in him, and he kept his fees about 20 years behind the times. Of course this has resulted in an almost complete lack of retirement savings, and has limited his options in his final years and I would not assume his patients noticed his very low fees. I suspect some did notice after he was gone, that no one will spend the time and offer his rate for services again for so little for the rest of their life. But at this point why rate a retired or retiring physician?

  • Some physicians are willing to treat hard problems and challenging individuals. In some settings, like Canada, hard patients are simply rejected by general practitioners according to many Canadian patients and friends living in Canada. If you treat only Catholic nuns, you tend to not get hostile ratings. If you treat someone who has had ten to twenty past unsuccessful physician experiences you are risking your career. It is utterly amazing that some do take the risk. And pay in ratings, as they become number eleven in the resentment history.

Physician-Rating Websites are Biased in Picking Doctors, Exaggerating Consumer Opinions, Says Paper

Published by S at 9:00 am under Healthcare Reform

School of Public Health at the University of Minnesota Paper: Physician-Rating Websites are Biased in Picking Doctors, Exaggerating Consumer Opinions, Says Paper to be Presented at INFORMS Healthcare Conference June 2011

For several years, Medical Justice has made the case that most doctor rating sites have several fundamental flaws; many lack statistically a significant sample size of patient ratings which impacts the quality of information available to consumers. Doctors see between 1,000 to 3,000 patients a year, depending on specialty. Yet, most doctors only have a handful of online ratings. A paper, "The Information Value of Online Physician Ratings", being presented at a healthcare conference sponsored by the Institute for Operations Research and the Management Sciences (INFORMS®) supports this point.

"The Information Value of Online Physician Ratings" is co-authored by Ritu Agarwal, professor of information systems and director of the Center for Health Information and Decision Systems, Guodong "Gordon" Gao, assistant professor, and PhD candidate Brad Greenwood, of the University of Maryland's Robert H. Smith School of Business, along with Jeffrey McCullough of the School of Public Health at the University of Minnesota.

The authors investigated potential biases among Internet users rating general practitioners on websites. In particular, the study looked at which doctors patients chose to rate, how they rated those doctors and the intensity of the patients' opinions.

Agarwal states, that physicians are concerned whether "these ratings a true measure of clinical quality." Or are they just the rantings of a disgruntled minority.

In a podcast on The Science of Better, Agarwal explains "patients may not always agree with their (doctor's) recommendation. If you don't agree and are disgruntled, you may go online and complain about a procedure that was perfectly valid given the medical condition that you had."

Agarwal also addresses the validity of online ratings. "If people who are happy don't bother to go online and provide a rating, then you are going to get a skewed distribution of ratings online and the impression that you would walk away with is the doctor is not a very good doctor."

The study's findings support these concerns about bias in the current online rating environment. Among the authors' conclusions:

  • Patients posting their opinions about doctors on online ratings are more prone than offline populations to exaggerate their opinions
  • Online ratings are more informative when identifying low-quality physicians, but not as effective in discerning high quality physicians from median ones.
  • Patients are most likely to provide ratings for their most flagrant or negative experiences with physicians.

Agarwal states "Patients need high quality information about the most consequential service that they consume: healthcare."

We agree.

That is why Medical Justice is working with select partners to help create a system that will balance the legitimate needs of patients with the reasonable concerns of doctors; a system that will provide much needed "quality information." Medical Justice/Dental Justice is leading the charge to promote ratings systems that include a critical mass number of credible reviews to provide the most accurate picture possible of a doctor's practice.

A Sample Response:

One Response to "Physician-Rating Websites are Biased in Picking Doctors, Exaggerating Consumer Opinions, Says Paper"

R.P 29 Jun 2011

I have read the reviews of some of the physicians at online physician review sites of physicians I have been to for treatment after I received treatment. From what I have found is that as stated here disgruntled patients will post and hide behind the cloak of anonymity promised by some sites to be totally denigrating beyond belief. In some instances the disgruntled patient may even attack the physician's personal appearance in very demeaning manners that in one case was not even close to the physician's actual personal appearance.

Another case where the physician's practice is treating the mentally ill, some of the reviews were scathing beyond belief. It was apparent that the psychological aberration of the patient(s) came to the surface and the patient derived some sort of empowerment by demeaning the physician. What I later found was that the patient went to all the online review sites and laid their anger at the world out but focused on the treating physician in writing as the cause for all their misfortunes in life.

In one case the "reviews" were degrading beyond belief. I left my e-mail address within the body of my faux negative review of the physician and the perpetrator contacted me. What she confessed is that she was never a patient of the physician but an ex-girlfriend who was expressing her jilted scorn by posing as a patient.

I believe it can be very damaging to allow such libelous comments to be printed when the physician has no recourse or recovery. I believe it to be very irresponsible of physician review sites to allow patients with "issues" and "axes to grind" complete anonymity to post.

I am sure it will be all but impossible to discover how a negative review may affect a physician's patients or prospective patients but review sites do need to take more responsibility to insure the negative reviews are not baseless fabrications by the mentally ill or someone who was not a patient as I found was simply angry with a physician from a personal level.

Bank Towers, Tamiami Trail, Naples, FL
disclaimer privacy