The Necessity of Quality Measures When Profiling Thought Leaders in Medicine

Industry Insights from Paul Meade, M.Sc., MPH

There are several companies that claim to identify and profile key opinion leaders in healthcare, and they using an array of methodologies and techniques to arrive at defining the practitioners who drive a given therapeutic area or disease state. And yet, while all claim to have built a better mousetrap for profiling these thought leaders, what is sadly lacking among many these vendors is quality control and good quality measures.

Notwithstanding the variety of definitions of a key opinion leader emblematic of the lack of standardization across the industry, there is also a wide variation in the quality of the output from one vendor to the next. For instance, some companies utilize automated web-crawlers (think: glorified Google Alerts) to mine public information—for the most part, scientific publications data, and simply foist upon their clients a “data dump.” The content of these data dumps is often completely devoid of sound quality control—for example, it is not uncommon for an IT-based vendor to offer up John Smith, John D. Smith, and JD Smith as three separate KOLs, when, in reality, Dr. John D. Smith is a true one-of-a-kind. Likewise, we have also seen lists where vendors may conflate Dr. David Lee from Sacramento with Dr. David Lee from Shanghai, when human researchers would see these to be different doctors with a rather simple QC exercise.

And while some companies take the effort to “scrub the data” and fix these more obvious errors, many do not. The result of these vendors lacking quality control, or a system of quality measures in their standard operating procedures, means that companies purchasing this information receive an inferior product. Unfortunately, enough of these types of vendors exist on the global KOL research stage that many of the people who need great research on leading physicians and other HCPs are starting to think that all KOL vendors paint with the same sloppy brush.

Sometimes a company’s lack of quality control is in the actual construction of its core methodology.  As with the previous example of the web-crawler companies, when companies lack the people power of industry veterans, there’s really no way to properly ensure quality control in the core methodology.  We’ve seen companies who claim a new innovation, such as the analysis of social networks, make rather unsound judgments at the expense of those who pay them.

These companies also take a rather simplistic view of the standards of excellence that thought leaders themselves define in the area of medicine, by simply reducing huge bodies of work down to publications searches and perhaps a few more data overlays from time to time. When you look at these types of searches, they typically yield high levels of individuals who don’t even play on the KOL stage, namely biostatisticians.  One recent study yielded a list populated by nearly 30 percent with biostatisticians. Then the vendor attempted to argue to the client that these people were indeed key opinion leaders, because they were listed as authors on important journal articles. I certainly do not wish to diminish the important role biostatisticians play in designing and analyzing the results of a research study, but to label them KOLs by the traditional definition is sheer nonsense. It only serves to demonstrate the lack of industry or medical knowledge that these vendors have with regards to understanding the needs of their clients, and this obviously undermines the quality of their findings.

Another example of a methodology that lacks good quality measures can be found in the ways that some vendors using peer nomination as a means of identifying thought leaders in medicine. I recently saw an assessment where a noted and respected thought leader responded to a survey that asked him to nominate other key opinion leaders like him. He dutifully nominated six other KOLs; however, five out of the six were colleagues within his own medical facility! Now, while these five colleagues may have been KOLs in his mind, it’s pretty easy to see that there was a bit of friendly bias in his nominations. And since the vendor lacked good quality measures, they simply passed along these nominations to the client without questioning or validating these nominations. It happens all the time.

The bottom line here is that unless vendors begin to adopt some standard measures of quality control to verify and validate their information, those buying such information at biotech, pharmaceutical, device, and diagnostics companies are at the mercy of people who are, at best, sloppy and unscientific, and, at worst, highly unscrupulous. Yet, there are a few companies that take pride in the quality of their work, and do take the time to rigorously “clean the data.”

We have built our team at Thought Leader Select, mindful of this very specific purpose—forming a collection of people who have extensive healthcare and industry backgrounds, people who understand the information they gather on the medical profession from start to finish with each client project. We recognize the difference between a biostatistician and a principal investigator, and where each one contributes value to the process of drug development. Why—because some of our people have actually worked in some of these contexts, with some of these healthcare professionals, prior to coming on board with Thought Leader Select.

I am proud to say that our team builds strong quality measures (it’s an ongoing process of tweaking our high standards) throughout our identification and profiling methodologies and we attempt to validate all of the information we gather on thought leaders. It’s simply the right thing to do, not only by the companies who hire us, but to give a proper accounting for the thought leaders themselves! We know that decisions being made on the basis of accurate profiling information can have an enormous impact on our clients. It’s logical to surmise that if more vendors claiming to have sound methods for indentifying and profiling key opinion leaders adopted some good, common sense quality measures, there would be less variation in the quality of the data. But while these companies either choose wanton ways of doing business or simply bask in their own ignorance of that which they assess, industry clients can choose to work with those companies that demonstrate, in an ongoing commitment, to adopt the principles that define quality control in key opinion leader research.

More Posts

Technology is Getting Smarter

By Paul Meade, Patient Engagement Lead  It has become apparent that in order for a healthcare organization to succeed, they need to be able to


Send Us A Message