Ask the Expert: The Difference between Qualitative and Quantitative U.S. Payer Research – with Use Cases

One of the biggest sources of confusion with my clients is whether to do small N or larger N qualitative payer research versus adding in Quantitative research. While the needs of each organization are different, here is my take.

Qualitative Payer Research:

One thing to keep in mind in any United States payer study is that the clear majority of lives in the U.S. are at least influence by the ‘Big Three’ PBMs. Estimates vary, but it’s clear that >80% of lives are covered by the biggest PBMs. Thus, I’ve made the argument that increasing the number of respondents can increase project budget without providing additional incremental insights. This is a little different if you find yourself with significant behavioral or attitudinal segmentation. But as I’ll demonstrate below, qualitative methods aren’t as good as Quantitative studies for market segmentation anyway.

So, what’s the right number of respondents? I consider the specifics company confidential to Chiral Logic – and I have a surprisingly high number of readers of my blog from competitors (thanks guys!) – but the basics are, if you can get 50% of lives for an early stage asset (and that’s all the money you have for a study…) you MIGHT be alright. Following the bouncing ball – you can get a ton of marketing insights, especially if response is homogeneous, from 10-12 conversations. The trick is you need the RIGHT Payers. It helps if your moderator has years of experience and isn’t intimidated to push back.

Large N (25-30 respondents) projects are often requested by pharma companies who want to go the extra mile. And, if your guide is too long to get through in an hour, or if you expect heterogenous responses, this might make sense. We moderators hate these studies though – it’s very difficult to stay excited about a project after the 25th HOUR of asking the same questions to different payers. Beyond that, you’re retreading the same path, repeatedly…


Quantitative Payer Research:

Quant is great for market segmentation. It’s ESSENTIAL to translate your qual research into the forecast and your qual was reported by respondent. (Chiral Logic doesn’t report our findings by respondent, instead using a methodology that can be directly tied to forecasting). Unfortunately, quant is expensive, requires longer timelines, requires a different skillset & partners, and can return the same insights that were found in qual.


Typical Use Cases:

Development assets typically need a small N assessment to get a developing picture of the competitive landscape, likely restrictions both at launch and at maturity, net pricing, and what kinds of value substantiating programs will maximize access.

Before launch (using either information from FDA submission or expected equivalents), I like to complete a comprehensive payer qual study. This study should be sufficiently powered to uncover attitudinal and behavioral segmentation. One of the benefits of the Chiral Logic approach is that we can scale the study, while it’s happening, to generate the insights required.

Then, in cases where forecasting requires, a follow-on quant study can fill in the gaps and eliminate any strategic discrepancies. Also, there are some launches that are so important to the company’s future that no stone should be left unturned…and in these cases a qual/quant methodology can be optimal.

There’s always the problem of getting a ‘normal’ distribution of U.S. payers (something that we’ve been thinking about for years and have a reasonable work-around). But this, I’ll leave for another blog. As my thoughts on the inherent dangers of these developing on-line platforms where marketers can communicate ‘directly with payers…’

The Fallacy of the van Westendorp Methodology

The Fallacy of the van Westendorp Methodology, in general, and especially in Pharmaceutical Pricing

If you want to get a pharmaceutical pricing expert’s blood boiling, present the results from the van Westendorp method and then expect the same pricing team to scale the results into the pending (demand/board/plan) forecast. The reason this angers us is that it’s not possible.

For those of you not as familiar with van Westendorp, I will provide a summary: Van Westendorp was a true genius. I’m speculating, but I don’t think he ever thought his method would be applied to complex products in dynamic markets. He proposed that products could be priced by asking the same consumer the following questions:

  • At what point is this product too cheap, such that you’d question the quality?
  • At what point does the product provide value, such that you’d seek it out?
  • At what point would you find it expensive, but still pay for it?
  • At what point is the product too expensive, such that you’d seek alternatives?

Dr. Bruce Isaacson, from MMR Strategy Group wrote this excellent post in 2013.

The results from the methodology look like this:


The Y axis represents percent of respondents at any given (unit) price, shown on the X axis

The result of this methodology is AREA where the product provides value. Demonstrated as ‘Pricing Space’ in the above graphic. Conceivably the manufacturer can select any price in the acceptable area and be successful. But where to price? Where in that AREA provides the profit maximizing price? You need more research. But the graph is actually just a confusing mess – what does it say about pricing at £5.50? Why is that a price we should consider? A full 50% of respondents think it’s CHEAP! Thus, maybe we should start looking at the RIGHT most point of the grey area – but even that doesn’t work, because you have equal number of consumers thinking it’s CHEAP and TOO Expensive…confusing.

So…Some pricing experts (mainly consultants who want to bill you to produce the van Westendorp) say that this methodology gives them a good starting point in which to ask questions during qualitative interviews. And I suppose this is true. It’s completely unnecessary, of course, because we have cognitive and actual pricing comparator analogues for almost any developmental product, service, or device in health care. These analogs readily provide the groundwork for our pricing studies. One only needs to be creative in seeking these out when designing qualitative studies. And it is precisely against these cognitive and actual comparators that we’ll be selling against…so you’d better know where you stand.

My favorite counter-example that demonstrates the ridiculousness of Van Westendorp is a beer at a sporting event. I’m a behavioral, analytical pricer. I don’t give as much credence to what people think they WOULD do, especially in instances where I can measure what they actually do. I’ve never done the experiment, but I’d be willing to bet a month’s salary that 90%+ of fans in line to buy beer at sporting events would describe the beverages as “too expensive” – they’d much prefer to buy an alternative; but in that context, they can’t, so they readily buy a product that they KNOW is overpriced.

But there are two more problematic and fundamental problems with this method. First, despite being intuitive, there’s NEVER been an academic study that has confirmed that van Westendorp works as well as other methods (like discrete choice or conjoint).1 And furthermore, it’s extremely unlikely that one could be designed in the pharmaceutical arena – at least under the current global pricing landscape.

Think about it – ALL BRANDED PHARMACEUTICAL products are ‘too expensive’ for global payers. There’s a term for drugs that aren’t expensive: Multi-source Generics. Even more acutely, in the United States, where byzantine buffers between WAC and Net are required to appease PBMs and maximize profitability, payers will blanche at LIST prices that they will gladly ‘pay’ once discounts are put on the table. United States pharma pricing is simply too complicated to test using this overly simplistic method. Ah, but you say, we test the NET price in our van Westendorp. Well then, you’ve just lumped a plan with 100,000 lives in with CVS Caremark, ESI, and United. Pricing products with those mega payers pari passu to ‘Northeast Newark Carpet Associates Plan’ defeats the purpose of performing the research in the first place. Furthermore, doing weighted average pricing using a never-substantiated pricing methodology is akin to doing high-level probabilities for soothsaying.

So why do we see it used? Because the graph looks pretty. And it SEEMS analytical. It’s the difference between junior level, in-the-trenches pricing and Senior Executive box-checking. And it’s extremely difficult to put that much faith into the knowledge and expertise of the interviewers performing the study. But that’s what you must do. Pharma companies are at the mercy of these interviews. Payers ‘game the system’ and it’s much easier to put your faith in a pretty graph that SEEMS empirical, than to seek out the best, most knowledgeable interviewers you can find. (see below for quantification of how much van Westendorp might cost you…~35%? Yikes!)

Key takeaways:

  • Van Westendorp has never been academically confirmed
  • If your pricing people love it, ask tough questions and consider replacing them if they don’t understand this post
  • Resist the temptation to go with something that ‘feels right’ but doesn’t add to the discussion
  • Consider cognitive and actual comparators – in the end these are mental competitors to your product’s value proposition
  • It’s not tied to competitive response or net profit generation (even if weighted by lives)
  • Unfortunately, we actually buy things we think are outrageously expensive more often than we’d like – and this is especially true for U.S. payers
  • Find the best interview moderators you can find – they will generate better discussion guides too, but with the right interviewer the subtly will be captured and appropriate ‘push-back’ will be made

1 Comments are active. I will edit this post – and credit the commenter if anyone proves this claim false. If, for nothing else, that I’d love to see the academic reference. Note also, that if the reference is in the Pharma industry, I will post a redaction and delete the post. You might also wanna check with these guys at Pricing Solutions, who found van Westendorp to be ‘biased…and 35% low’.