Beyond the primary outcome: How exploratory analysis can deliver unexpected new insights

The company has done everything right. It has formulated a product using existing peer-reviewed literature, vetted the claims with third-party subject matter experts, and designed the study based on industry best practices. And yet, the findings were not statistically significant. What happened and what’s next?

Let’s rewind the clock and explore. Clinical trials require significant time, effort, passion and budget, often with critical business decisions, funding and jobs depending on the outcomes. Disappointment, frustration and even anger are all too common when results fall short of expectations.

While in an ideal world science focuses on learning and exploration, that’s not always the reality in business-driven research, including the substantiation studies conducted for the dietary supplement industry. When the financial stakes are high, the unforeseen and unwelcome outcomes prompt a key question, "What now?" This is especially the case when trial success is viewed in black-and-white terms. But it doesn’t need to be that way.

The following playbook can help companies pave a data-driven path with the power to bolster both the science and marketing behind their products. 

Reassess the study design and trial process

First, examine the possible reasons behind the disappointing results. Start by revisiting whether the correct endpoint was studied. This examination is especially relevant when the existing literature on the active ingredients is scarce, inconclusive, a novel application is being targeted or a product is intended to improve a complex area, such as cognitive function or GI (gastrointestinal) health.

Related:Clinical trials: Putting the proof back in health claims

Next, investigate confounding variables that could have influenced the results. Conduct a thorough dosing analysis to reassess critical factors, such as amount, timing and usage instructions. And make sure to revisit the inclusion and exclusion criteria; different subpopulations can respond to formulations quite differently.

Venture beyond the binary placebo comparison

Next, a firm should explore the data it possesses to identify actionable findings beyond the powered outcome. This can be a treasure trove, especially with large-scale studies featuring heterogeneous participant populations. When primary endpoint analysis does not yield statistically significant results, post hoc analysis can be a valuable tool to further analyze the responses from various trial participants to explore potential correlations. While post hoc analysis alone cannot substantiate claims, it can deliver valuable insights for designing future trials that have higher likelihood of producing statistically significant results. Additionally, such exploratory findings can be valuable in effectively positioning a product to a firm’s target audience. Here’s a place to start: 

Optimize product potency/dose 

Real-world product usage data from a trial yields gold. Conduct statistical analysis on subgroups of participants based on consumption behavior to identify revisions to product potency or usage instructions that could potentially optimize effects for target segments.

Establish safety benchmarks

Understanding the range of possible side effects (or lack thereof) compared to placebo can help a brand make valuable claims about the safety of its product and build trust. Use the side effect data collected in the trial to add to the safety profile, leveraging the incidence of types of side effects, severity of side effects and side effects compared to the placebo group.

Substantiate non-health claims 

While all claims must be substantiated, non-health claims require a much lower bar of evidence than health-related claims. Leverage trial data on the experience and impression of participants who ingested a product to discover and substantiate non-health claims. These claims may be related to sensory enjoyment of, willingness to pay for, and willingness to recommend a product.

Redirect marketing efforts and targeting

Review the consumer insights and testimonials to assess participant impressions of a product. Analyze the diverse participant dataset to identify specific subpopulations who are more likely to experience a positive health effect from a product. Mine the data to reveal which subpopulations may be more likely to pay for or recommend a product. Leverage the analysis to adapt target marketing to subgroups more likely to have higher customer lifetime value to optimize customer acquisition costs.

Leverage insights from the exploratory analysis to conduct a follow-up clinical trial using subpopulations (e.g., age groups, gender, ethnicities) more likely to experience benefits, health areas more likely to be impacted by the product or revised product formulation/potency/usage instructions. Additionally, analyze qualitative participant feedback using natural language processing (NLP) models to identify patterns and novel insights, such as unexpected benefits and side effects, or even guidance on taste or smell, that may inform R&D strategy.

Unexpected results create new opportunities. Pivot brand positioning or product development efforts toward adjacent health areas identified in the trial where the product may have greater effectiveness.

Create a continuous customer feedback loop

Establish a system for continuous collection of consumption, side effect and health outcome data from existing customers/users of the product, and leverage these real-time, real-world insights to help guide future product development, marketing strategy and follow-up clinical trials.

Large and heterogeneous datasets open a whole new world of opportunities to assess product effectiveness for different populations, unearth hidden correlations and explore variances in consumer perceptions. It’s this kind of data-driven decision-making that’s powering breakthroughs in R&D and personalized marketing.

Remember how much “serendipity” has played a pivotal role in scientific discoveries in the past. Let’s increase the odds of manifesting serendipity with rich data sets and a willingness to dig in. We urge industry to go beyond the black-and-white success metrics of yesteryear because we can. Ultimately, all data is good data if you can learn from it and act on it.