Breaking new frontiers: Assessment and re-evaluation of clinical trial design for nutraceuticals

Introduction

Nutraceuticals, such as dietary supplements, are defined as products isolated or purified from foods that are generally sold in medicinal forms not usually associated with food and demonstrated to have a physiological benefit or provide protection against chronic disease (1). However, due to their multifunctional nature, study of their mechanisms of action, safety, and efficacy through clinical studies is challenging and new approaches to their investigation are warranted (2).

From a regulatory perspective, nutraceuticals are categorized under a variety of terms, depending on the country's regulatory framework. For example, in Canada there are natural health products (NHPs) and supplemented foods, under which nutraceuticals can be included. The European Food Safety Authority (EFSA), US Food and Drug Administration (FDA), and Health Canada use evidence-based reviews to evaluate the strength of scientific evidence that support marketing claims for foods and dietary supplements, specific to the terms used by each country. This review process assesses causation and is largely based on evidence-based medicine (EBM), a concept where the underlying hypothesis is that the intervention mitigates a condition in a causal manner (3). EBM aims to integrate the best available evidence into the decision-making process. Central to the concept of EBM is the RCT, which permits strong causal inference between an intervention and its outcome. The RCT is central to the regulatory process for both the approval of drugs and health claims of nutraceuticals, making it a critical piece in the evaluation of these products.

This perspective will discuss limitations of the RCT and will show that the traditional RCT drug model is neither sensitive nor relevant to nutraceuticals because (i) true placebos are not possible, (ii) effect sizes in ‘healthy' volunteers are most often modest, (iii) statistical analyses need to be refocused, (iv) endpoints must be global or multifunctional, (v) proof of efficacy should advance to that of probable harm due to lack of the intervention, and (vi) of the requirement for ‘healthy' participant enrollment. Therefore, we propose N-of-1 studies as Level 1 evidence better suited for nutraceuticals.

The RCT drug model is neither sensitive nor relevant to nutraceuticals

Currently over 400,000 RCTs, are being conducted throughout the US and in 220 other countries (4). Case series studies, case reports, and big data continue to inform routine care of patients (5–8). For example, adoption of some surgical techniques were not based on RCTs but compelling visual evidence (9, 10). It was estimated that 49% of drugs approved by FDA during 2005–2012 were based on “surrogate endpoints” rather than clinical outcomes (11). This suggests that biochemical changes that may or may not lead to clinical improvements were also found to be sufficient for approval of drugs despite the lack of clinical evidence (11).

Areas of research such as in psychotherapy where individualized interventions cannot be generalized or applied via the RCT have been accepted in treatment modalities (12, 13). Previously, RCTs failed to inform accurately, examples being tolbutamide (anti-diabetic drug), which led to secondary failure of response in patients (14) and the ALLHAT trial on thiazide diuretics (15, 16). The “shaky conclusions” of ALLHAT studies that were cloaked in EBM reported chlorthalidone as superior to the existing “gold” standard diuretics (17). Results from RCTs incorporated into care were later found to be inaccurate or insufficient for reliance (18). However, systematic reviews and meta-analyses are considered the new gold standard (19). Bothwell et al. stated “even though RCTs were developed to produce generalizable and universal knowledge, they have remained entangled in local social, economic and political conditions” (20). Due to their exorbitant cost, publishing positive results became a focus, resulting in an imbalance in medical knowledge, leading to publication-biased evidence (21–24).

Easily obtained victories for pharmaceuticals have been based on crowd-based medicine efforts such as vaccinations and other population-based interventions. Indeed, though RCTs were originally designed to decrease bias in research, they have become a point of “conflicting interest” (25, 26).

Sackett et al. stated: “individual patient care is not restricted to randomized trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions” (27). However, EBM appears to have been extended into evidence-based nutrition (EBN), making thoughtful and reasonable assessments of nutraceuticals challenging.

The application of RCTs to disciplines not related to testing drugs, such as foods, beverages, and dietary supplements, is fraught with challenges, the most prominent of which are the inherent differences between drugs and nutrients. Drugs are directed toward treatment of disease. They have isolated functions and are designed to target single organs or tissues and are not homeostatically controlled by the body. Unlike most drugs, nutrients work in complex networks, target all cells and tissues, and have multifaceted effects and outcomes. Therefore, RCTs for nutrients must be designed to capture this multifunctionality. Since many nutrients are homeostatically regulated, the body's baseline status affects the response to a nutraceutical intervention. For those not homeostatically regulated, excessive amounts are either not bioavailable, or are metabolized relatively quickly and/or eliminated, thus leading to their generally high level of safety.

True placebos are not possible in nutrition studies

Large effect sizes can be expected in drug studies, intended to show superiority or comparability, as there is a no-intake control group or a true placebo. There is a distinction in the reduction of the symptoms of disease, which is present in the enrolled population and the absence of the intervention does not cause disease (23, 28).

In contrast, true placebos are not possible with nutraceutical studies and the interventions involve the intake of nutrients, the absence of which may cause disease. In the absence of a true placebo group, and the requirement of ‘healthy' participants, nutrients have a smaller effect size and a longer response time to detect health outcomes. Essential nutrients are necessary for health and thus the underlying hypothesis that low or inadequate intake of nutrients causes or contributes to disease (29). Heaney stated, “with nutrients, the question is always not ‘whether' but ‘how much'?” (29). As such, EBN is a complex and intricate puzzle. Only one function of a nutrient (the first to appear and lead to disease or death) is used to define the disease (i.e., deficiency syndromes like beriberi and scurvy). But our understanding of all the other functions (beyond deficiency) is where we find the challenge of demonstrating the many benefits of a single nutrient when the proof of efficacy is limited to EBM (29).

Effect sizes are modest: Contending the requirement for ‘healthy' participants

The regulatory requirements of enrollment of populations “defensible as healthy” in studies intended for substantiation of structure/function claims limit the purpose of nutraceutical investigations. This accentuates the small effect size, decreasing the gap between the placebo/control and intervention (30). In the absence of an acceptable definition for ‘healthy,' a regulation that uses such a guideline to reject a study for claims is questionable. Changing medical treatment algorithms (31) that continuously redefine healthy populations while moving the dichotomy in the definition of health vs. disease and advocating earlier pharmacological treatment narrows the population that may be called ‘healthy.' Not only are these restrictions impractical for participant recruitment, but they are also not conducive for moving markers of respective outcomes and highlights flaws in logical application.

Persistent advocacy and requirements result in findings of clinical trials being limited to narrow and unrepresentative populations. This is a disservice to consumers seeking alternative solutions to improve their health. More recently, the concept of ‘healthy' has been upended by investigations showing that those previously considered ‘healthy' may in fact be metabolically unhealthy (32). A paradigm change is needed in defining the term ‘healthy' when applied to the nutraceutical industry.

Statistical analyses should be refocused: Statistical vs. clinical significance

The tendency of scientists and researchers to rely and focus on p values as the “gold standard of statistical validity,” above and beyond other frameworks for data analysis, which include statistical power, false positives and negatives have been challenged (33). The p value does not address the question if the probability of the study hypothesis is correct in the first place, and furthermore does not account for the actual size of the effect. Nuzzo argued that “researchers need to realize the limits of conventional statistics” and “bring into their analysis elements of scientific judgment about the plausibility of a hypothesis and study limitation” (33).

Recent statements by the American Statistical Association (ASA), EFSA, and New England Journal of Medicine (New Engl J Med) point away from such requirements that are founded on misinformation and urge movement toward providing clinical significance. “The p value was never intended to be a substitute for scientific reasoning,” said Wasserstein, stating “Well-reasoned statistical arguments contain much more than the value of a single number and whether that number exceeds an arbitrary threshold” and is a directional move away from and toward a “post p < 0.05 era” (34).

The EFSA Scientific Committee suggested that statistical significance should not be the primary objective in analysis and focus should be directed on confidence intervals and biological relevance (35). New Engl J Med followed with a statement regarding publications and p values stating that nutrient studies should be directed to solve a health indication, not for statistical convenience (36).

Endpoints should be global and multifunctional

The multifunctional nature of nutraceuticals and their multi-targeted outcomes are not readily determined within the scope of the current RCT model. These multifunctional effects could be considered a complex intervention (37) and are not often captured in the setting of a single primary endpoint, resulting in the intervention being found ineffective and leading to higher proportions of false negatives. The Medical Research Council of the United Kingdom stated that complex interventions require a different approach. While a single primary outcome is usually the most straightforward for statistical analysis, it may not provide adequate assessment of the success of an intervention that has effects across a range of indications (37).

A global outcome that assesses and scores the effects of an investigational product across systems corresponds more closely to the multifunctional outcome proposition of nutrients in the human body. When designed to reflect health rather than disease, a global outcome captures systemic effects (37). Previously, such a concept has been proposed as a composite of several outcomes providing a higher statistical power to the study to detect small changes across multiple organ systems (38).

Global indices of health are appropriate in the nutritional sciences realm since functional foods, nutraceuticals, and other NHPs do not fit into the pharmaceutical model. A global index is different from a composite of several outcomes. A composite index cannot identify the variable that contributes to the majority of the effects, and when combined as a primary outcome can also mask a real benefit of a single constituent endpoint by being diluted from multiple null effects within the cluster (39, 40). A global index weights each outcome allowing for a better and more informed assessment of the outcome(s) contributing to the results. Health-related quality of life questionnaires such as the 36-item Short Form Survey (SF-36) or Profile of Mood States (POMS), and objective biomarkers might be examined as an approach for creating global health indices that better capture the efficacy of the intervention. This has been previously reported where Antony et al. suggest combining outcomes of cognition with physiological biomarkers of immunity and metabolism to arrive at a global index for cognitive health (41).

Furthermore, the index should evaluate and quantify positive movement of a negative health marker. Such concepts have previously been evaluated in quantifying disease activity in patients with rheumatoid arthritis and includes patient history, laboratory tests, physical examination, and imaging studies (42). Multiple common indices, were used to assess adiposity, such as body mass index (BMI), bioelectric impedance analysis (BIA), and anthropometric measures like skinfold thickness and waist: hip ratio as well as computed tomography imaging (42). Thus, a global index as the primary outcome comprising multiple single endpoints could improve statistical precision, increase efficiency, reduce trial size and cost, and may provide trial results earlier (43).

Advancing from proof of efficacy to proof of probable harm

Heaney questioned “whether we need as much proof of efficacy for a nutrition policy decision as we do for approval of powerful, expensive and potentially dangerous pharmaceutical agents” (29).

Reductions in intake of a nutrient may lead to an increase in the risk of disease or may result in disease. It is unethical to lower nutrient levels to study efficacy endpoints. Heaney et al. proposed a shift from examining proof of efficacy to that of probable harm, a calculus of benefit compared to harm, to be evaluated on a nutrient-by-nutrient basis (29). Investigating pre-diseased populations where the placebo or control group presents either low or harmful levels of surrogate biomarkers may be beneficial in the application to nutraceuticals. Using the calculus of benefit vs. harm, the proof of harm is established in a pre-diseased population since they will eventually progress to a diseased state in the absence of an intervention. If the safety of nutraceutical interventions can be demonstrated through high-quality and comprehensive studies, then the calculus of benefit vs. harm of the intervention shifts toward benefit.

A recent study examined this concept in an RCT model and found that following the progression of the placebo group from baseline to the end of the study was valuable. Left untreated, an at-risk population progressed to disease, which was observed in the placebo group (44) confirming Heaney's concept (29). On further investigation and application of the Framingham risk score, it was found that left untreated, those with cardiovascular risk factors may progress to a more hypertensive and hypercholesterolemic state (44).

A small effect size and study populations, as well, the flawed expectations that nutraceuticals should behave like drugs often result in RCTs generating null results. If one were to stop at this point, most RCTs would suggest that the ingredient and or the formulation was not efficacious.

Therefore, application of EBM principles to EBN solely for meeting the standards of acceptable evidence for policy makers is contentious. Applying the principles of EBM to EBN in its entirety is troublesome and unsuitable for the assessment of nutrients, dietary supplements, and foods. A paradigm shift is required in trial design for evaluating nutrient principles and examining RCT designs that are better suited for nutraceuticals. Blanket application of the RCT concept to the nutraceutical model is not sensitive to establish health promotion and optimization and, in many instances, has led to an exercise in futility.

Thus, it would be reasonable to expect that the evidence required to prove efficacy for drugs and nutrients would need to be different. Both drugs and nutrients have different attributes in contributing to the welfare of an individual. Importantly, drugs are therapeutic in nature, whereas nutrients may not only reduce the risk for disease but may help promote optimal health and performance.

N-of-1 studies: The RCT at level 1 evidence stage

There are various considerations for designing high-quality studies for nutrition and nutraceutical research. Considerations have been previously presented by Lichtenstein et al. and the Agriculture and Agri-Food Canada “Best Practices for Food-Based Clinical Trials” (46, 47). Following rigorous design considerations, there have been large-scale, long-term, double-blind, placebo-controlled trials of dietary supplements including but not limited to the Age-Related Eye Disease Study (AREDS) and AREDS 2 (48), Alpha-Tocopherol, Beta-Carotene Cancer Prevention (ATBC) Study (49), Carotene and Retinol Efficacy Trial (CARET) (50), Cocoa Supplement and Multivitamin Outcomes Study (COSMOS) (51), Selenium and Vitamin E Cancer Prevention Trial (SELECT) (52), Supplementation en Vitamines et Mineraux Antioxydants (SU.VI.MAX) (53), VITamin D and OmegA-3 TriaL (VITAL) (54), Women's Antioxidant and Folic Acid Cardiovascular Study (WAFACS), Physicians' Health Study-I and II, Women's Health Initiative (WHI), and Women's Health Study (WHS). However, these large-scale trials are extraordinarily expensive, not always feasible, and mostly unnecessary in the context of evaluating the health benefits of nutraceuticals.

Previous research in nutraceuticals validates the importance of measuring responsiveness to treatment at the individual level (24). About 25% of participants responded positively to an improvement in blood pressure and approximately 33% to insulin sensitivity (24). An estimated 66-75% of participants were non-responders to the intervention, which indicated that the intervention worked in a particular subset of participants, a group that could not be identified prior to the study. Examination of group variability allows researchers to make informed decisions of the inter-individual differences of clinical relevance (55). Population-wide recommendations, currently in place for reduction in dietary sodium, were based on the benefits in hypertensive subjects and implementation of folic acid fortification programs were based on neural tube defect prevention (56). The evidence demonstrates a precedence exists for nutrition study results in a responder population that can be generalized to the greater population when positioned correctly. Studies on probiotic interventions for antibiotic-induced diarrhea have indicated similar results (57).

The history of N-of-1 show this type of study was developed due to the need for determining “optimal therapy” for a patient and proved to be “spectacularly helpful” in patient treatment (58). Since its publication, clinicians have reported on thousands of N-of-1 trials. These studies improved patient outcomes and drug development, expanded to encompass many indications (59), and have reported their value for conditions where conventional RCTs are available (60). N-of-1 trials are defined in part by randomizing multiple crossover trials in a single participant (61–64). This N-of-1 study design has been utilized or proposed in nutrition and nutraceutical study designs (65–69). For example, the Westlake N-of-1 Trials for Macronutrient Intake (WE-MACNUTR) examined the use of personalized dietary recommendations on glucose metabolism and gut microbiota in the prevention of chronic disease (66, 68). Further, Soldevila-Domenech et al. reviewed the use of nutraceutical interventions, including phenolic compounds, omega-3 polyunsaturated fatty acids, vitamin D, vitamin C and other micronutrients, aimed at improving cognitive function in older adults with the potential application to N-of-1 designs specifically for dementia prevention (69).

Recently, N-of-1 designs, as part of the umbrella of single-case experimental designs (SCDs), has been discussed in detail by a Nutrition Research Task Force of the American Society for Nutrition (70). They state that “to determine whether, and to what extent, people respond differently to interventions, different designs needs to be used” (70). A general example of an N-of-1 trial design used for comparing responses to two interventions is shown in Figure 1. The intention of this design is to utilize multiple N-of-1 studies in informing decision making.

FIGURE 1

Figure 1. The N-of-1 trial design used in comparing responses to two interventions. Adapted from Zucker et al. (45).

When compared to the traditional RCT, N-of-1 provides greater strength and more reliable data in clinical decision-making (60). Further, this design may provide specificity and inherently address the heterogeneity of treatment effects when compared to RCTs (70). Models have been expanded to accommodate crossover features including carry-over effects (71) and maintains randomization, blinding, and formal outcome assessment. The Oxford Centre for Evidence-Based Medicine has classified N-of-1 trials as Level 1 evidence, the same level of evidence for systematic reviews of RCTs (72). N-of-1 studies are a unique method identifying optimal intervention approaches for individual cases. Accumulation of multiple N-of-1 studies utilized in meta-analysis can be generalized across investigational populations (60, 70).

The N-of-1 trial allows for rapidly identifying responders vs. non-responders, requires enquiry into clinical significance, and provides a venue for the use of a global index outcome that is more generalizable to the nutraceutical industry (73). Furthermore, an N-of-1 at a Level 1 evidence stage allows for the measurement of probable harm in the absence of an intervention, provides clinical relevance, allows for the accuracy of detection of efficacy, and can function within the guidelines of the traditional RCT. Such a model would be of value in providing reliable and reasonable information regarding the intervention.

Conclusion

The conventional RCT model does not withstand the scrutiny required to make it useable as a source of evidence for the proof of efficacy of nutraceuticals. The N-of-1 RCT design obviates the limitations and lack of sensitivity of the traditional RCT used to test drugs in the therapeutic milieu. The N-of-1 design provides a more relevant control group, addresses concerns about effect size, can capture global or multifunctional outcomes, allows for advancing from proof of efficacy to that of probable harm in the absence of an intervention, and measures clinical significance. N-of-1 studies provide faster information for identifying optimal individualized interventions, and facilitating translational research by capturing principles that can be generalized to population subgroups (70). Accumulation of multiple N-of-1 studies utilized in meta-analysis can be generalized across investigational populations (70). This Perspective provides an argument for the application of the N-of-1 RCT design, an established Level 1 evidence stage, for proof from interventional studies of nutraceuticals.

Data availability statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Author contributions

ME and JB contributed to conception and design of the perspective. ME, EL, JA, and JB wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Acknowledgments

The authors wish to acknowledge Drs. Marc Moulin and Katarina Doma for their assistance in formatting and referencing.

Conflict of interest

Authors ME, EL, DC, JA, and NG are/were employed by KGK Science Inc.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Health Canada. Policy Paper - Nutraceuticals/Functional Foods and Health Claims On Foods. Therapeutic Products Programme and the Food Directorate from the Health Protection Branch (Archived 2002). ARCHIVED - Policy Paper - Nutraceuticals/Functional Foods and Health Claims On Foods - Canada.ca (2022).

2. Durazzo A, Lucarini M, Santini A. Nutraceuticals in human health. Foods. (2020) 9:370. doi: 10.3390/foods9030370

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Greene JA. Prescribing by Numbers: Drugs and the Definition of Disease. JHU Press (2007).

Google Scholar

4. ClinicalTrials.gov. Trends, Charts, and Maps. Baltimore, MD: John Hopkins University Press. (2022).

5. Brasil P, Pereira JP Jr, Moreira ME, Ribeiro Nogueira RM, Damasceno L, Wakimoto M, et al. Zika virus infection in pregnant women in Rio de Janeiro. N Engl J Med. (2016) 375:2321–34. doi: 10.1056/NEJMoa1602412

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clin Pharmacol Ther. (2007) 82:143–56. doi: 10.1038/sj.clpt.6100249

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. (2005) 58:323–37. doi: 10.1016/j.jclinepi.2004.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Varkey JB, Shantha JG, Crozier I, Kraft CS, Lyon GM, Mehta AK, et al. Persistence of Ebola virus in ocular fluid during convalescence. N Engl J Med. (2015) 372:2423–7. doi: 10.1056/NEJMoa1500306

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Grüntzig A. Transluminal dilatation of coronary-artery stenosis. Lancet. (1978) 1:263. doi: 10.1016/S0140-6736(78)90500-7

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012. JAMA. (2014) 311:368–77. doi: 10.1001/jama.2013.282034

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Feyman Y. Shocking Secrets of FDA Clinical Trials Revealed. Forbes (2014). p. 24.

12. Seligman ME. The effectiveness of psychotherapy: the consumer reports study. Am Psychol. (1995) 50:965. doi: 10.1037/0003-066X.50.12.965

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Marks HM. The progress of experiment: science and therapeutic reform in the United States, 1900–1990, Cambridge: Cambridge University Press (2000).

PubMed Abstract | Google Scholar

14. DeLawter DE, Moss JM, Tyroler S, Canary JJ. Secondary failure of response to tolbutamide treatment. J Am Med Assoc. (1959) 171:1786–92. doi: 10.1001/jama.1959.03010310018005

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Pollack A. The minimal impact of a big hypertension study. New York, NY: New York Times (2008).

PubMed Abstract | Google Scholar

16. Epstein S. Impure Science: AIDS, Activism, and the Politics of Knowledge. Univ of California Press (1996).

PubMed Abstract | Google Scholar

17. Meltzer JI. A specialist in clinical hypertension critiques ALLHAT. Am J Hypertens. (2003) 16:416–20. doi: 10.1016/S0895-7061(03)00570-3

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Feinstein AR, Horwitz RI. Double standards, scientific methods, epidemiologic research. N Engl J Med. (1982) 307:1611–7. doi: 10.1056/NEJM198212233072604

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Furunes T. Reflections on Systematic Reviews: Moving Golden Standards?, Taylor & Francis (2019). p. 227–231.

Google Scholar

20. Bothwell LE, Greene JA, Podolsky SH, Jones DS. Assessing the gold standard—lessons from the history of RCTs. N Engl J Med. (2016) 374:2175–81. doi: 10.1056/NEJMms1604593

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Burdett S, Stewart LA, Tierney JF. Publication bias and meta-analyses: a practical example. Int J Technol Assess Health Care. (2003) 19:129–34. doi: 10.1017/S0266462303000126

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Zarin DA, Ide NC, Tse T, Harlan WR, West JC, Lindberg DA. Issues in the registration of clinical trials. JAMA. (2007) 297:2112–20. doi: 10.1001/jama.297.19.2112

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Blumberg J, Heaney RP, Huncharek M, Scholl T, Stampfer M, Vieth R, et al. Evidence-based criteria in the nutritional context. Nutr Rev. (2010) 68:478–84. doi: 10.1111/j.1753-4887.2010.00307.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Heaney RP. Nutrients, endpoints, and the problem of proof. J Nutr. (2008) 138:1591–5. doi: 10.1093/jn/138.9.1591

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Bajwa SJS, Theerth KA, Gupta A. The increasing trend of observational studies in clinical research: have we forgotten and started defying the hierarchy? Indian J Anaesth. (2021) 65:186–90. doi: 10.4103/ija.IJA_176_21

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Probst P, Grummich K, Klaiber U, Knebel P, Ulrich A, Büchler MW, et al. Conflicts of interest in randomised controlled surgical trials: systematic review and qualitative and quantitative analysis. Innov Surg Sci. (2016) 1:33–9. doi: 10.1515/iss-2016-0001

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. (1996) 312:71–2. doi: 10.1136/bmj.312.7023.71

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Shao A, Mackay D. A commentary on the nutrient-chronic disease relationship and the new paradigm of evidence-based nutrition. Nat Med J. (2010) 2:10–8.

29. Heaney RP, Weaver CM, Blumberg J. EBN (evidence-based nutrition) ver. 2.0. Nutr Today. (2011) 46:22–6. doi: 10.1097/NT.0b013e3182076fdf

CrossRef Full Text | Google Scholar

30. Saldanha LG. US Food and Drug Administration regulations governing label claims for food products, including probiotics. Clin Infect Dis. (2008) 46:S119–21; discussion S144-51. doi: 10.1086/523328

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Margolis CZ. Uses of clinical algorithms. JAMA. (1983) 249:627–32. doi: 10.1001/jama.249.5.627

CrossRef Full Text | Google Scholar

32. Zembic A, Eckel N, Stefan N, Baudry J, Schulze MB. An empirically derived definition of metabolically healthy obesity based on risk of cardiovascular and total mortality. JAMA Netw Open. (2021) 4:e218505. doi: 10.1001/jamanetworkopen.2021.8505

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Nuzzo R. Scientific method: statistical errors. Nature. (2014) 506:150–2. doi: 10.1038/506150a

PubMed Abstract | CrossRef Full Text | Google Scholar

34. American Statisical Association. American Statistical Association Releases Statement on Statistical Significance and p-values (2016).

35. EFSA Scientific Committee. Statistical significance and biological relevance. EFSA J. (2011) 9:2372. doi: 10.2903/j.efsa.2011.2372

CrossRef Full Text | Google Scholar

36. Harrington D, D'Agostino RB Sr, Gatsonis C, Hogan JW, Hunter DJ, Normand ST, et al. New guidelines for statistical reporting in the journal. N Engl J Med. (2019) 381:285–6. doi: 10.1056/NEJMe1906559

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. (2008) 337:a1655. doi: 10.1136/bmj.a1655

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Goldberg R, Gore JM, Barton B, Gurwitz J. Individual and composite study endpoints: separating the wheat from the chaff. Am J Med. (2014) 127:379–84. doi: 10.1016/j.amjmed.2014.01.011

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Cordoba G, Schwartz L, Woloshin S, Bae H, Gøtzsche PC. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ. (2010) 341:c3920. doi: 10.1136/bmj.c3920

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Palileo-Villanueva LM, Dans AL. Composite endpoints. J Clin Epidemiol. (2020) 128:157–8. doi: 10.1016/j.jclinepi.2020.07.017

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Antony JM, Weaver I, Rueffer M, Guthrie N, Evans M. The essentials of a global index for cognitive function. Transl Neurosci. (2017) 8:87–96. doi: 10.1515/tnsci-2017-0014

PubMed Abstract | CrossRef Full Text | Google Scholar

42. McCollum L, Pincus T. A biopsychosocial model to complement a biomedical model: patient questionnaire data and socioeconomic status usually are more significant than laboratory tests and imaging studies in prognosis of rheumatoid arthritis. Rheum Dis Clin. (2009) 35:699–712. doi: 10.1016/j.rdc.2009.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Heaney RP. The nutrient problem. Nutr Rev. (2012) 70:165–9. doi: 10.1111/j.1753-4887.2011.00469.x

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Evans M, Lewis ED, Crowley DC, Zeng A, Struve J, Guthrie N. A randomized, placebo-controlled, triple-blind study to determine the effect of Farlong Ginseng Plus^® NotoGinseng extract on cholesterol and blood pressure. Curr Nutraceuticals. (2021) 2:e140721194774. doi: 10.2174/2665978602666210714131146

CrossRef Full Text

45. Zucker DR, Ruthazer R, Schmid CH, Feuer JM, Fischer PA, Kieval RI, et al. Lessons learned combining N-of-1 trials to assess fibromyalgia therapies. J Rheumatol. (2006) 33:2069–77.

PubMed Abstract | Google Scholar

46. Lichtenstein AH, Petersen K, Barger K, Hansen KE, Anderson CAM, Baer DJ, et al. Perspective: design and conduct of human nutrition randomized controlled trials. Adv Nutr. (2020) 12:4–20. doi: 10.1093/advances/nmaa109

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Agriculture and Agri-Food Canada Best Practices for Food-Based Clinical Trials - Guidance for Planning Conducting Conducting and Reporting on Human Studies to Support Health Claims. In: Food Regulatory Issues Division (ed) (2013).

Google Scholar

48. Writing Group for the AREDS2 Research Group. Effect of Long-Chain ω-3 Fatty Acids and Lutein + Zeaxanthin supplements on cardiovascular outcomes: results of the Age-Related Eye Disease Study 2 (AREDS2) randomized clinical trial. JAMA Intern Med. (2014) 174:763–71. doi: 10.1001/jamainternmed.2014.328

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. N Engl J Med. (1994) 330:1029–35. doi: 10.1056/NEJM199404143301501

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Goodman GE, Thornquist MD, Balmes J, Cullen MR, Meyskens FL Jr, Omenn GS, et al. The beta-carotene and retinol efficacy trial: incidence of lung cancer and cardiovascular disease mortality during 6-year follow-up after stopping beta-carotene and retinol supplements. J Natl Cancer Inst. (2004) 96:1743–50. doi: 10.1093/jnci/djh320

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Sesso HD, Rist PM, Aragaki AK, Rautiainen S, Johnson LG, Friedenberg G, et al. Multivitamins in the prevention of cancer and cardiovascular disease: the COcoa Supplement and Multivitamin Outcomes Study (COSMOS) randomized clinical trial. Am J Clin Nutr. (2022) 115:1501–10. doi: 10.1093/ajcn/nqac056

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Lippman SM, Klein EA, Goodman PJ, Lucia MS, Thompson IM, Ford LG, et al. Effect of selenium and vitamin E on risk of prostate cancer and other cancers: the Selenium and Vitamin E Cancer Prevention Trial (SELECT). JAMA. (2009) 301:39–51. doi: 10.1001/jama.2008.864

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Hercberg S, Galan P, Preziosi P, Bertrais S, Mennen L, Malvy D, et al. The SU.VI.MAX Study: a randomized, placebo-controlled trial of the health effects of antioxidant vitamins and minerals. Arch Intern Med. (2004) 164:2335–42. doi: 10.1001/archinte.164.21.2335

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Manson JE, Cook NR, Lee I-M, Christen W, Bassuk SS, Mora S, et al. Vitamin D supplements and prevention of cancer and cardiovascular disease. N Engl J Med. (2018) 380:33–44. doi: 10.1056/NEJMoa1809944

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Ward WE, Chilibeck PD, Comelli EM, Duncan AM, Phillips SM, Robinson LE, et al. Research in nutritional supplements and nutraceuticals for health, physical activity, and performance: moving forward. Appl Physiol Nutr Metab. (2019) 44:455–60. doi: 10.1139/apnm-2018-0781

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Petryna A. When Experiments Travel, When Experiments Travel. Princeton University Press (2009). doi: 10.1515/9781400830824

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Evans M, Salewski RP, Christman MC, Girard S-A, Tompkins TA. Effectiveness of Lactobacillus helveticus and Lactobacillus rhamnosus for the management of antibiotic-associated diarrhoea in healthy adults: a randomised, double-blind, placebo-controlled trial. Br J Nutr. (2016) 116:94–103. doi: 10.1017/S0007114516001665

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Guyatt G, Sackett D, Taylor DW, Ghong J, Roberts R, Pugsley S. Determining optimal therapy—randomized trials in individual patients. N Engl J Med. (1986) 314:889–92. doi: 10.1056/NEJM198604033141406

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Punja S, Schmid CH, Hartling L, Urichuk L, Nikles CJ, Vohra S. To meta-analyze or not to meta-analyze? A combined meta-analysis of N-of-1 trial data with RCT data on amphetamines and methylphenidate for pediatric ADHD. J Clin Epidemiol. (2016) 76:76–81. doi: 10.1016/j.jclinepi.2016.03.021

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Mirza R, Punja S, Vohra S, Guyatt G. The history and development of N-of-1 trials. J R Soc Med. (2017) 110:330–40. doi: 10.1177/0141076817721131

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Guyatt GH, Keller JL, Jaeschke R, Rosenbloom D, Adachi JD, Newhouse MT. The n-of-1 randomized controlled trial: clinical usefulness: our three-year experience. Ann Intern Med. (1990) 112:293–9. doi: 10.7326/0003-4819-112-4-293

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Government of Canada. Clinical Trials for Natural Health Products. (2022). Clinical Trials For Natural Health Products - Canada.ca (2022).

63. Guyatt G, Sackett D, Adachi J, Roberts R, Chong J, Rosenbloom D, et al. A clinician's guide for conducting randomized trials in individual patients. CMAJ. (1988) 139:497–503.

PubMed Abstract | Google Scholar

64. Davidson KW, Silverstein M, Cheung K, Paluch RA, Epstein LH. Experimental designs to optimize treatments for individuals: personalized N-of-1 trials. JAMA Pediatr. (2021) 175:404–9. doi: 10.1001/jamapediatrics.2020.5801

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Kaput J. Developing the pathway to personalized health: the potential of N-of-1 studies for personalizing nutrition. J Nutr. (2021) 151:2863–4. doi: 10.1093/jn/nxab243

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Ma Y, Fu Y, Tian Y, Gou W, Miao Z, Yang M, et al. Individual postprandial glycemic responses to diet in n-of-1 trials: Westlake N-of-1 Trials for Macronutrient Intake (WE-MACNUTR). J Nutr. (2021) 151:3158–67. doi: 10.1093/jn/nxab227

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Potter T, Vieira R, de Roos B. Perspective: application of N-of-1 methods in personalized nutrition research. Adv Nutr. (2021) 12:579–89. doi: 10.1093/advances/nmaa173

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Tian Y, Ma Y, Fu Y, Zheng JS. Application of n-of-1 clinical trials in personalized nutrition research: a trial protocol for Westlake N-of-1 Trials for Macronutrient Intake (WE-MACNUTR). Curr Dev Nutr. (2020) 4:nzaa143. doi: 10.1093/cdn/nzaa143

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Soldevila-Domenech N, Boronat A, Langohr K, de la Torre R. N-of-1 clinical trials in nutritional interventions directed at improving cognitive function. Front Nutr. (2019) 6:110. doi: 10.3389/fnut.2019.00110

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Mattes RD, Rowe SB, Ohlhorst SD, Brown AW, Hoffman DJ, Liska DJ, et al. Valuing the diversity of research methods to advance nutrition science. Adv Nutr. (2022) 13:1324–93. doi: 10.1093/advances/nmac043

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Lindsey J, Jones B. A model for cross-over trials evaluating therapeutic preferences. Stat Med. (1996) 15:443–7. doi: 10.1002/(SICI)1097-0258(19960229)15:4<443::AID-SIM161>3.0.CO

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Centre for Evidence-Based Medicine. OCEBM Levels of Evidence (2022). OCEBM Levels of Evidence — Centre for Evidence-Based Medicine (CEBM). Oxford: University of OxfordA (2022).

73. Antony J, Iyer J, Guthrie N, Evans M. The Supremacy of Aggregated N-of-1 Trials for Evidence-based Nutrition Studies (P06-096-19). Curr Dev Nutr. (2019) 3:nzz031.P06-096-19. doi: 10.1093/cdn/nzz031.P06-096-19

CrossRef Full Text | Google Scholar