Nutrition science under fire: Diet trials lack of ‘rigour’ slammed by researchers

Writing in JAMA Network Open​, researchers compare discrepancies in the registries of diet trials and drug trials – finding that a massive 86% of diet trials amended their desired study outcomes along the way.

The team, led by first author Dr David Ludwig from Boston Children's Hospital, argue that amending outcome measures in this way leaves nutrition research open to the possibility of bias, while also warning that even diet trials published in high quality journals ‘lack rigour’ and are often too small, or short duration and are unable to rigorously control diet adherence.

"High-quality trials are hard to do because diets, and the behavior of humans who consume them, are so complicated," the team write – noting that because few big companies stand to profit, "typical diet trials must get by on shoestring budgets, rarely exceeding a few hundred thousand dollars, compared with drug trials that may cost several hundred million dollars."

The team call for more investment in rigorous diet research – arguing that nutrition research is not only under-funded, but also more challenging to conduct than drug studies.

Trial comparison​

Ludwig and colleagues compared trial information by conducting a literature search of trials for obesity published in top journals including The New England Journal of Medicine​, JAMA​, The BMJ​, The Lancet​, Annals of Internal Medicine​, and The American Journal of Clinical Nutrition​.

All studies were registered in advance on ClinicalTrials.gov​, a measure intended to keep investigators accountable to what they originally planned to test, ensuring that they don't report findings selectively.

The team identified 343 diet studies, and, for comparison, 148 drug trials for obesity however after strict exclusion criteria were applied just 9 drug trials and 21 diet trials were included in the final analysis.

For diet trials, exclusion criteria included if the intervention did not focus on a specific diet (for example the use of a dietary supplement, food extract, or specific dietary pattern)

“Additionally, trials in both categories were excluded if the article was not in the indicated journals (eg, BMJ Open) or was not an original randomized clinical trial; the primary registry was not in ClinicalTrials.gov; the primary outcome was not related to body weight, adiposity, or energy balance; or the primary outcome was measured in less than 28 days,”​ explained the authors.

Prone to bias?​

When the teams compared the original ClinicalTrials.gov registry descriptions with the final published papers they found that 18 diet trials (86%) showed ‘substantive discrepancies’, while only two of the drug trials (22%) had similar levels of discrepancy.

According to the authors, these changes typically involved a change in the time frame of the primary outcome or the number of co-primary outcomes.

One diet trial, for example, initially listed ‘weight at five years’ as the primary outcome, but later amended this to ‘change in body fat at one year.’

Other trials initially planned several primary outcomes or measurements at several time points, but later pared these down to a single outcome or single time point in the published study.

The team noted that problems with diet trial registries may arise due to their greater heterogeneity and lower budgets when compared to drug studies “and the inadequacy of infrastructural support for nutrition research.”

A 'Manhattan project' for diet trials?​

Ludwig and colleagues also suggest several immediate remedies, such as creating specialised registries for diet trials to reflect their special challenges, but also called for ‘a sort of Manhattan Project’ to pin down the effects of diet changes once and for all.

They note that while this would require a substantial investment, "the amounts involved would total a fraction of a cent for every dollar spent treating diet-related conditions like obesity, Type 2 diabetes, and cardiovascular disease."​

Source: JAMA Network Open​
Published online, Open access, doi: 10.1001/jamanetworkopen.2019.15360​
“Discrepancies in the Registries of Diet vs Drug Trials”​
Authors: David S. Ludwig, et al​