The $12M NuSI/ Ludwig Study ~ Part II: $12 Million for 12% Weight Loss?
UPDATE: 12/17/2018 Original Posting: 12/3/2018
During the writing of a new installment in this series, I revisited the following paper:
A randomized study of dietary composition during weight-loss maintenance: Rationale, study design, intervention, and assessment
Yes, folks, so full of themselves were these researchers, that they felt the need to write an entire paper (submitted approx. one year in advance of the "real study") outlining their amazing study. Regarding this post, focusing on the OUTRIGHT FAILURE TO IMPLEMENT THEIR STUDY DESIGN AS STATED, I stress the following from the previous paper.
ONCE AGAIN, the stated goal of the weight loss run in is 12% ... not 10 or 5 or 15 or whatever ... 12%. I'm just going to quote the relevant part here (cleaned of references, etc. and formatted for readability, all emphasis mine)
Energy intake was restricted to 60% of estimated needs to achieve a target weight loss equating to 12 ± 2% of baseline body weight, with a minimum of 1200 kcal/d. We asked participants to weigh themselves daily using Wi-Fi scales, and the amount of food provided was increased or decreased as necessary to reach the target weight loss over 9 to 10 weeks.
Previous studies have indicated metabolic adaptation characterized by declines in resting energy expenditure (REE) and TEE with weight loss corresponding to 10% of baseline weight. With 20% weight loss, REE may not decline further, but TEE may continue to decline at a slower rate due to shifts in non-REE.
We selected the 12 ± 2% weight loss target to ensure significant metabolic adaptation without imposing greater burden (on participants and the study) of a larger weight loss target.
Only they must not have done this as weight loss in this study was roughly 10.5% ± 5%. FOR SHAME.
/UPDATE
SUMMARY
This post focuses on a critical issue with the Run-In phase of the recently released $12 million dollar NuSI funded study led by David S. Ludwig MD, PhD: Effects of a low carbohydrate diet on energy expenditure during weight loss maintenance: randomized trial. On occasion I reference the 2012 predecessor: Effects of Dietary Composition on Energy Expenditure During Weight-Loss Maintenance.
Virtually every stipulation leading up to the study, the Abstract and Full Text of the journal article, and continuing through to Ludwig's November 28, 2018 response to Kevin Hall in BMJ, has stated that the MAIN purpose of the Run-In phase was to achieve 12% ± 2% weight loss. The study was intended to test the so-called Carb-Insulin Hypothesis during 20 weeks of weight-stable maintenance on diets of varying carbohydrate content.
OUTCOME:
This indicates an unacceptable variation in the post-weight loss (PWL) "biological state", and to use this as the baseline "anchor" for diet comparisons is negated by either Study Design and/or execution thereof.
OUTCOME:
With only 10 weeks of 60% calorie restriction during Run-In, the subjects who completed this phase averaged 10.5% weight loss (std.dev. 1.7%), ranging from only 5.6% up to 16.7% weight loss. This doesn't change much for analyses of just those completers for whom complete energy data are available (same mean, SD = 1.5%, range 5.9% to 16.0%).
This indicates an unacceptable variation in the post-weight loss (PWL) "biological state", and to use this as the baseline "anchor" for diet comparisons is negated by either Study Design and/or execution thereof.
This is a follow up on Part I where I discussed issues with the study design in:
Effects of a low carbohydrate diet on energy expenditure during weight loss maintenance: randomized trial
... the big $12 million NuSI sponsored study.
As I read the press fawning over this, and the study, something kept niggling at me ... TWELVE PERCENT. That number ... 12 ... that as a matter of serendipity is how many millions this study roughly cost.
From head low carb cheerleader Anahad O'Connor over at the New York Times we got the headline: How a low carb diet might help you maintain a healthy weight.
From head low carb cheerleader Anahad O'Connor over at the New York Times we got the headline: How a low carb diet might help you maintain a healthy weight.
Each participant went through two phases of the study. First, they were put on strict diets that lowered their body weight by about 12 percent, which was designed to stress their metabolisms.
"At that point their bodies are trying to regain the weight,” Dr. Ludwig said. “It pushes the body and predisposes to weight regain.”
And in the LA Times, lead investigator and media obesity darling, Dr. David S. Ludwig himself weighed in with: The case against carbohydrates gets stronger.
For our clinical trial —[humblebrag] one of the largest feeding studies ever conducted [/humblebrag]— we collaborated with Framingham State University and the company that manages its food service. [ whut whut? No mention of NuSI or your pal Gary Taubes? For shame! ] We recruited 164 students, faculty, staff and community members who agreed to eat only what the study dictated for a full academic year.We started the participants on a calorie-restricted diet until they lost 10%-14% of their body weight. After that, we randomly assigned them to eat exclusively one of three diets, containing either 20%, 40% or 60% carbohydrates.
I could perhaps forgive O'Connor, he's just regurgitating talking points after all. But Ludwig, as lead investigator, and speaking on behalf of the whole research team, gets no such pass. Here's the write-up from the registration of the trial at clinicaltrials.gov
In case you can't read the image, the important highlighted region reads:
"Following 12±2% weight loss on a standard run-in diet"
There's no wiggle room here. We're not talking about a 2% standard deviation, we are talking about a range of weight loss, in percentage of starting weight, from 10 to 14%. If there is ANY question regarding this "±2", Ludwig makes it clear by only citing the range in the LAT.
This was the original intent of the study, as described on the NuSI website (2/9/2014 archive)
Although altered slightly on the latest incarnation.
The original was 12% in 12 weeks. They backed off to 10-to-12 percent, which was actually the target in the predecessor study (minimum of 10%, ended up averaging 13.5%)
Even the literature used to recruit subjects claims that they can expect to lose this much!
 |
| From: Informational Flyer |
THE ACTUAL RESULTS OF THE RUN-IN
From the raw data, for all subjects who completed the Run-In phase (not just the 109 for which there is complete data for all energy assessments) the weight loss percentages were as follows:
Mean: 10.5Minimum: 5.6Maximum: 16.7Standard Deviation: 1.7
These differ in the maximum of 16.0 for compliant maintainers with complete data, but mean is still 10.5 with a standard deviation of 1.5.
If we're being generous, this amounts to a 10.5 ± 5% weight loss.
Despite the following:
"The research team monitored participants’ body weight and adjusted the amounts of food when necessary to achieve the target weight loss."
... we have this, from the text of the journal article itself (they appear to report the totals for all who completed run-in):
Of these, 164 achieved 12% (within 2%) weight loss and were randomly assigned to one of three macronutrient diets for the test phase, comprising high (n=54), moderate (n=53), or low (n=57) levels of carbohydrate. Table 2 presents the characteristics of the randomized sample at the pre-weight loss time point.
Umm, NO. No, they did not Dr. Ludwig. They actually achieved an average weight loss of around 10.5 percent, ranging from a little as 5.6% to as much as 16.7%. I would note that 10.5 is only a half percent from the lower limit of the target they keep repeating to this day ... REPEATEDLY! (I'll let it slide that it is impossible for Table 2 to be pre-weight loss while reporting weight loss percentage ... but who really looks at the titles on tables and those sorts of inconsequential details anyway.
In the VERY NEXT PARAGRAPH, we get the real numbers ... I mean after just claiming the 164 who were randomized achieved a weight loss of 10 to 14% ... then this:
During the run-in phase, mean weight loss for randomly assigned participants was 9.6 kg (SD 2.5kg), corresponding to 10.5% (SD 1.7%) of pre-weight loss body weight.
This sort of inconsistency doesn't even require peer review, it's basic editorial review, but surely peers should be on the look out for such glaring discrepancies!!!
Now I'm sure many are shaking their heads and thinking "why is she obsessing over this" ... something on the order of "12% or 10.5% ... is that really worth going on about?"
Well, Kevin Hall PhD has done a re-analysis of the data, that has now been published in the Rapid Responses at the BMJ: No Significant Effect of Dietary Carbohydrate versus Fat on the Reduction in Total Energy Expenditure During Maintenance of Lost Weight. This analysis is done using the baseline (pre-weight loss) TEEs as the anchor, as designated in the original filings at clinicaltrials.gov. The investigators made a late revision in the protocol to use the post-weight loss TEE measurements as anchor instead. When you do the originally intended analysis, the supposed increase in TEE on the low carb diet vs. the low fat (high carb) diet, diminishes so as to become statistically insignificant. In a future post, I will elaborate on a different issue with the post-weight loss TEE data -- that it was obtained during the same period as weight stabilization from -- and some issues with the TEE data itself.
Ludwig has also formally responded: Author Response to Hall and Guo Regarding Data Reanalysis and Other Criticisms
In it, Ludwig refers to some of the Peer Review criticism/points and resolution. This is dated September 18, 2018!!! I'll copy the text of the crucial paragraph here in full, with some commentary (some of which may be repetitive of Hall).
1. The initial listing was clearly an error:
A) As a general rule, anchor data should be collected as close to randomization as possible, to decrease error introduced by time-varying covariates. The pre-Run-In measurement involves a 3- to 4-month delay prior to initiation of the Test diets.
One wonders why the baseline data was a perfectly acceptable anchor in the 2012 study! (Indeed they didn't even gather post-weight loss DLW-TEE data in that study.
B) In addition to this delay, the pre-Run-In measurement is strongly confounded by weight loss, whereas the specific aim of the study is to examine TEE during weight maintenance. (Indeed, the title of the study in the registry is: Dietary Composition and Energy Expenditure During Weight-Loss Maintenance.)
Here they are essentially arguing the point regarding the anchor -- that it be a most comparable pre-test (maintenance phase) anchor. I tend to agree, IF the assessment had been done following an appropriate period of weight stability with maintenance intake.
C) The expressed purpose of the Run-In was to produce 12% weight loss, changing biological state (i.e., creating a predisposition to weight regain) to test the study hypotheses. Thus, it would be inconsistent with study aims and methodologically inappropriate to use the pre-Run-In time point to establish a precise and accurate anchor for determining how the Test diets change TEE. Doing so would necessitate a substantially larger number of participants (and cost) to account for the additional imprecision, with no scientific benefit.
This red-emphasized sentence is the crux of the matter. The whole purpose of the Run-In was to create a sample representative of the population that has recently lost a significant amount of weight - the target of which was CLEARLY 12%. In this statement, they are acknowledging that this biological state is a crucial baseline factor prior to maintaining the weight loss on the varied test diets.
The investigators FAILED to ensure this most important factor of all. As such, analyzing the data as they have done -- using a post-weight loss anchor -- is introducing yet more confounding variables. Variables a well-designed run-in phase is designed to avoid. Specifically:
- Baseline energy requirements were estimated using a standard regression equation and fixed activity multiplier, rather than. Restriction to 60% of this estimate produced a variety of relative caloric deficits. This is evidenced by the wide range of average weekly rates of weight loss: 0.56%/week ↔️ 1.67%/week (0.56 ↔️ 1.60 for successful completers with full EE data).
- By failing to adjust, or adjust sufficiently, during the weight loss phase, the "post weight loss biological state" of the subjects varied widely, from those losing as little as 5.6% of initial body weight to as much as 16.7% (5.6% to 16.0% for successful completers with full EE data).
Comments
Post a Comment
Comment Moderation is ON ... I will NOT be routinely reviewing or publishing comments at this time..