The $12M NuSI/Ludwig Study ~ Part V: Intake, REE and TEE Measures

Summary:

Continuing on with discussion of:  Effects of a low carbohydrate diet on energy expenditure during weight loss maintenance: randomized trial

Previous posts in this series:

Part I: Critique of the Study Design

Part II: $12 Million for 12% Weight Loss?

Part III: Some "Early" Lessons

Part IV: Insulin Resistance Does Not Hamper Weight Loss

This post should perhaps have come first, but it has taken a while to look deeply at the data for the primary outcome -- total energy expenditure measured by doubly labeled water -- and related outcomes of intake and resting energy expenditure.

In this study, all participants were paid to participate, AND provided free food for ~8 months.  Said food was professionally and meticulously prepped, weighed, measured, individualized to provide each subject with some pretty exact caloric level and macronutrient composition.  The test phase (in other words, the study proper) involved maintaining a consistent weight (to within ±2 kg), for a considerable length of time:  20 weeks, assessed at the midpoint and the end.

In the previous study, a roughly 300 calorie/day increase in TEE was observed during 4 weeks of isocaloric feeding, producing no weight loss.  This was essentially dismissed as "not enough time to show weight loss"  {paraphrase}   ....  So in the current study, the average change in TEE masks the often wild swings of up to 2000 calories per day between 10 week maintenance time points.  If this increase (or decrease) in expenditure is truly real, the subjects would HAVE to have adjusted intake accordingly over the course of several weeks in order to maintain consistent weight.  

This post is about the massive discrepancies between intake and TEE data (also compared with REE), and the inexcusable near-dismissal of these discrepancies in the journal article.

Bottom Line:

Either the DLW-TEE data is (for reasons that can be discussed) unreliable/inaccurate, or the intake data is unreliable. If the TEE data is problematic, how can a result of a difference of means of 250 cal/day be justifiably defended as significant/meaningful, when we see no such difference in intake? If the intake data is problematic, then how can we conclude anything from a "diet comparison", because significant deviations in caloric intake would render the strict macro compositions totally irrelevant. 

Either way, the researchers failed to address this glaring problem with the results study.  Further, the discrepancies in these two data sets are so egregious that it is impossible that both can be considered accurate and reliable.  It has to be one or the other.  Based on the data for Resting Energy Expenditure (REE), the needle points to the intake data as the more reliable measure.

---

---

My first inclination, once I realized I could access the raw data from the NuSI/Ebbeling/Ludwig study, was to just go there ... because it became clear from the get-go that only selective information was being shared.  Before I even got to that data, I looked at Supplementary material NOT included in the BMJ article proper.  This included various plots showing individual trajectories that were quite informative (and obscured by the means reported).  I highlighted just two such examples for the low carb group in a tweet to Dr. Ludwig.

.Was any attempt made to match DLW-TDEE w/ the actual wgt maintaining caloric intake? Is the intake data available anywhere? How about a plot similar to this one for the dietary intake?  Does it somewhat match DLW? Weekly weight plot maybe?  Some TDEE swings seem too incredible. pic.twitter.com/5w8J5JItkp

— Evelyn (@CarbSane) November 16, 2018

(Dr. Ludwig never responded to my question).

Remembering that this is a carefully orchestrated feeding study where the subjects are not only provided all of their food for free, they are also compensated for participating.  They eat several of their meals in a "supervised" setting, and weigh themselves daily on a wireless scale that transmits the data to FS²-central.

It is explicitly stated that in order to maintain weight to within ±2kg, the amount of food provided would be adjusted.

The graphic in the above tweet simply highlighted two of the low carb subjects with two of the most wild swings in TEE.  

If these TEE readings are true, intake would HAVE to have changed rather dramatically in order to maintain a stable weight.

So, where is the acknowledgment and/or accounting for of such changes in the data?  Hmmmph.  It's not there.   From the study proper, we only get (edit/formatted for clarity):

"Other outcomes

We assessed energy intake during the test phase among participants in the per protocol analysis, providing an estimate of energy requirements during weight loss maintenance.

Although estimates of energy intake are less accurate and precise than total energy expenditure (and our methods would tend to selectively underestimate those with high energy expenditure, as considered in the supplemental methods), the results are generally consistent with the findings for total energy expenditure.

Compared with levels at the start of the trial, energy intake changed in participants assigned to the high, moderate, and low carbohydrate test diets, respectively: 139 kcal/d (−4 to 282), 175 kcal/d (42 to 308), and 269 kcal/d (143 to 396), with an overall P=0.36." 

The first thing I would note is the absence of the word "significantly" (as with TEEs where P=0.002), and failure to acknowledge that a P-value of 0.36 is not even in the neighborhood of statistically significant.   This is for changes in the amount of food they actually fed the participants.  This type of result is usually reported as "the change in energy intake was similar between groups".   The above also does not address the consistency, or as it turns out lack thereof, between intake and TEE.

So off I went to take a look at the raw data for myself ...

... and for the first week or so could hardly wrap my head around what I found.  Not in the context of this "250 calorie metabolic advantage" nonsense I was seeing out of Dr. Ludwig and team and the ridiculous media spin in outlets like the New York Times.  I kept second guessing myself.  I must be reading this wrong.  My data import app must have malfunctioned.  But NO!

My analyses will be for the 105 "successful completer" subjects (40 LC , 35 MC , 30HC) for whom complete insulin and energy measures are available at all time points (BSL, PWL, MID and END)

Here are the individual plots for the absolute (e.g. as measured, not adjusted or normalized, just the most raw of measures) energy data for each subject.    I'm going to display larger images on the first 12 of each set, then go "small" on the rest, though you can scroll and/or click to inspect at will.  They aren't the highest of resolution, but they should be readable when zooming.   Some notes, then the plots.

• Gold background are the 40 LC,  blue background are the 35 MC, green background are the 30 HC subjects.
• 0=baseline (BSL) , 1= post weight loss (PWL) = 12 weeks in, after 0 weeks weight loss and during 2 weeks stabilization , 3 = MID = 10 weeks into maintenance on test diet , END = 20 weeks into maintenance on test diet.
• Only TEE (blue) and REE (pink) were collected at Baseline, these level have been traced through with same-color dotted lines for reference.
• Intake (green) reported for PWL, MID and END only.
• I tried to keep the y-axis consistent but it was not possible, but for the most part, the major horizontal grid lines represent 500 calorie increments.

There are three things I noticed from the numbers, that are really brought out by scanning through the plots -- in other words, don't even bother expanding some of the small plots, just look at the lines!

First:   The plots for REE are relatively flat-lined across the board.   Whatever the effect of diet on this at any point, there appears to be little if any impact on the basal metabolic rate of the subjects.  Even the "outliers" here don't jump around much.  (In the hopes of gleaning some actionable information from this study, I will be addressing this in a separate post looking at the Mifflin-St. Jeor estimates vs. REEs.)

Second:  The plots of TEE are all over the place.  Some jump up, some dive down, some go up and down, some go up up, some down down, and all manner of patterns in between. This is also irrespective of assigned test diet.  We're talking changes in EE of hundreds and even 2000 cals/day just from (presumably) altering macro content of the diet?   How is that even believable?????

Third:  The plots of Intake and TEE don't match up MOST of the time.  The differences are considerable.  This is no small point!  The deviations are obvious and of great magnitude both in absolute calories or when expressed in one as a percent of the other.  This goes UNADDRESSED by the authors in the paper.  Rather, they give the impression (and I quote) that they are "generally consistent with the findings for total energy expenditure".

About that Relationship Between TEE and Intake:

Here are some more "official statements" from the researchers (reformatted for clarity, etc.), this time from the methodology paper:

We stabilized body weight at the end of the run-in phase, prior to the start of the test phase.

The energy level for weight stabilization was estimated based on recent rate of weight loss for each participant (energy intake during weight loss [kcal/day] + (rate of weight loss [kg/day] × 7700 kcal/kg). Although the conversion factor of 7700 kcal/kg is not appropriate for calculations of long-term energy balance, this calculation is useful for estimating energy needs over the short term.

During the test phase, we monitored body weight and adjusted energy intake to achieve weight stability, defined as weight change not exceeding± 2 kg. 

We obtained weight measurements from Wi-Fi scales every day and regressed weight (g) on time (days). A slope ≥15 g per day over 14 days indicated the need to adjust energy intake to achieve stability. Considering deviation from the PWL anchor weight and slope of the regression line, we made adjustments in energy intake when necessary but not more frequently than every 2 weeks.

If a hypothetical subject was stabilized on 2000 kcal/day, and some magical magicness of dietary macrology resulted in a jump of energy expenditure to 2500 kcal/day, said subject is losing ... oh yeah ... let's drag out the calculator!

• 500 kcal/day * 14 = 7000 kcals
• 7000 kcal ÷ 7700 kcal/kg = 0.909 kg
• 0.909 kg times 1000 g per kg = 909 grams
• 909 grams ÷ 14 days = 64.5 or ~ 65 grams per day

Thus, an imbalance in energy and intake of 500 calories per day translates to a roughly 65 g/day change in body weight -- over 4 times the 15 g/day threshold to trigger an adjustment to intake.   Working the other way a gain or loss of 15 grams per day would equate to an imbalance of just 115 calories per day.

The table below was compiled by comparing the TEE and Intake values for all compliant completers at all time points -- weight maintaining! -- at which both were assessed.  There was no baseline intake, so this involves three time points and 105 subjects for a total of 315 points of comparison between TEE and Intake.  These values should be close for a person who is weight stable.

Before any more in depth discussion, I draw your attention to the bottom right corner of the table.  Those are not typos.   Furthermore, both Intake and TEE were determined over a 14 day span, these are not just oddball isolated days! There was a "data point" where the Intake exceeded the TEE by 767 kcal/day, and even more shockingly, one where TEE exceeded Intake by a whopping 3402 kcal/day!!!!

Extremes aside for now, and forgetting about the diet groups for a moment, there is an average difference between all TEE and Intake of roughly 500 calories per day.  Over the 20 weeks of "maintenance", if this discrepancy went unaccounted for we are talking about a 20 lb weight loss that should have been realized!   And since these are readings from all three maintenance-level time points (PWL, MID, END), this is an average persistent deficit.

I used the Mifflin-St.Jeor equation x 1.5 to estimate baseline energy needs (per study protocol), and can then calculate the approximate intake for weight loss and calorie deficit.  These same 105 subjects lost an average ~9.5 kg, which is right around 20 lbs over the course of 10 weeks on an average deficit of ~950 kcal/day.  Thus roughly half the calorie deficit for twice as long ... these subjects should have pretty much doubled up on their weight loss by study's end.   But of course, they did not.

Houston, we have a problem here!  The middle columns of the table only itemized out the number of readings where TEE exceeded Intake and vice versa.  As can be seen, TEE exceeded Intake the vast majority of the time (83%) averaging a greater than 600 calorie/day differential.   When Intake exceeded TEE (17% of the time), the differential averaged only around 200 calories/day.

The percentage of times where TEE and Intake were within 200 calories of each other was a paltry 22%.  That's just over 1 in 5 readings.  If we widen that margin to within 500 calories, just over half, 55%, of readings qualify.  This means that almost 1 in 2 readings involved a differential between Intake and Energy expenditure exceeding 500 kcal/day!  Need I remind the audience here that 500 Calories/day is the "standard" prescription for weight loss of about a pound a week?

Just to drive the final point home, I compiled the plots of the "extreme six" subjects -- those with the min (max negative) and max differentials between TEE and Intake.  In purple across the top, I have included body weight.  I have also noted the time point where the extreme occurred.  (I apologize for the resolution here, not sure why it is so poor).

Oh ... and one more thing ... As you can see there were even reported Intakes that were below either REE measured at a time point and/or baseline REE.  Indeed 5.7% of reported Intakes were less than measured REE, ranging from only ~5 to almost 700 kcal/day, and averaging ~200 kcal/day below.

So I ask ... 

... are the plots of energy expenditure (TEE, blue) in any way, shape or form, "generally consistent" with the plots of intake (green)????   Is the collective TEE data in any way "generally consistent" with the Intake data collected during the same time?  

If your answer is yes, I am going to have to challenge your visual and analytic capabilities.  This is not a matter of scientific interpretation here.  Look again.

Again from Ludwig and colleagues:

"Energy intake data were compiled for the two-week doubly-labeled water assessment periods at START, MID, and END for participants who achieved weight-loss maintenance within ±2 kg of the START anchor weight. We calculated average intake in kcal/d based on provided energy, weights of leftover menu items following supervised meals, and estimated proportions of menu items consumed (as recorded by participants in the portal) following take-out meals and snacks. Due to reliance on self-report for documenting consumption of take-out meals and snacks, our estimates of energy intake lack precision and accuracy compared to TEE.

Moreover, we did not fully capture energy intake from ad libitum snacks provided to participants who continued to lose weight and had difficulty consuming large meals (see above), possibly leading to selective underestimation of energy intake among those with higher TEE.

While recognizing these limitations, we included energy intake as a post hoc outcome to evaluate consistency with TEE data.

Only they did NOT evaluate consistency.

Concluding Thoughts ... 

It is important to keep the Carb-Insulin Hypothesis (or as I call it, TWICHOO) in mind whenever interpreting the results of a scientific experiment.   

The current version of the hypothesis being tested makes the claim that carbs spike insulin and insulin partitions fat to storage, locking it down and essentially starving the rest of the cells, driving hunger and overeating.  When insulin is lowered with carb restriction, all of this fat is now freed from it's adipose prison cells, ready to go to work and be burned like a California wild fire in the rest of your body's cells.   This, as posited in the hypothesis, will lead to increased energy expenditure (TEE) so that a person can eat more while maintaining a lower body weight.

This means that if TEE increases, Intake must increase commensurately or weight loss will occur.  It is critical to the test in this study -- where weight was held relatively stable (my analysis only included successful completers) -- that where TEE increases, Intake must increase commensurately.   

This simply did NOT happen.

While the researchers set about P-hacking in the mean weeds, the data scream out repeatedly to draw attention to the reliability of the TEE measurements by doubly-labeled water.  While they could manage statistical significance to TEE differences by diet group, they were UNABLE to do the same (not even close, P=0.38) for Intake differences.   And that, right there, should be the end of it.

If the reported Intake -- determined largely by the carefully measured and prepared foods provided -- differs from TEE by such a large magnitude, either that TEE measure is in significant error, or compliance with the test diet is so poor as to be unreliable.  I don't think the researchers really want to address that last point, because that is the crux of their $12 MILLION dollar endeavor!  If we cannot have confidence in the caloric intake, then we equally lose confidence in the macronutrient make up of actual intake.   If we cannot have confidence here, then all of these results are moot!  

This idea that your "average 82 kg person" expends some 250-ish more calories per day, and can thus eat that much more, on a 20% carb diet vs. a 60% carb diet is simply an anomaly of what looks to be an (increasingly?) unreliable measure.  

Future Posts will address:

• The discordance between REE and TEE data
• The relative "flat lining" of REE across the board
• The (rather excellent) correlation between measured REE and Mifflin-StJeor estimates
• How even their TEE "metabolic advantage miracle" evaporates when using percent of baseline (whether we use pre- or post-weight loss as anchor) changes as the benchmark instead of absolute or normalized differences.  
• Is it time to revisit DLW-TEE as a reliable measure?