NuSI Again: A Bar Too High, Occam's Razor and Rabbit Holes

I weighed in with some initial thoughts on Gary Taubes and Peter Attia's new venture NuSI here.  Yeah, I'm going all wet blanket on this deal, but for good reasons grounded in reality in my opinion.  I'll leave it to all the others to get all excited about this venture and all the "real science" we'll see come from it.   As Gary says of NuSI:

Its purpose is to facilitate and fund rigorous, well-controlled experiments targeted at resolving unambiguously many of the outstanding nutrition controversies — to answer the question definitively of what constitutes a healthy diet.

A bar too high?

What would constitute a rigorous, well-controlled experiment?  Well, as I pointed out in my last post, the biggest "knocks" the NuSI gang make on existing research are not really with experimental design per se, but rather with practical limitations in doing human clinical trials.  These are:

1. The inherent lack of compliance control in free-living studies.
2. The monetary and logistics obstacles inherent in metabolic ward studies resulting in studies that  of necessity involve smaller sample sizes and/or shorter durations than might be desired.

So ... are we to presume that NuSI will not be funding any free-living studies?  One would presume so, since they cannot claim existing science is inferior/inadequate because of lack of compliance control and then go out and fund similar studies.  The bar is set even higher on point 2, because the metabolic ward study suffers from a problem other than cost -- finding subjects willing and able to spend lengthy periods of time as subjects in such a setting.   However even presuming that obstacle can be overcome, we're back pretty much to a cost issue that is the basis for why most metabolic ward studies are small and short in duration. NuSI has set the bar VERY high in terms of expectations.  The "real science", as Fat Head calls it, had better be large, long metabolic ward studies ... or else NuSI itself would dismiss it as inadequate to answer the burning questions.

Let's look at NuSI's other problems with existing science.  Points 3&4 are really the same side of the coin that Taubes discussed in Calories, fat or carbohydrates? Why diets work (when they do).

3.  Caloric restriction of the control diet, and so effectively carbohydrate-restriction of all the competing dietary interventions;

4.  Caloric restriction of the carbohydrate-restricted diet, thus building into the experiment the assumption to be tested – that the carbohydrate content of the diet can influence weight loss independent of total caloric content.

That post focused on the Shai et.al. study basically making the point that Shai and her colleagues were just too mired in old paradigms about calories to properly design their study.   This study is among the "bad" free-living studies.  I say that studies like Shai (and others like Gardner, Foster, Dansinger, etc.) measure up differently on the "science" scale for various reasons.  Mostly these studies fall short because there is not sufficient monitoring and reporting compliance and those shortcomings are compounded catastrophically by intent-to-treat analyses and variations thereof.  There was one study a few years ago where they actually reported and compared predicted weight losses for subjects who dropped out.  Anyone recall the study I'm referring to?  Seriously, if we're talking about trying to reverse obesity in the most healthy and sustainable fashion, the major drawback of all of these studies is that we have no clue regarding compliance, and those that don't comply and even dropouts are often included in the numbers.  This tells us nothing, and I think the intent-to-treat analysis is what is wrong in many cases, not the "science".  

But back to Shai for a moment, because people should be reminded of what a big deal Gary Taubes makes over this "bad science" to try to support his alternate hypothesis!  It's why I "called him a sheister", only I didn't really, it was a play on words with Shai's name in:  Gary Taube$, Shai-ster.  Taubes repeated the lecture analysis in one of his book press releases (aka blog posts), Taubes: Calories, fat or carbohydrates? Why diets work (when they do).  It really does make one wonder how NuSI intends to identify well designed studies to fund when one of its co-founders flat out says that dietary intake reports such as those in Shai should be taken with a grain of salt, and then launches into a serious analysis of the data nonetheless.  I wrote my own follow-up post about this debacle,  CarbSane:  Calories, fat or carbohydrates? Why diets work (when they do).

Occam's Razor  

There have been a number of studies where macronutrient ratios have been altered rather considerably in isocaloric contexts.  One such study was a small metabolic ward study in obese women, Grey & Kipnis.  NuSI includes this study in their review of the literature but complains that the low carb diet of the weight maintenance phase was too high at 25% carb.  I'm going to get back to that in a moment, but I addressed G&K in my Why diets work post just linked to.  

Protocol 1 - Effect of Carbohydrate Content in Caloric Balance:  Subjects ate ad libitum for 2 weeks to establish weight maintenance calorie needs, then fed at that caloric level for 3 weeks on a LC diet (C/F/P = 25/53/22), and switched to a HC diet (C/F/P = 62/18/20) for the final 3 weeks. 

Now G&K suffers from the problems of metabolic ward studies.  It was small (7 subjects in this leg) and each diet was only for 3 weeks, but the difference in carbs was substantial!  Yet 5 of the subjects weights stayed rock steady level throughout, while one gained a little, one lost a little.  The rollercoaster curves are for fasting insulin which, as you can see, is all over the place.  Taubes & Attia look at this study and it's that carb consumption is not restricted enough to see an effect in three weeks.  But they apparently ignored the second leg of the study.  Again, admittedly very small (only 3 subjects) but here's their protocol:

Protocol 2 - Effect of Carbohydrate Content in 1500 cal CRD:  Here the subjects again established baseline for 2 weeks and were then cycled between 1500 cal formula diets:  4 weeks HC (C/F/P = 72/1/27), 4 weeks LC (C/F/P = 0/74/26) and finishing up with 4 more weeks HC.

They all lost weight, pretty steadily, when calories were cut.  And they kept protein roughly constant and switched them between a no-fat to no-carb back to no-fat diet for 4 weeks each.  

It's obvious to everyone but the most blinded TWICHOOB that it is calories and not carbohydrates (or insulin) that determine body weight.

The Occam's Razor in all of this is that based on every means and measure, on every graph, one thing is clear:  Americans consume more calories now than we did before the obesity epidemic.  The caloric amount varies -- I've seen ~300 cal/day quite frequently, and as high as 600 cal/day.   You can't get away from this fact, and yet they will contort the macro intakes using percentages and spin things to blame carbohydrates.  But there's no denying that we are overconsuming, and the other edge of this razor is this:  Our carbohydrate consumption has not changed that considerably.  Yoni Freedhoff at Weighty Matters summarized my point better:

What she's saying is that from a macronutrient percentage perspective, the difference between the 1970s consumption of a diet containing 45% carbs (for women) and the 2000s diet of 52% (and for men the difference between 42% and 49%) is pretty insignificant and that 1970 diets were anything but low-carb and yet our weights were so much better.

Because part of G&K ran afoul of NuSI's point 5 against existing science:

5.  Testing diets of only marginally different nutrient content — “low carbohydrate” diets, for instance, that are still 30% or more carbohydrate.

They lower that bar to 25% to dismiss G&K, but the weight maintaining diets in that study varied carb content between 25% and 62% -- that is pretty darned significant, especially when folks are trying to blame a 7% increase in carb proportion of the American diet for the obesity epidemic.  Let me repeat that in a slightly different fashion:

Conventional Wisdom:  The obesity epidemic was caused by increased consumption of food (calories) 

TWICHOOBS:  The obesity epidemic was caused by an increase in the proportion of carbohydrate calories of 7%.  

Put this way, TWICHOO seems ever the more ridiculous.  Even if we turn our eye on the types of carbohydrates, it's a tough point to sell because flour and sugar have been around far far longer than this obesity problem.  

Rabbit Holes

A relative new-comer on the LC/paleo scene is Abel James.   One of his favorite sayings when trashing folks who look at the science, is to belittle them for chasing down rabbit holes.  Part of that goes to the very core of doing the studies or analyzing the results of studies that have little or no practical application or relevance.  And yet, as I've just discussed vis a vis their point 5, NuSI seems to want to do just that.  Even if, in a metabolic ward context, NuSI funded scientists operating with optimal intelligence could demonstrate that humans get fatter on a 50% carb diet vs. a 5% carb diet ... would this answer the burning question of what caused the obesity epidemic?  No.  Because there was no such existing carbohydrate restriction, nor was there any such massive increase.  Thus it is a rabbit hole like any other.  Indeed it would appear that NuSI's last two problems with current nutrition science relate to scientists not going down those rabbit holes.  iction to determine the cause of this obesity epidemic.  Nevermind that we NEVER ate anything close to low carb before this whole mess.  NuSI bemoans testing diets that are "only marginally different nutrient content" when we have this obesity epidemic resulting from changes in the nutrient content of the American diet that are so nominally different, they're not even worth studying!  Better to go down the "good science" rabbit hole testing extreme low carb diets that weren't consumed before this epidemic.

It would appear that when judging the merit of nutritional science, Taubes and others of like mind, dismiss any research not designed to answer their questions.  Like there's no other question to be asked or answered, and it's those questions that are viewed as rabbit holes.  Their last points of contention with the bad nutritional science out there:

6.  Manipulation of fat and protein content in the diet, rather than the carbohydrate content itself.

7.  The use of habitually lean subjects to test the effect of nutrient composition on the accumulation of excess fat.

Point 6 seems aimed at that Bray overfeeding study where they kept carbs constant as a percent.  I discussed that study in  The First Law of Thermo still doesn't violate The Second Law.  That study did address the relevant question of whether protein can be "fattening".   The verdict?  Not really, the caloric surplus led to relatively equivalent increases in fat mass irrespective of protein content.  But this study may illustrate a role for protein in the obesity epidemic, as assessed by BMI.  The low protein group ate rather low protein (6%) while the moderate protein group was average American consumption (15% though in many studies normal content tends to run a few percentage points more, though not the 26% of the high protein group).  There can be no argument that Americans are overeating, but we're overeating with a protein level that promotes weight gain -- *weight* that is not distinguished between fat and lean mass.  In the Bray study, the fat mass gains were roughly equivalent while the lean mass changes were quite telling.  The low protein group actually lost about 1.5 lbs of lean mass while the medium and high protein groups put on roughly 6.3 and 7 lbs of lean mass respectively.  It's fair to say that "sufficient" protein favors weight gain in an overfeeding context, but higher protein does not complicate this much.  It certainly doesn't protect against fat gains or weight gain.  I have some further thoughts on this to share another day.

As to point 7, this is true to a point, but for there to be an obesity epidemic in the first place, a goodly portion of habitually lean humans in the 70's somehow became susceptible to obesogenic foods.  A goodly number of habitually lean humans began overeating and getting fatter and fatter as a result.  The foods were there ... McD's ... cakes ... white bread ... TV dinners ... Kraft Mac & Cheese ... Coke ... breakfast cereals (oh and the kids ones had more sugar and were touted to having it!) ...  This epidemic simply cannot be blamed on single digit differences in carbohydrate consumption well, WELL, above the level it would even be relevant.  Rabbit hole.

---

I can hear it already.   Tearing down.   No value.  Unsupportive of big ideas.   Well, next up ...

Just Do It