Calling for the End of Diet Comparison RCTs

just cuz it's cute

You might have heard ...

There's a new "paleo diet" comparison study out this month:  Long-term effects of a Palaeolithic-type diet in obese postmenopausal women: a 2-year randomized trial.  In the interests of completeness, I'll probably throw up a short post on this in the next week or so.  Ho hum ... nothing much to write home about.

Which is nothing new.  

The baseline diet of these women consisted of roughly 2000 calories/day, 17% protein, 45% carb and 34% fat  (presumably the rest alcohol).  The paleo target was 30% each protein and carb, and 40% fat, while the Nordic Nutrition Recommendations diet aims were 15% protein, 55-60% carb and 25-30% fat.    Care to guess what the diets really were?   For the paleo diet:  at 6 months protein was about 23.5%, only about half way to the 30% target, they did make the 30% carb, but overshot the fat to 43.5%, at 24 months they were at 22/33.5/40.5  for P/C/F.   And the NNR diet:  6 months P/C/F = 18/44.5/32 (more alcohol?) and by 2 years they were at 17.5/43/35.    They were supposed to increase carbs and instead cut them both in absolute and percentage bases.

I'm reminded of another study, that by no means stands out as an anomaly, where the participants failed to meet targets.    

In the current study, the NNR dieters were supposed to increase carbs by 10-15% but they actually cut their carb intake, both absolute and by percentage and failed to even make the upper limit of the fat recommendations.  Meanwhile the paleo dieters failed to meet protein intake by quite a bit.  Given that one of the controversies over the paleo diet is its high protein content, what is the purpose of going for 2 years if they aren't even flirting with this goal??  

So this is problem number one:  Adherence to the plan.  You are comparing two diets that are not only not what was prescribed, but also, in the end, not all that different from one another or the baseline diet composition for that matter.

They cut intake some, they lost some weight:  6.5 and 2.6 kg for PD and NNR respectively at 6 months ... but averaged only 4.6 and 2.9 kg respectively at the 2 year mark.  I would note that means the PD group regained almost 2 kg while the NNR group lost a tad more with time.  I'm willing to bet, that as with Shai and Gardner, and Dansinger, and Foster, and all the rest ... when one looks at the individual data we'll find that success tracked highly to adherence, no matter the diet.

There was little to write home about in terms of different health outcomes.   As there almost never is.  The answers to this dilemma will not come from RCTs unless, I suppose, you can find 100 people willing to live in a metabolic ward type setting for a couple of years.  Even so, one has to realize that even a few years is unlikely to expose long term risk, just as all the clinical trials on drugs in the world cannot elucidate long term issues some may develop.   And here is where the rub is.  

Even IF you get verifiable compliance, what useful information there is in the individual data (good and bad) is almost always obscured by the means.  But that same individual data is also not all that meaningful to apply to populations.   Here is where I think all of the epidemiological studies on traditional cultures trump whatever information we could ever hope to get from a bunch of isolated clinical trials.  

As I write this, I can't help but have NuSI come to mind, and the studies they have in the works.  One they are going to start later this year is a continuation of the Ebbeling et.al. study blogged on here, and was addressed a few other times, most linked to here.   As a quick summary, in that study they put folks on a slightly higher protein (25%) calorie restricted diet to lose weight, and they were then stabilized at that weight by increasing calories and monitoring.  Once weight maintenance calories were established they went through 3 different macro diets for 4 weeks each.  The result was they maintained weight on each diet but for some unknown reason their TDEE and REE on the low carb leg were higher.   So NuSI aims to repeat that study except with more subjects so that for weight maintenance, one third will receive each diet for 20 weeks.

Now tell me, what will we learn?    Why don't they do this study using the different diets during the weight loss phase as well?   This would at least be more like the "real world".   Or why not dispense with the weight loss phase entirely?    They will fix one fatal flaw in Ebbeling et.al., which is to control for protein across the diets (20%).  So the makeup of the diets will be P/F/C  of  LF 20/20/60, LGI 20/40/40 and LC 20/65/15.  But if we have learned anything from all the diet comparison studies to date, it is long term adherence that matters so what would the results of 20 weeks on some different diet than that used to lose weight do for us from a practical standpoint?  We're still in honeymoon territory!!

NuSI has a short term ketogenic diet study in the works.  This one would be relevant how?  Even IF a keto diet could be shown to have some sort of metabolic advantage in the short term, what does this tell us?   Nobody ever eats that way except for a few people that insist it is some metabolically advantaged state and the very narrow scope in which the diet can be therapeutic.  Wouldn't it be better to use the resources to monitor THOSE people for a long time since they have a reason to be on such a restrictive extreme diet in the first place?  There's no way the planet is going to switch to such a diet, and there's no indication any cultures ever truly ate like this either.   

Lastly NuSI has a long term, large, free-living study planned.  This one will pit an extremely low fat (< 10%) diet vs. an extremely low carb (< 10%) diet, with both diets emphasizing real whole foods.  Hmmm ... didn't someone just call that Ma Pi some cultish diet?   But I digress.  They are going to do this for a year.  Interestingly the principal investigator is Gardner.  Hmmm ... that should ring a bell!   He was the principal on the Grandaddy of them all!  The A to Z weight loss study!   The Atkins group lost more weight vs. Zone but not statistically more vs. any other diet.  In typical trajectory fashion.   But I guess this needs repeating?  Because Shai didn't test extreme enough diets I suppose, and the 6 year followup was so promising for even their version of low carb.  

You see, what I get from studies like Ebbeling, is that the more you do to ensure compliance, including providing most all foods prepared for people, the better the result.   You are always going to have your zealots who manage to latch onto some extreme diet and stick with it.  Unfortunately for many of these, this seems to be to the detriment of their health despite claims otherwise.  All of which just makes for more confusion for everyone else.  

All of these paleo types would do better to discuss what individual cultures ACTUALLY ATE rather than their fad diet marketing and profiteering.  These clinical trials?  Useless.  If you want to look at removing some purportedly harmful dietary agents, then that's what the studies need to do, in controlled fashion from both an experimental design point of view and an implementation/compliance point of view.   This still will not provide all of the necessary answers, but at least it will provide reliable data for the rest of us to discern which dietary factors we might consider changing on the backdrop of how humans have eaten through the centuries and how certain of us respond, at least for finite times, to certain foods in certain disease states, etc.

The diet comparison studies for general health and weight loss are on the whole a meaningless waste of time, effort, and most importantly money that could be better spent healing people that need healing and supporting people that need support, etc.    Imagine if you will, the money being spent providing real food meals for parents to provide for their children in impoverished neighborhoods.  With carbs and fats and proteins.   Just imagine.

UPDATE about 1 hr after posting original:

It's really soon to post as an edit, but this is an afterthought that should have made the post.  If I were looking to RCTs to pick a dietary path, my interest would lie in what worked best for those who stuck with something, as well as general adherence rates.  Still, to me the latter are not all that meaningful.  I know, based on my life and the nature of the man I share it with, that certain diets would never fly for the long haul.  He's a meat eater, and a carb eater, and he would not last long on a diet that restricted one or the other severely or eliminated them entirely.  While I don't share his *need* in the same way, I couldn't imagine life as a vegan, nor does the high fat greazy keto diet hold much appeal to me.  I am more of a light meat with some starch on the side or fruit and dairy kinda gal.

One of the problems I discussed in this post, was the problems with the often-used "intent to treat" analysis.  When they carry forth last known data points of subject that drop out (formally or just don't really fully comply) it renders the results even more meaningless.  When I was more solidly low carb, I wanted to know if this was healthy over the long term.  I wanted to know what following such a diet did to others -- those who followed it, not those who gave up.  Perhaps some info on hormones and such I didn't have the financial wherewithall to have tested or lacked any baseline data for anyway for comparison.    This data is pretty much nowhere to be found largely because, as discussed from the get-go, most of the diets in the RCT's are not even what is prescribed.

So I wanted to add that to me, non-randomized uncontrolled studies that were well documented and monitored, would prove far more valuable than these RCT's.  Imagine if Jimmy Moore or Dave Asprey or Peter Attia or any of these other n=1's had actually registered to be monitored and verifiably tested along the way.  I don't care that these would be subjects # 1 thru 3 in some study.  If someone could be verified to lose fat, gain muscle, improve lipids, have healthy organ function, etc., etc., on a verified nutritional, exercise and supplement/pharma (or lack thereof)  regime -- to eat 3000 cal/day and really lose 100 lbs in 5 days doing that (I jest) -- this would be useful information to those seeking possible routes to try without feeling like they are risking their health based on claims made by people whose livelihoods depend on your continued interest.