Calling for the End of Diet Comparison RCTs

just cuz it's cute
You might have heard ...

There's a new "paleo diet" comparison study out this month:  Long-term effects of a Palaeolithic-type diet in obese postmenopausal women: a 2-year randomized trial.  In the interests of completeness, I'll probably throw up a short post on this in the next week or so.  Ho hum ... nothing much to write home about.

Which is nothing new.  

The baseline diet of these women consisted of roughly 2000 calories/day, 17% protein, 45% carb and 34% fat  (presumably the rest alcohol).  The paleo target was 30% each protein and carb, and 40% fat, while the Nordic Nutrition Recommendations diet aims were 15% protein, 55-60% carb and 25-30% fat.    Care to guess what the diets really were?   For the paleo diet:  at 6 months protein was about 23.5%, only about half way to the 30% target, they did make the 30% carb, but overshot the fat to 43.5%, at 24 months they were at 22/33.5/40.5  for P/C/F.   And the NNR diet:  6 months P/C/F = 18/44.5/32 (more alcohol?) and by 2 years they were at 17.5/43/35.    They were supposed to increase carbs and instead cut them both in absolute and percentage bases.

I'm reminded of another study, that by no means stands out as an anomaly, where the participants failed to meet targets.    

In the current study, the NNR dieters were supposed to increase carbs by 10-15% but they actually cut their carb intake, both absolute and by percentage and failed to even make the upper limit of the fat recommendations.  Meanwhile the paleo dieters failed to meet protein intake by quite a bit.  Given that one of the controversies over the paleo diet is its high protein content, what is the purpose of going for 2 years if they aren't even flirting with this goal??  

So this is problem number one:  Adherence to the plan.  You are comparing two diets that are not only not what was prescribed, but also, in the end, not all that different from one another or the baseline diet composition for that matter.

They cut intake some, they lost some weight:  6.5 and 2.6 kg for PD and NNR respectively at 6 months ... but averaged only 4.6 and 2.9 kg respectively at the 2 year mark.  I would note that means the PD group regained almost 2 kg while the NNR group lost a tad more with time.  I'm willing to bet, that as with Shai and Gardner, and Dansinger, and Foster, and all the rest ... when one looks at the individual data we'll find that success tracked highly to adherence, no matter the diet.

There was little to write home about in terms of different health outcomes.   As there almost never is.  The answers to this dilemma will not come from RCTs unless, I suppose, you can find 100 people willing to live in a metabolic ward type setting for a couple of years.  Even so, one has to realize that even a few years is unlikely to expose long term risk, just as all the clinical trials on drugs in the world cannot elucidate long term issues some may develop.   And here is where the rub is.  

Even IF you get verifiable compliance, what useful information there is in the individual data (good and bad) is almost always obscured by the means.  But that same individual data is also not all that meaningful to apply to populations.   Here is where I think all of the epidemiological studies on traditional cultures trump whatever information we could ever hope to get from a bunch of isolated clinical trials.  

As I write this, I can't help but have NuSI come to mind, and the studies they have in the works.  One they are going to start later this year is a continuation of the Ebbeling study blogged on here, and was addressed a few other times, most linked to here.   As a quick summary, in that study they put folks on a slightly higher protein (25%) calorie restricted diet to lose weight, and they were then stabilized at that weight by increasing calories and monitoring.  Once weight maintenance calories were established they went through 3 different macro diets for 4 weeks each.  The result was they maintained weight on each diet but for some unknown reason their TDEE and REE on the low carb leg were higher.   So NuSI aims to repeat that study except with more subjects so that for weight maintenance, one third will receive each diet for 20 weeks.

Now tell me, what will we learn?    Why don't they do this study using the different diets during the weight loss phase as well?   This would at least be more like the "real world".   Or why not dispense with the weight loss phase entirely?    They will fix one fatal flaw in Ebbeling, which is to control for protein across the diets (20%).  So the makeup of the diets will be P/F/C  of  LF 20/20/60, LGI 20/40/40 and LC 20/65/15.  But if we have learned anything from all the diet comparison studies to date, it is long term adherence that matters so what would the results of 20 weeks on some different diet than that used to lose weight do for us from a practical standpoint?  We're still in honeymoon territory!!

NuSI has a short term ketogenic diet study in the works.  This one would be relevant how?  Even IF a keto diet could be shown to have some sort of metabolic advantage in the short term, what does this tell us?   Nobody ever eats that way except for a few people that insist it is some metabolically advantaged state and the very narrow scope in which the diet can be therapeutic.  Wouldn't it be better to use the resources to monitor THOSE people for a long time since they have a reason to be on such a restrictive extreme diet in the first place?  There's no way the planet is going to switch to such a diet, and there's no indication any cultures ever truly ate like this either.   

Lastly NuSI has a long term, large, free-living study planned.  This one will pit an extremely low fat (< 10%) diet vs. an extremely low carb (< 10%) diet, with both diets emphasizing real whole foods.  Hmmm ... didn't someone just call that Ma Pi some cultish diet?   But I digress.  They are going to do this for a year.  Interestingly the principal investigator is Gardner.  Hmmm ... that should ring a bell!   He was the principal on the Grandaddy of them all!  The A to Z weight loss study!   The Atkins group lost more weight vs. Zone but not statistically more vs. any other diet.  In typical trajectory fashion.   But I guess this needs repeating?  Because Shai didn't test extreme enough diets I suppose, and the 6 year followup was so promising for even their version of low carb.  

You see, what I get from studies like Ebbeling, is that the more you do to ensure compliance, including providing most all foods prepared for people, the better the result.   You are always going to have your zealots who manage to latch onto some extreme diet and stick with it.  Unfortunately for many of these, this seems to be to the detriment of their health despite claims otherwise.  All of which just makes for more confusion for everyone else.  

All of these paleo types would do better to discuss what individual cultures ACTUALLY ATE rather than their fad diet marketing and profiteering.  These clinical trials?  Useless.  If you want to look at removing some purportedly harmful dietary agents, then that's what the studies need to do, in controlled fashion from both an experimental design point of view and an implementation/compliance point of view.   This still will not provide all of the necessary answers, but at least it will provide reliable data for the rest of us to discern which dietary factors we might consider changing on the backdrop of how humans have eaten through the centuries and how certain of us respond, at least for finite times, to certain foods in certain disease states, etc.

The diet comparison studies for general health and weight loss are on the whole a meaningless waste of time, effort, and most importantly money that could be better spent healing people that need healing and supporting people that need support, etc.    Imagine if you will, the money being spent providing real food meals for parents to provide for their children in impoverished neighborhoods.  With carbs and fats and proteins.   Just imagine.

UPDATE about 1 hr after posting original:

It's really soon to post as an edit, but this is an afterthought that should have made the post.  If I were looking to RCTs to pick a dietary path, my interest would lie in what worked best for those who stuck with something, as well as general adherence rates.  Still, to me the latter are not all that meaningful.  I know, based on my life and the nature of the man I share it with, that certain diets would never fly for the long haul.  He's a meat eater, and a carb eater, and he would not last long on a diet that restricted one or the other severely or eliminated them entirely.  While I don't share his *need* in the same way, I couldn't imagine life as a vegan, nor does the high fat greazy keto diet hold much appeal to me.  I am more of a light meat with some starch on the side or fruit and dairy kinda gal.

One of the problems I discussed in this post, was the problems with the often-used "intent to treat" analysis.  When they carry forth last known data points of subject that drop out (formally or just don't really fully comply) it renders the results even more meaningless.  When I was more solidly low carb, I wanted to know if this was healthy over the long term.  I wanted to know what following such a diet did to others -- those who followed it, not those who gave up.  Perhaps some info on hormones and such I didn't have the financial wherewithall to have tested or lacked any baseline data for anyway for comparison.    This data is pretty much nowhere to be found largely because, as discussed from the get-go, most of the diets in the RCT's are not even what is prescribed.

So I wanted to add that to me, non-randomized uncontrolled studies that were well documented and monitored, would prove far more valuable than these RCT's.  Imagine if Jimmy Moore or Dave Asprey or Peter Attia or any of these other n=1's had actually registered to be monitored and verifiably tested along the way.  I don't care that these would be subjects # 1 thru 3 in some study.  If someone could be verified to lose fat, gain muscle, improve lipids, have healthy organ function, etc., etc., on a verified nutritional, exercise and supplement/pharma (or lack thereof)  regime -- to eat 3000 cal/day and really lose 100 lbs in 5 days doing that (I jest) -- this would be useful information to those seeking possible routes to try without feeling like they are risking their health based on claims made by people whose livelihoods depend on your continued interest.


I was surprised at who got all excited about this study given that a 15-second read of the abstract showed that at 24 mos it was, as you say, 'ho hum.'
carbsane said…
Chuckle ... If the reporting is accurate (ahem, further chuckle) the PD group cut calories more from 6 mo to 24 mo and gained weight, while the NNR group cut fewer calories vs. baseline during that same period than they did at 6 month and continued to lose a bit more weight.
charles grashow said…
Beans, Beans theMagical Fruit: Why the Paleo Diet Should Not Exclude Legumes
carbsane said…
That's a wonderful paper. It is unfortunate that legumes got lumped in with grains. There are some key differences where the latter may not be a good choice but the former are. Let each class of food stand on its own merits.
silentshade12 . said…
Well, as Plant Positive said in his videos on legume, beans are the equivalents (actually, much, much healthier - but that's beside the point) of meat, so if you're going to push a diet that is heavy on meat, you better make people think they have no other alternative to meat.

Beside, except for celiac, there is nothing wrong with whole grains either.
eulerandothers said…
I have to put aside my distaste for Taubes in this case. I notice that John Ioannidis, Ph.D. is listed on the roster of investigators. Unless he also is tempted to take professionalism down a notch (and I hope that's not the case), he'll at least pay attention to the data and results and have some strong opinions about how meaningful they are. His name - attached to this - is a good thing, IMO.

I have said before that if Taubes wants to show that calories don't matter, all he has to do is put himself in Times Square in a climate-controlled box and stuff his face with steak and butter all day long, with a calorie counter marking all his gluttonous excess for the world to see. And then not get fat. Something along the lines of a David Blaine stunt. Because it WOULD be magic if he could 'sell it.' Of course, he'd have to stick with it for at least a month or two and I believe the blood test results would have to be posted alongside those calorie counts. Plus sheer poundage of meat and lobster and eggs and butter consumed.

Peter Attia is no dope. I think I read something in a blog entry he wrote about how he is not putting his kid on a low-carb diet. Smart. I know people who would 'drink this kool-aid' and impose it on their kids. Just because the medical establishment is SO wrong.
carbsane said…
NuSI got some big names attached as advisors or investigators. I am not really sure what that means. The most cynical view is that money talks. But let's face it, none of these proposed studies are any "better science" compared to that which Taubes and Attia have trashed along the way, and I don't seen much coming from any of it either way. If NuSI came my way and offered me a ton of cash to do any of these studies I wouldn't turn it down.

I hope in the new Ebbeling they will take EE readings at weight stability on the weight loss diet before the switch to the maintaining diet. Which will still tell us nothing much worth acting upon.

I believe both Taubes and Attia are married to smart women who are protecting the children ;-)
ExEffectsGuy said…
all he has to do is put himself in Times Square in a climate-controlled box and stuff his face with steak and butter all day long, with a calorie counter marking all his gluttonous excess for the world to see. And then not get fat.
I believe there is a blogger or two that has tried this and other than their words, I think visual evidence has shown it just plain don't work!
Tsimblist said…
Just watched that video and the altered voice of Plant Positive reminded me of another video producer by the name of Michael Greger M.D. from
charles grashow said…
The impact of dietary fatty acids on macrophage cholesterol homeostasis
eulerandothers said…
Carbsane, I think this guy Ioannidis' participation may be the result of having big bucks thrown at him (NuSi did get a chunk of cash from a wealthy patron) but I don't know that his critical faculties are going to be easily compromised by whatever spin Taubes wants to put on the study results. In that case, Ioannidis' rep would take a hit and he's unlikely to want that, even for the sake of some low-carb guys.

At any rate, the peer-review process, which bedevils these attempts to show the strength of low-carb will be a good thing. At least the numbers will be good!

For Attias and Taubes, the publicity will be good. That's always the case. Even bad publicity can be good publicity if people read about you. Especially if you continue to stir up your supporters.
ZM said…
What's the point of this?
carbsane said…
I think it's a win win for all involved ... except some poor schmo diabetics perhaps, that might have benefited from the funds being spent on a non-for-profit diabetes center instead of this.

I'm sure if they get anything the scientists will couch it in appropriate terms and no reputational flogging will be had if they leave open the possibility that future studies might one day settle it.

That's where we're supposedly at anyway.

Maybe I'm just getting too cynical in my middle age ;-) But as I wrote a while back, none of these experiments will answer the questions or meet the bar NuSI itself set. We didn't eat 30% carb diets let alone 10% ... ever ... so even IF we reduced fat from 40 to 30% and increased carbs reciprocally, you'd have to show a difference between those two diets, or maybe you go 50/30. These RCT's are so utterly meaningless!!
LWC said…
You know, that's as good a benchmark as any I've seen. If you (or your wife) won't feed your kids what you eat because it's not healthy for them... then something's amiss here.
scott said…
Any idea how much error is in the energy intake data?
Reijo Laatikainen said…
Yes, the adherence is the key. In some RCTs food items are given for free for years and sometimes participants are even paid for their efforts. Weight loss and other health outcomes seem to get better if such incentives are used. If you want to succceed with your longer term RCT, be prepared to spend big bucks. I just wonder what could make people change their diet big time without using monetary incentives. Pretty sad actually how difficult it is to change dietary habits substantially in these weight loss studies.
carbsane said…
There is no way to tell. They all apparently underestimate throughout the study which isn't a very good indication. :(
charles grashow said…
charles grashow said…
This should be interesting


Staffan Lindeberg, MD, PhD; Maelan Fontes, Ms & Pedro Carrera Bastos, Ms (Lund University, Sweden)


Scientific, Historical and Evolutionary basis of Nutrition:
Myths and Controversies in Nutrition
Evidence based Nutrition:
Introduction to scientific research
Types of research studies
Biostatistics 101
Principles of Evidence Based Medicine
GRADE system applied to Clinical Nutrition
Main biomedical databases
How to conduct a proper bibliographical search
How to critically analyze a scientific paper
History and Evolution of the Human Diet:
Introduction to Evolutionary Medicine
Brief evolutionary history of the human diet
Health, fitness and body composition markers in non-industrialized populations
Major dietary changes after the Agricultural and Industrial Revolutions and the Modern
Review of the intervention trials testing Paleolithic diets
charles grashow said…
carbsane said…
Uggh ... this is Ioannidis (knew that rang a bell)
eulerandothers said…
The article you link to - that's the kind of thing he does. I think that's all he does.
carbsane said…
Well then he gives faux cred to NuSI because he's a hack, to be blunt. This guy didn't analyze a bunch of studies and FIND errors, etc. he "modeled" it. Horrible science. Just horrible.
eulerandothers said…
I don't understand the statistics side of the work well enough to appreciate what he's done. But this is all he does. I came across an article about him - come to think of it, I think it might have been in a mainstream publication.

He does publish a lot. So the good thing is he has a reputation to protect. If his work with NuSi is controversial, then it won't be good for him or for them. If a lot of low-carbers think everything you do is ground-breaking, that's the Taubes story. I don't think Ioannidis is going to get that kind of free pass.

(BTW, that's not the mainstream article I read about Ioannidis!)

Maybe it will get ugly - but at least it will be a brawl among statisticians and there are some heavy hitters out there. I am looking forward to it.
carbsane said…
See my point about the stats is that he modeled something. He didn't actually LOOK at the studies and find errors or possible ways bias was introduced by design, or conclusions were skewed by fancy statistickery (just made that up :-) ) -- no he just made up some algorithm by which he determined that this might have happened.

I liken it to some of the risk studies where the models predict considerably different results than were actually observed due to weighting, etc. Or like that hyperglycemia causes dementia study where it could be shown in non-diabetics to seemingly have some effect, but the diabetics actually had lower rates of observed dementia.

Stuff like that drives me nuts.

I don't think much about his involvement either way. It may give false cred just to have him on board though.
Sanjeev Sharma said…
> wonder what could make people change their diet big time without using monetary incentives

this is also a formula for long term failure.

When participants in executive function and various economics/sociology experiments explain their actions/adherence by recourse to the external incentive, when that external incentive's removed the adherence rebounds to a worse than baseline level.
Blogger said…
Are You Having Trouble STICKING with the Paleo Diet?

Want to eat healthy delicious recipes TONIGHT?

Download your amazing Paleo Hacks Cookbook.
Blogger said…
New Diet Taps into Innovative Concept to Help Dieters Get Rid Of 15 Pounds within Just 21 Days!
Blogger said…
EasyHits4U - Your Free Traffic Exchange - 1:1 Ratio, Business networking. FREE Advertising!