The Scientific Method 101

OK ... I'm about at my wits end with some of these discussions about the net.  It's as if basic logic is not enough for some folks.  The scientific method for testing hypotheses is not difficult folks.  You don't need a genius IQ or even a high school diploma to "get it".  

Let's say, hypothetically that based on my observations, I propose A causes Z.  Others propose B, C or LMNOP cause Z.  I design my experiment.  Realistically it's impossible to control for all possible other factors -- called confounding variables or simply confounders -- but I make a good faith effort to do so.  In other words, I consider that B or C etc. may be the actual cause and merely occur concurrently with my beloved "A".  


And so I control.  In my experiment I have a study group (aka treatment group) where I vary "A", and my control group.  My control group will be as like my study group in other variables as possible.  Therefore when I see a statistical difference in Z in my study group as compared to the controls, this is convincing evidence that my manipulation of A was indeed the cause.  True, we cannot rule out that there was a confounder "Q" I did not account for, but good scientists make it a habit to control for at least the obvious confounders.

So ... let's say I say A causes B.  And someone else, say George Thibideaux, aka GT, says C causes B.  I design my study.  I randomly assign subjects meeting certain criteria to two groups.  I then vary A in my study group while keeping C the same between the two.  I see a change in B.  This does not "prove" A causes B, but it is consistent with the hypothesis that A causes B.  Further, if I repeat similar experiments (always keeping C constant) enough times, and the expected result for my hypothesis occurs time and again, this becomes convincing evidence that, indeed, A causes B.   

I'm sure everyone would agree with the above.  If you do not, there really is no reason to read on because you are so hopelessly biased because you know where I'm going with this that you cannot accept basic science.  

Let's put dietary "stuff" to my ABC's.  Let's say "I" am actually Gary Taubes.  Gary Taubes claims carbohydrates, and solely carbohydrates "A", irrespective of calories "C" (yes, that's what he now claims) causes us to get fat.  So his "A" is carbs (through the virtue of insulin stimulation) and his "B" is weight gain.  Now some formerly bunny-eared anonymous blogger, aka your's truly claims that it is "C" that leads to "B", that "C" being caloric surplus irrespective of macronutrient composition.  

We test this.  How should GT test his?  By controlling for calories.  If you do not, it's a moot point and I'm willing to bet that most if not all of Andreas' 14 studies (to zero?  Really, you're serious?) did not control, let alone carefully, for calories.  The studies that control calories demonstrate no weight loss with even dramatic changes in macronutrient composition.  On the other hand, if I want to test my hypothesis, I can control for carbs and alter calories.  Guess what.  The data support that every time.  PERIOD.


Comments

. said…
Some good articles for reflection:

Höfler M. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol. 2005 Nov 3;2:11. http://www.ete-online.com/content/pdf/1742-7622-2-11.pdf

Phillips CV, Goodman KJ. Causal criteria and counterfactuals; nothing more (or less) than scientific common sense. Emerg Themes Epidemiol. 2006 May 26;3:5. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1488839/pdf/1742-7622-3-5.pdf

Maziak W. Point-counterpoint. The triumph of the null hypothesis: epidemiology in an age of change. Int J Epidemiol. 2009 Apr;38(2):393-402. Epub 2008 Dec 17. http://ije.oxfordjournals.org/content/38/2/393.full.pdf

Fraser GE. A search for truth in dietary epidemiology. Am J Clin Nutr. 2003 Sep;78(3 Suppl):521S-525S. http://www.ajcn.org/content/78/3/521S.full.pdf

Taubes G. Epidemiology Faces Its Limits. Science; Jul 14, 1995; 269, 5221. http://bit.ly/rrKwjl

Berger VW, Ioannidis JP. The Decameron of poor research. BMJ. 2004 Dec 18;329(7480):1436-40. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC535970/pdf/bmj32901436.pdf
. said…
All the metabolic ward studies Colpo quotes in his book THE GREAT CHOLESTEROL CON can be downloaded from here -> http://depositfiles.com/files/5833g450z
Frank said…
Not only they did not control for calories but not for protein either...

I'd like to ear Andreas on why these studies should be use to prove anything beside that LC works well to spontaneously reduce caloric intake. And i'm also wondering if he read the whole paper as i'm pretty sure the reason given by the researchers must be the higher protein intake in the LC group. Is there anyone claiming in these papers that LC works well through a reduction of insulin?
Mirrorball said…
Evelyn: "If you do not, it's a moot point and I'm willing to bet that most if not all of Andreas' 14 studies (to zero? Really, you're serious?) did not control, let alone carefully, for calories."

None of the ones I checked controlled for calories (I didn't check all of them though). Several people have pointed this out in comments at SG's blog.
CarbSane said…
Hi Frank ... I may or may not have posted my comment to you on another thread welcoming you back (dropped the laptop and broke the charger "plug" right off today! Yikes! It works again ...)

But good to see you. I looked at his 14-0 post and note that Dansinger and Foster are missing. Oh ... but Dansinger didn't find statistical difference. Well, Shai IS on the list and she didn't find one for LC v. MDTN either, and that one is the most damning which is why I'd like his response to it. Sure ... if we ignore 1/3rd of that data and take poorly collected (what a few days for some of the participants from each group over two years?????) intake data at face value, LC lost a bit more than LF. But you also have to believe that the low carbers kept to their diets and regained about half their initial losses. OTOH, these folks cannot explain away Grey & Kipnis except to call my blog twaddle and accuse me of stalking and mental instability. Okey dokey! :-)
Sanjeev said…
why these studies should be use to prove anything beside that LC works well to spontaneously reduce caloric intake
____________
I perceive a need to have "low carb/insulin" be the explanation, rather than a technique that works in line with the explanation, which is conservation laws.

I personally don 't see the need. Feinman claimed low carbers encourage people to "find what works". Well if low carb is a TECHNIQUE you CAN drop it easily after giving it a good go, and pursue another technique that implements the solution, and may be a better fit to the individual.

If (low carb)/(insulin reduction) is THE ANSWER, THE EXPLANATION (the one that many have wedded themselves to) and conservation laws tell you nothing (a corollary to the earlier wedding), is it any wonder they have such a hard time moving on?
CarbSane said…
Thanks Mirrorball. I admit, with limited time to pursue things that interest me more, I can't be bothered to look into these supposed slam dunk lists of studies supporting LC. It works when it does. Stipulated. Andreas is saying nobody really knows why for sure. Really? That the carb/insulin hypothesis is the best we have. Sigh.

When someone can explain this without resorting to their own ad hominems on me, I'm listening (as Frasier would say).

http://carbsanity.blogspot.com/2011/04/fasting-insulin-weight-loss-ii.html
CarbSane said…
@Sanjeev, if I understand your point here it is that TWICHOO should explain it in every case but it does not. Not even close. And thus that LC works but in apparently different ways is in and of itself a refutation of TWICHOO.
. said…
I beleive for healthy individuals any dietary intervention, either low or high in whatever macronitruents, by representing an approximation to the ancestral/traditional food patterns and by eliminating the so called neolithic agents of disease, is usually better for health and will help with weight loss. When it comes to certain diseases, like diabetes and cancer, low-carb is certainly a good approach, as long you focus on healthy food (not Atkins bars and similar processed foods) and control calories.
Frank said…
HI CS

Thanks for the kind words. I realize i've been really bitter recently in my comments, and I had stop coming on various blog since actually, all of this is depressing me more than doing any good. I think I came back too fast :D

Yes, it's indeed funny how it's easier to call you out a psycho and a stalker than adressing all the good things that you have brought up on your blog.

Are his 14 studies correcting for water and glycogen loss? I haven't look at all of them in details.

Finally, here's my tought on all of this.

No one in the research feild seems to seriously consider Taubes idea. Most pro coach, in the fitness industry, that I know of, especially the evidence-based one, are not using a low-carb approach for everyone. It's pretty damn hard to find research papers that clearly support the theory, most of what we know regarding IR is much more closely related to NEFA, as you have exposed many time here already. I'm not even sure there is one paper out there which make the assertion that eating carbs cause IR.

So, who have we left believing in carbs ­­ = insulin = IR/weight gain? Anonymous blog readers, as far as I could read, usually with not much scientific background, who read GCBC, tried a LC diet, got good result from it, and now are emotionnaly attached to it and defending it like it's their baby.

I'm not trying to attack anyone here. I'm just saying it the way I perceive thing. Taubes theory shall probably never make it in the scientific community. At least not with the evidences that we have so far.

So, why are we arguing with these kind of people anyway? All that mental masturbation is all good as long as they do it togheter, but as soon as they leave their glass house, no one takes them seriously.

Ah, still some bitterness left, it seems... but it's all good, i'm on a psycho blog anyway, after all :D
CarbSane said…
O Primitivo: I generally agree that in most contexts the so-called NAD's are appropriately labeled. I do not agree that they are necessarily linked to obesity, however. I've yet to see any compelling evidence to diet -> metabolic disease -> obesity. Rather there is lots of compelling evidence for diet -> obesity -> metabolic disease.
Tonus said…
@Frank: "So, why are we arguing with these kind of people anyway?"

I think that there is some context that we should keep in mind here, that helps explain the mindset of at least some of the layperson-types who try to speak authoritatively on the subjects of health and human physiology (did that sound hifalutin' enough?).

I get the impression that there's something of a groundswell of cynicism towards scientists and researchers, reflected in topics such as the debate over the climate and dietary recommendations. There is the impression that science, once considered noble and clean, has been subsumed by politics and money. Take note of a theme that runs through Naughton's Fat Head and Ravnskov's work (amongst numerous others): scientists and researchers have been misleading us and lying to us for years, and deserve our mistrust. Naughton punctuates this in one of his recent talks, where he makes the (mostly) joking comment that "scientists are liars."

With that sort of atmosphere surrounding the diet and health debate, it's no surprise that many of us who haven't a whit of training in science or health or medicine (or even statistics, for crying out loud) nonetheless feel entitled to wag a finger at scientists and researchers who we feel are being, at the least, not totally honest with us. It doesn't matter that we don't have nearly as much of a clue as we pretend to have. We have something better: our cynicism!

Of late I'm less inclined to comment on the science because I'm simply not qualified to do more than ask for clarification of one point or another. I'm also past the point of wishing that someone had a magic diet bullet, so maybe that's part of it-- I don't need to feel validated, I don't really have a dog in this race. What good does it do anyone if I stick my uninformed nose into the discussion? Well, unless it's a particularly good zinger. I'm just amazing with those.
. said…
Maybe there are several paths to disease, and not all of them include obesity. For example, you don't have to be obese to acquire diabetes. But many obese people become diabetic. Certainly there are environmental factors we are not aware, but diet should be a major factor. Obesity is a disease that leads to other metabolic diseases, or just a symptom of a metabolic disease already in progress? Janet Ruhl suggests that thin type 2s disprove that obesity causes diabetes. Is this a rule or an exception?
Rad Warrier said…
"I get the impression that there's something of a groundswell of cynicism towards scientists and researchers, reflected in topics such as the debate over the climate and dietary recommendations."

Since I don't have religious faith in religion itself, let me be a little irreligious and politically incorrect here. I feel that many vociferous low carbers have a need to believe in a religion and the low carb faith comes in handy as an attractive religion that is anti-mainstream, anti-establishment, "modern", and (pseudo) scientific ;-). And like many adherents to organized religions, these LC'ers become highly intolerant of any other point of view. You know, there his only ONE WAY, and that is my way ...

Regards,
Rad
Sanjeev said…
> TWICHOO should explain it in every case but it does not

I don't need more reasons to question TWITCHOO - even setting aside the disconfirming research, the theory's purported sources and antecedents don't support it. That's enough for me.

I was just surmising why some people are so attached to it.

My analogy was people are confusing the tool (low carb) with the work the tool does (lets some count calories implicitly by reducing hunger).

Or maybe raising the tool's importance above and beyond what the tool does.

and they're losing sight of the fact that other tools work for some too. ELMM, Veganism, Shangri-La (and its variations, crazy - spicing for example)
Duffy Pratt said…
The whole idea of a "cause" can be very slippery. The best any scientific study can do is to show correlation. To be more precise, the best it does is to demonstrate that a correlation has not yet been falsified. Whether there is causation involves other considerations (and often I think those considerations may be unscientific.) On the flip side, given accurate enough data, it only takes a single counterexample to falsify a hypothesis.

Suppose the question we want to answer is "What has caused the obesity epidemic?" or "Why do cultures tend to develop obesity problems when they come into contact with the west?" For these questions, saying that large portions of the populations ate more calories than they required y does not supply a satisfactory explanation. It's true, but it doesn't say why it happened.
CarbSane said…
O'Primitivo: The ending of that Taubes piece is interesting! Unfortunately I can't C&P but it deals with observational studies. Taubes should take that advice with all of his "what about the _fill-in-the-blank_ people and their obesity" observations he insists are consistent with carbs -> obesity. However I don't really agree about when a lack of correlation is found. Where there's correlation there may not be causation, but where there's causation we'll see correlation. However, in experiments we manipulate one variable (hopefully) and measure outcomes. Therefore, while statistically cause is virtually impossible to attribute well designed experiments support it.

In the diet studies, when cals held constant and change up macro -> weight stability. Macro %'s held constant and reduce calories -> weight loss, increase calories -> weight gain. We know what caused the weight outcomes.

Because of Eades you have all sorts of folks running around the internet talking about black swans and Popper and falsifying and whatnot. In Eades' world 1000 studies demonstrating no metabolic advantage are outweighed by one that shows one. All we have to do is reword our hypothesis and find the one black swan. Eades concludes that the one study would thus "prove" an MA exists. Nonsense!
Frank said…
@Sanjeev

Indeed there is a lot of confirmation bias.

I understand that saying that people are getting obese because they eat too much is useless. But why does it has to be carbs = insulin = make you eat more? What about the fact that people are eating much more in restaurant? That food is much more calorie dense that it use to be? Much more palatable? What about emotionnal eating? What about the fact that we are sedentary 95% of our waking time?

One must not look very far to understand why we eat more than we use to be, and why we put on weight. And then the real things get set, ie. weight gain = metabolic dysfunctions = diseases.

It is pretty well known that simply losing weight will make about every biomarkers improve. Just look at the twinkie diet guy.

@Tonus

Indeed, science has no good reputation lately. And it's true that there is some (big) weakness to science. But once you know science, when you known it's weakness, it's much better to do things with science and its weakness than without science at all.

@Rad

I'm a bit like you. I'm a young, lean and quite muscular guy who never have had to lose weight in his life. So I never had to favor a particular diet. Currently my master degree paper is on the effect of low or high carb meals (for 48h) post-workout on the effect of insulin sensitivity in the elderly. I've been reading the litterature quite a lot lately and it's not so promising. We shall see, tho.

I'm working with client on a daily basis tho and I know I would do them a disservice if I was misleading them in believing that they can never eat carbs again if they want to stay lean.

As far as I could gather so far, a diet based on the mediteranean one, where you adjust carbs intake relatively to training level, is the best way to go, offers a lot of flexibility and can easily be sustained most of the time. It's also what is in line with what can be found in the research.

So obviously, when someone has had very good result using a particular diet, it will be very hard for him/her to come to the conclusion that the diet might not have been optimal. And this is what we are looking for, aren't we? The optimal way of doing thing, not just something that works.

There are clearly downside to LC, especially in training individual with an active lifestyle... and shouldn't that be everyone?
Frank said…
@O Primitivo

BTW, thanks for the metabolic ward studies! I was missing few of them!
Tonus said…
I think that the idea that CI/CO is useless because it doesn't explain the WHY is an attempt to dismiss the idea in favor of the metabolic advantage concept. And I remain utterly confused as to why anyone would assume that the two must be exclusive.

We know that people can control their weight by managing their intake and expenditure of calories. Aside from a few quacks (with their anecdotal claims of 500-1000+ metabolic advantage) most people following any diet advice understand this. I see CI/CO as a part of a larger puzzle, as a starting point.

Even if you assume that one macro-nutrient (or food) is more 'efficient' than another, at some point you'll need to figure out how much to eat in order to lose or maintain weight. Leaving it up to your body to manage this strikes me as an attempt at finding that magic bullet. It's the same attitude that leads to sales of every sort of strange exercise doo-dad under the sun.

This is (as I've said before) why the miracle stories of low-carb providing weight loss in the face of dramatically increased calorie counts are so harmful. It will stop at least some people who are at a crossroads from making the obvious choice and seeing improvements, and will keep them on the "something else must be wrong" roller-coaster.
CarbSane said…
@O Primitivo: Regarding Jenny Ruhl, she seems to have come to this issue from a position of her own frustrating journey. She, herself, admits and discusses her rare form of the disease (genetic) but then seems to align herself with other non-obese diabetics. Indeed what I would call the Fatty Acid Hypothesis of Frayn (and others) is consistent with lean diabetics (see: ). Marry that with Freeland's critical VAT threshold theory: http://carbsanity.blogspot.com/2010/08/critical-visceral-adipose-tissue-theory.html
CarbSane said…
Oops, here's the Frayn post link: http://carbsanity.blogspot.com/2010/07/progression-of-insulin-resistance.html