The Scientific Method in the Low Carb Echo-Chamber

I don't know how many of my readers share this impression I have of the low carb online community on the whole, but here it is.  To me it seems this community has fallen victim to the same kind of group-think many have long decried, hanging out at sites where non-believers are turned away.  No differing opinions allowed in such echo chambers.  

The result of which is that we see the sort of repetition of catch phrases that would make a political party proud.  You know what I'm talking about ... when the various mouthpieces making the rounds of the Sunday shows, morning shows and various cable and talk radio shows all seem to have the same speech patterns.

And so it is now that low carb cookbook authors and comedians give speeches on how to critically evaluate scientific studies.  The former individual I'm speaking of is Dana Carpender.  Her speech on the recently concluded Low Carb Cruise (Jimmy Moore) was posted online:  Part 1, Part 2.  (Direct links:  Part 1, Part 2.)

I have much to criticize on this speech, that I may or may not share in some future post.  I'm not sure if it's worth it.  She seems like a nice enough person who has the sort of personality that I'd probably make fast friends with were I to meet her in an LC-free context.   But after telling us that she's just a housewife who is a college dropout (of unspecified field of study but I think a science major is probably out of the question), she procedes to instruct the audience on evaluating studies and takes a few swipes at researchers that would, in another context, warrant an email from the ......  Just rubs me the wrong way.  But I digress.

Anyway, I listen to the speech and the catch phrase that perks my (former bunny) ears (around the 7:30 mark of Part 1) is:

"The point of scientific study by the way 
should be to disprove your hypothesis."

at least one audience member close to a mic: "Yes"

  Dana:  "Yeah"

Now where did she learn this from?  Why are people following along chiming in with approval?  I'm guessing it traces back to Gary Taubes and/or Mike Eades.   Neither of whom, I might add, appear to have ever engaged in actual scientific research.  You'll have to take my word for it, but yours truly here actually has - both in academic and industrial contexts - for well over a decade.  

I posted previously a rather long, somewhat rambling post about the issue I have with Dana's statement:  Testing Scientific Hypotheses v. Statistical Hypothesis Testing.  After Eades' black swanning attempt to dissemble Colpo's refutation of the metabolic advantage, black swans were popping up in the comments on blogs and forums far and wide.  And ever since Taubes started insisting his hypothesis should be the "null" hypothesis requiring extraordinary evidence to discard, this notion has taken off.  The Cliff Notes version of my prior post is as follows:  We design scientific study to generate data in hopes that it supports one's hypothesis.  Statistical hypothesis testing methods are applied to the data - often several times, several types, etc. - to assess the significance of the degree to which the data demonstrate an effect, etc.  Statistical hypotheses yield the most definitive conclusions when they are formulated so as to be disproved by the evidence.

To state that a scientist wants to disprove their hypothesis, or that they should be doing this, or this is the real purpose of scientific inquiry is just absurd.  Below is a graphic from a source that appears to be geared towards creating that reasonably educated child.

My only critique of this is that there's another option if you find yourself that poor dejected "No" dude at the center.  A good scientist would also consider whether it was the hypothesis itself is that is faulty.  They would then revise the hypothesis and repeat the process.  

Ultimately, however, the scientist is looking to be the "Yes" dude there!  

There is a lot of criticism to be had for not reporting/publishing "No" studies, but folks this is just a reality.  I encountered this problem with my masters thesis research.  As it turned out, the phenomenon I was investigating was applied to the physical outcomes of a process, but the mechanism was vastly different in my system than for the systems in which this phenomenon and the methods to study it generally apply.  My thesis and the subsequent peer review journal article had to be very carefully worded.   If not for the pressure to publish - so I could get my degree w/o having to start all over with another research project - it's doubtful I would have put so much energy into doing this.  In the "real world" this is expensive and time consuming.  Money and time that would be better spent, in most cases, on starting over.  At the very least, continued research to refine the hypothesis would likely take precedence over publishing - although there's the ever-present need to maintain funding sources.

That we know there's a correlation, even perhaps a causation between A and B but we don't know how A causes B, there can be ten different hypotheses as to how A causes B.  Say a study on one such hypothesis comes up empty.  In an ideal world this gets published to communicate the futility of this line of thought, but in the real world there's nothing nefarious about the fact that it just doesn't make it to the journal.  This is not to say that we don't ever see "No effect of ...." type studies, just that in general it's more difficult to get these published and/or there's often limited utility to the research group to expend the resources to publish rather than investigate hypothesis number two in hopes of hitting the jackpot.

Folks, if we want to advance the "high mindedness" of science of carbohydrate-restricted nutrition and/or promote the healthfulness, or at least dispel the paradigm of the unhealthfulness of high fat diets, it's going to require some genuine discussion on the science.  This includes dissenting opinions and challenging one's own biases.  For your individual learning, I hope you'll continue to read this blog (shameless self-promotion).  But I encourage you to read a variety of sources and not just confine yourself to those low carb echo chambers where 90% of the comments are "great post", "you're da bomb", and such.   I'd be a liar if I didn't say that when I get such posts here they make me smile and boost the ego.  But I do enjoy when those who find this blog from "enemy camps" challenge me and I encourage more (keep it respectful please) rather than less of this type of discourse.  (Apparently some other bloggers aren't so keen on having such discussions)

This is how we move things forward.  Not by reading sources that reinforce one dogma to replace another.  And not by dismissing out of hand any study that might produce results counter to the outcome you might have hoped for. 

Now .... go ... seek out those studies that support your hypothesis! (But don't ignore those that don't).   That's how science works!