Science ...
... it's called that for a reason.
This post was prompted by the discussions in this seemingly non-monumental post -- Are the Dietary Guidelines REALLY So Radical? -- that erupted to over 750 comments. The discussion somewhat culminated in this comment, and there are a few posts forthcoming that require this "backgrounder" on thoughts.
I like to blog about science. I like to deal in science. I consider myself a scientist in much the same way that formerly practicing MDs still consider themselves doctors. I used to conduct primary research. The kind that some of today's "scientists" have never even done outside of the classroom environment , if that (I refer to the likes of Gary Taubes, Zoe Harcombe and James DiNicolantonio to name a few). I studied science (a few disciplines, and some engineering too), and I had a career in scientific research (again a few differing fields and applications). I "can't help myself" whenever I read a scientific study but to analyze it. There is a lot of good research going on. There is a lot of crappy research as well. I'm hopeful that some find what I write about here helpful, though usually the more scientific the topic, the less interest a post seems to garner. This doesn't bother me, because in writing a blog post on something I'm looking into, I learn a lot myself, and as geeky as it sounds, this is something that is as personally rewarding or just plain enjoyable as some may find their own favored hobbies.
As I was writing this post and those it is background for, I realized that many links to former posts are around five years old, from within the first year of blogging. While my understanding of the concepts I've researched has widened, deepened, and been refined, much of the initial work holds up with few caveats. Perhaps it is time for that post about how there's no such thing as insulin resistance as we know it ;-)
In the five or so years that I've been doing this, I've come to formulate some of my own ideas regarding what is unknown. When I publish things up, however, I try very hard to distinguish these from what the scientific evidence can establish as "fact". {I use the quotes there, hopefully you know what I mean, I'm trying to keep this post reasonably brief.} From time to time I may speculate, and when I do so, I try to make it very clear that I am doing so. But most of the time I am discussing the work of others. If I were to constantly write blog posts to the standards that ought to be enforced for peer-review and academic educational materials, no posts would ever get published, and if they did they would likely be unreadable. There would be too many qualifiers needed. I give my readers credit that they understand then I use more definitive terms, I am conveying "what the evidence at hand supports", "what the scientific consensus is", etc. I wish others that blog on science, like a perennial medical student who think that makes him some sort of Authority, would do the same as a matter of routine.
All that said, what I do, and bloggers of a similar bent do, is secondary research anyway. There is a place for such research -- even, seemingly more and more so in the peer-review literature -- but the fact remains most of us doing so have no means to test what we come up with, and we're limited by not being privy to much of what is unclear or even left out of journal articles in the interest of brevity or because the primary researchers didn't find it noteworthy. Still as I feed my newfound fetish for older books on such topics as nutrition and physiology, it must be pointed out that there IS an awful lot that we DO know, and sometimes it amazes me how long ago some of these things came to be known. Probably the most frustrating thing has been finally getting my glycated paws on some obscure reference from Teh Bibble (aka Good Calories, Bad Calories) only to discover just how far Gary Taubes has led people astray. But there I go digressing again ...
Here is a flowchart of one version of the scientific method from Wikipedia. The one alteration I made was the arrow connecting the General Theories back to the Observations, as the way I see it, the blue/purple loop at the bottom there reflects the ongoing nature of science.
As I was writing this post and those it is background for, I realized that many links to former posts are around five years old, from within the first year of blogging. While my understanding of the concepts I've researched has widened, deepened, and been refined, much of the initial work holds up with few caveats. Perhaps it is time for that post about how there's no such thing as insulin resistance as we know it ;-)
In the five or so years that I've been doing this, I've come to formulate some of my own ideas regarding what is unknown. When I publish things up, however, I try very hard to distinguish these from what the scientific evidence can establish as "fact". {I use the quotes there, hopefully you know what I mean, I'm trying to keep this post reasonably brief.} From time to time I may speculate, and when I do so, I try to make it very clear that I am doing so. But most of the time I am discussing the work of others. If I were to constantly write blog posts to the standards that ought to be enforced for peer-review and academic educational materials, no posts would ever get published, and if they did they would likely be unreadable. There would be too many qualifiers needed. I give my readers credit that they understand then I use more definitive terms, I am conveying "what the evidence at hand supports", "what the scientific consensus is", etc. I wish others that blog on science, like a perennial medical student who think that makes him some sort of Authority, would do the same as a matter of routine.
All that said, what I do, and bloggers of a similar bent do, is secondary research anyway. There is a place for such research -- even, seemingly more and more so in the peer-review literature -- but the fact remains most of us doing so have no means to test what we come up with, and we're limited by not being privy to much of what is unclear or even left out of journal articles in the interest of brevity or because the primary researchers didn't find it noteworthy. Still as I feed my newfound fetish for older books on such topics as nutrition and physiology, it must be pointed out that there IS an awful lot that we DO know, and sometimes it amazes me how long ago some of these things came to be known. Probably the most frustrating thing has been finally getting my glycated paws on some obscure reference from Teh Bibble (aka Good Calories, Bad Calories) only to discover just how far Gary Taubes has led people astray. But there I go digressing again ...
Here is a flowchart of one version of the scientific method from Wikipedia. The one alteration I made was the arrow connecting the General Theories back to the Observations, as the way I see it, the blue/purple loop at the bottom there reflects the ongoing nature of science.
As it may have become clear in the mega-comments section on that recent post, or from much of what you might read on blogs and social media on scientific topics around the internet, there seems to be widespread misunderstanding of what the scientific method involves amongst the general public. Some commentary -- you can use browser search to find each section
- There are no null and alternate hypotheses in scientific research
- It's not science without a mechanism
- Good scientists don't spend their lives trying to disprove their hypotheses
- Good Scientists Stay In Boxes After All
- Good Scientists Recognize When To Give it Up
1. There are No Null and Alternate Hypotheses in Scientific Research
It is so -- so, so, very -- I cannot even begin to find all the adjectives how very, very, much very -- frustrating to keep seeing certain complex questions being couched in this dichotomous fashion. I discussed this at length in the following blog post:
In a nutshell, the Null/Alternate hypothesis pair are complementary (opposite) hypotheses used in statistical testing. For example to see if the change in weight in a weight loss study is statistically significant, or if there is more than a small chance that it could have occurred at random. The null hypothesis is presumed true as part of the testing process, and it is common to set up the test such that the null hypothesis is rejected to reach more definitive conclusions. These hypotheses are simple statements such as "the mean cholesterol levels of groups A and B are equal" (thus the opposite is simply that they are not equal, aka different). In any given experiment testing a single Scientific Hypothesis, one may do one or several Statistical Hypothesis tests.
Meanwhile, Scientific Hypotheses are rarely an either/or situation, especially for complex problems. TWICHOO is not an "alternate" to CICO (or as they like to call it, gluttony & sloth), there are any number of other hypotheses one could construct.
In formulating and putting forth a Scientific Hypothesis, the onus is on the person or persons putting it forth to do the background "homework" and then test it. A goodly portion of testing is done during the formulating process. The primary researcher relies on observations and existing literature just as the secondary one does. It is critical that the review of "facts in evidence" be thorough. Failure to do so erodes the credibility of the researcher. When Gary Taubes says:
"One of my goals here is to get the research community to understand that there is an alternative hypothesis that should actually be the null hypothesis — the hypothesis that requires remarkable evidence to reject. "
... he is communicating to the world just how unscientific his approach ultimately is, and/or just how poor he understanding of the process is. He should provide the evidence in support of his hypothesis, and once you distill his "works" down, it is shocking how little he actually puts forth in this regard. The above statement is not even accurate in the statistical context. The null hypothesis is simply the one -- of an either/or pair of complements/opposites -- that is presumed true in the testing process. Such "truth" is often arbitrary or irrelevant to what the scientist or statistician believes to be true. This is a procedural designation where the null hypothesis is the mathematical expression containing the equality. Anyone still doubting can find a similar discussion here.
2. It's Not Science Without A Mechanism
I struggled with this one, but the more I thought about it, the more this simple statement distinguishes a scientific approach from others. I don't recall seeing this in textbooks in so many words, perhaps because this is presumed to be the case, but the elucidation of a mechanism is what ultimately sets science apart. It is not enough to make some observations and throw a guess out as to why that is. That's the beginning, but it's not science yet!
You cannot go from the *what* to the *why* without the *how*.
Oh look! Obesity is on the rise. "It's the fructose!" cries Lustig. How so? Then the explanations are attempted, and yet there are very few that hold up under scrutiny (I'll revisit that topic on the blog here shortly!). How about one that's a bit more out there. Candida causes obesity!! This is one of three culprits identified by "researcher" (cough ... choke) Zoe Harcombe. How is that supposed to work?? Candida signals your brain to "feed me" ... I have some swampland ...
The mechanism is the key to the scientific hypothesis:
So on this point I will give Taubes some props. He laid out a pretty extensive list of mechanisms by which carbohydrates would be fattening. But just as I giveth, so must I taketh away, because there are simply too many existing observations and experiments that are inconsistent with and flat out contradict those same mechanisms (see this post on what his seminal 1973 source had to say 35 years before GCBC). Taubes likes to claim that the pathetic realm that is nutrition science has never tested any of his ideas, but when you look more closely you usually find that they have.
- Observations must be consistent with the mechanism
- The mechanism must be plausible.
- The mechanism is used to make predictions, design experiments and ultimately test hypothesis.
So on this point I will give Taubes some props. He laid out a pretty extensive list of mechanisms by which carbohydrates would be fattening. But just as I giveth, so must I taketh away, because there are simply too many existing observations and experiments that are inconsistent with and flat out contradict those same mechanisms (see this post on what his seminal 1973 source had to say 35 years before GCBC). Taubes likes to claim that the pathetic realm that is nutrition science has never tested any of his ideas, but when you look more closely you usually find that they have.
3. Good Scientists Don't Spend Their Lives Trying to Disprove Their Hypotheses
If I never hear another failed cruiseship comedian, Splend-iferous cookbook author or bemirdled diet doctor claim good scientists endeavor to disprove their hypotheses it would be too soon. A Scientific Hypothesis seeks to explain observations, it's supposed to be -- hopefully! -- correct. Surely nobody thinks scientists sit around thinking up every possible scenario that doesn't explain an observation! The primary researcher uses their proposed mechanism to design experiments. Their hope is that the results support their hypothesis. If A causes B by increasing hormone X and transporter Y and inhibiting factor Q, then one would expect increasing A results in increases in B, X and Y, and a decrease in Q. The scientist doesn't endlessly search for every other hormone, transporter and factor in my hypothetical alphabet soup here to see if it might be something else. One might test their hypothesis by varying levels of A, blocking X or Y, etc. This is done systematically with the predicted result in mind. There's nothing wrong with expecting an outcome, only if you don't deal with the unexpected.
If there are competing hypotheses out there, a scientist may keep those in mind by addressing how the results are explained by their hypothesis and how they are inconsistent with the competing one. Where possible, they may try to control for this potential confounding variable, but this would be a subjective decision (see point 4). In the end, one cannot control for every last possible factor, which is why we have randomization and large sample sizes to hedge our bets on.
Realize that all research that has gone before is effectively observation moving forward, and is taken into account at the beginning of the process. The secondary researcher, needs to do "excessively due diligence" because they cannot experimentally test their hypothesis. They are limited to what information is out there. Instead of the "Develop Testable Predictions" and "Gather Data" bubbles in the loop there, one might envision a "Have You Seen This" ... or "New Study Data Emerges", at which point the hypothesis needs to be re-evaluated to see if it is consistent. At the very least, a cursory review to formulate a hypothesis is "tested" in the purple loop by leaving no scientific stone unturned. Keep in mind that not every "have you seen this?" is worthy of consideration just because someone else is looking at something that might be remotely related (again, see #4).
The primary researcher should do a comprehensive review of existing work so as to stumble over black swans if they exist and not be surprised later. However such due diligence doesn't insure a white swan dance. This is what that purple loop at the bottom of the cycle is there for. These black swans are not to be ignored if they crop up! Still, the scientist doesn't set out to find or create them.
4. Good Scientists Stay In Boxes After All
I don't know how else to phrase this other than to point out the strange glorification of the "outside the box" thinkers amongst many who write about science. I suppose it is more exciting in a way. But as it turns out, a lot of mainstream science turns out to be right all along, because that's how it got to be mainstream. Science is cool, for sure, but it's most often not very sexy. If I had a dollar for every amazing discovery over my lifetime that was never heard about again, I'd be fairly well off. Newsflash:
Not every idea is worth further investigation.
When I say staying in a box, I'm not referring to boxing oneself in by clinging to a hypothesis despite stepping in swan poop every other stride and insisting it contains the gut flora composition that only a white swan can have. [See, for example David Ludwig, Richard Feinman, etc.] I am talking staying in the box by remaining in the realm of the plausible and doing the preliminary testing if needed before sharing wildly speculative ideas with the world. Also, if every line of scientific inquiry could be side tracked by some "visionary" exclaiming "what about xyz?", we'd never make any progress. The scientists' mission is not to debunk every hair-brained idea to come down the pike. The vast majority of "settled science" is not in need of debunking.
There appears to be no shortage of scientists to investigate the more out there amongst the unknowns. But guess what? When there's a "there" there, the lone "nut job" doesn't remain alone for very long. If you think you have come up with something (a) new and/or (b) countering what scientific consensus holds, I'm not discarding it out of hand, but I'm not throwing a ticker tape parade either. You get to do the boring humdrum stuff of explaining it to and convincing people that it's worthy of a look before accusing people of being dismissive and disrespectful. There's a threshold there. I can't lay out rules for such a threshold, but it's something intuitive, not unlike that moment when you realize someone has been lying repeatedly to you so you no longer trust them, or someone has finally embraced that one-too-many homeopathic sapien.
If you cannot rise to this threshold ... this challenge ... then you are not a scientist (or a non-scientist applying scientific rigor to a problem) and you cannot expect serious scientists or even lay people to pay attention to you. This holds true even if you are looking back at a long record as an accomplished scientist, though some are able to ride that gravitas into credibility in other realms. This seems especially true in your "softer" sciences where it appears that almost every Tom, Dick and Harry has something to say about something. There's certainly no law prohibiting anyone from doing their own review of the literature and formulating grand mal theo-petheses, but the onus is on whomever is putting this forth to meet the threshold for serious consideration.
Scientists would never get anything done if they were distracted from their focus by every thought that ever crossed someone's mind. Even if that mind has formulated greatness in the past. Linus Pauling serves as a cautionary tale of accepting past successes as proof of validity of new ideas. ◄◄ Great read.
Scientists would never get anything done if they were distracted from their focus by every thought that ever crossed someone's mind. Even if that mind has formulated greatness in the past. Linus Pauling serves as a cautionary tale of accepting past successes as proof of validity of new ideas. ◄◄ Great read.
5. Good Scientists Recognize When To Give it Up
Concluding Thoughts ...
The words science and sensationalism only share four letters in common. I hope I counted correctly. When science is sensational we're in the bonus round, but sensationalism is never a substitute for the real deal.
Science is not just sharing your thoughts, whether they be a few offhand comments, manifestos in the form of journal articles, blog posts or whatnot, or "non-fiction" books. You don't just throw stuff up against the wall to see if it sticks, demand others to admire your artwork, and expect them to clean up your mess.
I have recently discovered the permalink function in Blogger that allows me to generate the link to a post before I publish it. I know ... what took me so long? Here are some posts in the works for which this one is a backgrounder in some part or another. In no particular order, though this may likely reflect publishing schedule, these links go live when published:
Comments
Post a Comment
Comment Moderation is ON ... I will NOT be routinely reviewing or publishing comments at this time..