We live in a world of information-overload, but how are we to know which nutrition studies can be trusted and which can’t? I’m sharing my top tips to critically analyze nutrition information and decode the fads from the facts.
Here’s the thing: if you look hard enough you’ll be able to find research that supports any claim. The difficulty comes in figuring out how credible and applicable that research may actually be. To help you determine whether or not the study you’ve found is enough evidence to alter your lifestyle choices, here are some of the questions you should be asking/answering.
Are the studies peer-reviewed, and in a reputable journal?
Peer-review is a process by which a research team submits their paper to other research professionals not involved in their study. The idea is that this second team won’t have bias, and will be able to critique the study results better than the team involved directly in the research. While not a perfect process, it helps ensure that a study is actually reputable.
How many research studies are on this topic?
This is super important because even the most well-designed study could yield incorrect results. Having multiple studies that back up claims is important before making any big changes. It’s also important to look at if an article mentions a new study that disproves previous recommendations. Check and see how many studies prove that point, and how many disprove it. If you’ve got a lot on the “prove” side and only one on the “disprove” side it warrants more looking into. There are so many things that can influence the results of research which we’ll be talking about here.
What study design was used?
There are many different study designs, including observational studies and experimental studies. Observational, as the name implies, refers to a study where researchers observe the intervention they’re interested in without intervening in the way a person is already interacting with that intervention. Experimental, however, involves researchers introducing an intervention to study participants.
Both types of study design have their pros and cons. In general, however, observational studies form looser conclusions than experimental. This is simply because researchers have less control over all the different variables participants are exposed to. This is especially important in health studies as people are more likely to engage in multiple health-promoting activities. For example, an observational study might find that people who exercise more frequently live longer. Those same exercisers, however, might also drink more water, eat more fruits and vegetables, or go to the doctor more often. There’s no way to pinpoint that the exercise specifically was responsible for the increase in longevity.
Even experimental studies have weaknesses, however, which is why paying attention to all of the variables we’re discussing is important.
Is there a control group?
In experimental studies, a control group is key to see more specifically how an intervention works. Heard of the placebo effect? It’s crazy how much somebody just believing that an intervention will work can influence whether it does or not. To ensure that an intervention is actually doing something you need to compare the group with the intervention with a group without. This helps ensure that there isn’t some other external factors, or placebo involved.
Are the study subjects humans?
Tons of research is performed on animal models for a variety of reasons. That doesn’t mean that the research is wrong, it just means that we need to be careful about making sweeping claims based on how a rat or a mouse responded to a treatment. It should come as no surprise that we’re not mice, and therefore research in animal models should be interpreted cautiously.
How many study participants were involved?
The bigger the number of participants, the more applicable the results, in general. Again, there can be other flaws with the research, but you want to examine the sample size. If you’re looking at research involving a rare condition, a smaller sample size could still provide valuable insight. For general health information, however, you need a large sample size. This ensures that the results are more reliable, and not simply a coincidence.
What are the demographics of the study participants?
There are so many things that make us unique, that can also influence how our body might respond to an intervention. Age, race, sex, gender identity, sexual orientation, education level, socioeconomic status, where we live, and more can all influence results of a study. Pay attention to what types of characteristics the study participants have, and where they came from. If a study was performed in a different country, for example, keep in mind the characteristics of that country that might influence results. Countries with universal healthcare, for example, might yield different results than one that has a lot of folks without health insurance.
What lifestyle characteristics, if any, did the study control for?
Outside of demographics, this is an extremely important thing to look for. Lifestyle characteristics like eating and exercise behaviors, water and alcohol consumption, smoking, etc. can also influence study results. Good studies will control for these variables to ensure that a lifestyle behavior isn’t influencing results. It’s not perfect, but it improves study reliability.
How old is the study?
Now, don’t get me wrong, I have nothing against older studies. There are many that are still applicable today, however, it’s something to keep in mind. If all the studies someone is citing are decades old, for example, check to see what the new research says. Research methods change throughout the years which can influence applicability of a study today.
Did the researchers identify limitations of their research?
There isn’t a single perfect study out there, and researchers know this. This is why they’ll include a section of their study devoted to identifying other variables that could’ve affected their results and areas they’d like to look into more. Read through these to find out where the researchers themselves think weaknesses might be in their study.
At the end of the day, no research is 100% perfect. Hopefully these questions help you figure out whether or not a study is worth paying attention to. What other questions do you have about nutrition research?