This is the headline I yearn to see, but it’s obvious this “article” is even more bogus than those emails you get from Nigerian princes. Unfortunately, most Internet articles are not so easy to evaluate for validity. Here’s some “gotchas” I’ve learned about reading scientific research articles.
Let’s take a look at a real article, headlined “Living near major roads linked to risk of dementia, Parkinson’s, Alzheimer’s and MS”.
Headline: Notice in the bogus article there is a definite verb: “discover”. You will never (well, hardly ever) see a definite verb in a real headline. It’s just prudent journalism to caveat the headline, e.g., “linked to risk” in the real article. The drug approval process is so long and complex that in real life, you will never have a dude in a white lab coat whipping out the test tube and shouting “Eureka! I have discovered a Parkinson’s cure!”
Where: Where is the research published? If you are finding your research in (I’m making this up) Voodoo Science Digest or Oprah Talks about Movement Disorders , validity would be highly suspect. In the case of this article, it was “published this week in the journal Environmental Health.” Having research published in an academic journal like this is the gold standard — since articles are typically peer-reviewed in scholarly journals.
Reasonableness: Obviously the bogus article flunks this test – there aren’t even real words in the body of the article. But to my mind, the real article also flunks this test: Doesn’t nearly the entire global population live “near major roads”? Isn’t this sort of like saying “Wearing shoes linked to risk…”? So what research is this headline based on? Time to look more closely at some other factors.
Who: Who did the research? Was it a university? A research institution? I looked at the article and found the research was done at University of British Columbia by Weiran Yuchi, the study’s lead author and a PhD candidate in the UBC school of population and public health. Hmmm…this sounds a bit weak…researcher doesn’t even have a PhD yet, and research is not coming from a medical school. But nevertheless, UBC is a major institution in my northern neighbor (and oh yes, they have a really swell collection of totem poles). By the way, the across-the-border location is a reminder to us arrogant Americans that perfectly valid and innovative research could (and does) come from anywhere in the world, not just Harvard and Stanford.
How many: Most medical research boils down to identifying trends from a quantity of individual results. This is a bit simplistic, but the bigger the quantity, the more valid the trend. Think about the conclusions you could make after testing 6 people — this quantity is so tiny, you couldn’t identify any sort of trend accurately. In the case of the UBC article, pretty impressive numbers: “Researchers from the University of British Columbia analyzed data for 678,000 adults in Metro Vancouver. ”
Date: How recent is the research? If I am looking up a research subject, I could get Internet articles from the last 20 years – and knowledge is changing rapidly regarding Parkinson’s. The UBC article passes this criteria with flying colors: published 1/23/2020.
Anyone else?: Has anyone else done research in this area? Have they reached the same conclusions? In the case of the UBC article, the underlying hypothesis was that if you lived “near major roads” (as estimated by postal codes), you had more exposure to air pollution. People in this category had more cases of dementia, Parkinson’s, MS. So I Googled “air pollution impact on dementia”. Voila! A 1/4/19 article says: “A new study from the United Kingdom has found that high levels of air pollution are associated with an increased risk of dementia.” And Bingo! Here’s another headline: “Air Pollution and Dementia: A Systematic Review.” “Review” is a hint that there have been enough different studies that they can be reviewed to see if they come to the same conclusion. Yep: The authors reviewed 13 studies from all over the world and concluded: “Evidence is emerging that greater exposure to airborne pollutants is associated with increased risk of dementia.” As a bonus, note this review came from NIH, (US) National Institute of Health – about as scholarly as you can get.
More research: If the article does not include a phrase like “more research is needed”, you should be suspicious. Sure enough, the UBC article says, “[Researchers] are hopeful the larger dataset will provide more information …..”
Sniff test: With perfect timing, my sharp-eyed brother just sent me a video breathlessly headlined “Miracle surgery stops Parkinson’s sufferer from shaking” and asked “Is this legit?” Obviously, this video didn’t pass the sniff test for my rational engineer brother. When I looked over the video, I found that the surgery (described as “pioneering”) was a pallidotomy, a surgery that’s been done for PD for at least 20 years, and which (unlike the implication in the article/video) only resolves Parkinson’s symptoms for a few years – it does not cure the disease. So – repeat after me – “if it sounds too good to be true….it probably is.”