Watch out for preliminary research

I’m feeling a little vindicated here today, kids.

One of my fellow health care writers out there filed a very interesting report yesterday on Covering Health, the blog for the Association of Health Care Journalists. Andrew Van Dam quotes Dr. Steven Novella from the blog Neurologica on the topic of the danger of reading too much too soon into preliminary results from research studies: “Over-reporting of preliminary results also has the effect of confusing the public with lots of noisy information, most of which is not true.”

Dr. Novella was concerned about a poster displayed at the recent American Stroke Association International Stroke Conference that seemed to make the claim that diet soda consumption is linked with a high risk of cardiovascular disease. More specifically the poster claimed that people who reported drinking diet soda had a 61 percent greater risk of experiencing a stroke or other cardiovascular disease than people who reported that they didn’t guzzle down a couple of Diet Pepsis or Coke Zeroes every day. Novella points out a major flaw in a study leaning so heavily on people self-reporting the data in question: “At best such a study could show correlation, but by itself cannot build a convincing case for causation,” he wrote. He went on to add, “Further, we have a one-time self report, rather than reporting on soda intake at several points in time. The data itself is not very reliable.”

In other words, you can’t always count on people to even tell the whole truth when they report certain pieces of information, and you certainly can’t rely on that for the basis of your argument. And a snapshot isn’t always the best way to really get a handle on the reality of a situation.

You hear the same argument made when it comes to evaluating standardized test scores in schools. A standardized test measures how well a student performed on one test on one particular day. But what if the student hadn’t slept well the night before? Or was coming down with a cold? Or had a stomachache? Or had to sit by their archnemesis and couldn’t concentrate? Or ran out of No. 2 pencils?

There are better ways to measure data and to draw conclusions. And Van Dam and Novella both wanted to make the point that it’s important for people who write about research to not put too much emphasis on the preliminary results of research because it may be that the preliminary data isn’t enough–or it isn’t accurate because it’s still, yes, preliminary. That’s a crucial bit of information to remember for those of us who write about health care.

We owe it to our readers to be responsible, after all. Look what happened with the whole fiasco that happened when one doctor did a tiny little study with a handful of children and claimed that certain routine childhood vaccines caused autism. Even before the truth came out about Andrew Wakefield, no one was ever able to repeat his findings. And yet how often did the general public get that message? It was reported, yes, but it got drowned out by a lot of sensationalism and hype over celebrities who embraced shoddy science. We’re just lucky that there were some persistent journalists like Brian Deer out there, plugging away to get the real information out eventually.

My goal is to continue to be thoughful, thorough and analytical when analyzing medical research when writing articles in the future. I’ve seen the damage done by reporting that wasn’t so careful, and I’ve seen the benefits from journalism that was. Also, thanks to Van Dam, I’ve discovered the website Health News Review, which evaluates how well a journalist performs when it comes to writing accurately about health or medicine in an article. It will be a source of inspiration.

And on a personal note….So back to the whole diet soda issue. The next time of my loved ones sends me a well-meaning article about the dangers of diet soda, I will at least take the time to analyze the article and see where the data is coming from. And I’ll take that into consideration. And it’s sketchy, I will just roll my eyes and hit up the fridge for another cold silver can of Diet Coke. Mmmmm, I think I might go have one right now anyway.


About jenniferlarsonwrites

I'm a freelance writer and editor based in Nashville, Tennessee. I have a master's degree in journalism from the University of Maryland and a bachelor's degree in English from Rhodes College. I'm a born-and-bred Southerner who spent a few years in Southern California, a rabid baseball fan and a would-be grower of tomatoes. You can also visit me at LinkedIn or on Twitter at @JenniferLarson.
This entry was posted in Health, Journalism. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s