Friday, December 30, 2016

Statistically speaking...

Statistics are everywhere these days.  You can't really blame the statisticians for getting all kinds of excited though.  The amount of data being produced today is prodigious to put it mildly.  Personally I say the more the merrier.  I love stats and I love data so I'm not at all put off by it all.  But statistics itself is a field concerned with interpreting data and data is just information and information can be good or bad or a little of both.  True, you have dishonest statisticians and you have dishonest people using honest stats for dishonest purposes.  But you also have bad information that results in bad stats being produced by honest statisticians and used by honest people.

It is sort of like Facebook.  On any given day I can read a lot of true things about friends, family and the world in general.  I have no reason to believe that the folks over at Facebook are anything by good, honest, hard working people.  Nor do I doubt my friends and family have the best of intentions.  But every once in a while I see a post that tells me my favorite celebrity has died when in fact they have not.  The bad information really ruins an otherwise wonderful experience. 

Recently there was a fairly large brouhaha on the Internet when a researcher published her results of a meta study which concluded that organic food had no more nutrition than food that was produced using pesticides and genetic modification.  Everyone got really excited about this.  What most people heard was, "Organic food is no better than other food".  Which really wasn't what the study was saying in the first place.

First, meta studies are tricky to get right.  They are very prone to researcher bias because the researcher is responsible for picking out studies that can best be used to answer a particular question accurately.  The researcher, as the gatekeeper of the information, can easily be swayed by their own personal biases, but also by the organization or company commissioning the research in the first place.  Meta studies can be very useful, but they do have to be taken with an amount of proverbial NaCl.  Too often the researcher chooses studies that answer the question the way he or she thinks it ought to be answered. 

Second, the study didn't really answer the question most people are really asking.  How do I know this?  Because of the reaction to the study.  What people really want to know is if organic food is better for us in general.  Not just if it is nutritionally better for us.  True, people should be concerned if organic food has no nutritional value, or far less value over non-organic food. However, the study only demonstrated that they were both about equal.  But the response, as I mentioned above, was more along the lines of, "Organic food isn't any better for you".

I personally eat organic because I don't want chemicals that are potentially harmful in my food.  So although the study helped answer the question, "Is organic food better nutritionally" (It may or may not be - remember, this is a meta study), it didn't answer the question, "Is organic food better for you overall". 

This is true with all statistics though.  We must ask questions about the statistics being offered.  It is our responsibility to be good and wise consumers of not only food but also information.  We don't have to accept everything we hear and read nor do we have to become fatalists (as some of my friends have).  We simply have to become better.