The Problem with Studies

Everything is a study these days. They are endlessly cited on the news, on social media, by leaders and medical professionals. Data promises a kind of certainty, a shortcut to decisions and ideas. If we can study the issue then we can all agree, it will be like 2+2=4. It’s alluring, given how diverse people are and how complex our world is.

Data does help, on a personal level, especially considering how self-deceptive we can all be. We tell ourselves and others that we go to the gym “about 3 times a week,” and the key word there is about. That’s where the aspirational sneaks in, where there’s the wiggle room between the person we are and the person we want to be. If we check the computer at the YMCA, it actually says we check-in 1.7 times a week. A food journal points out the truth, even if we desperately say “I don’t eat anything all day.” A sleep app on a smartphone points out we sleep 6.3 hours a night on average, not the 7 hours we claim. The data keeps us honest considering how aspirational we can be. Our words often say more about the person we want to be, than the person we are.

But we also know that the larger societal studies are often wrong, contradictory, or biased. We were told for decades that eggs raise human cholesterol, but no one bothered to study it and see that it’s not true. Pharmaceutical studies conducted by drug companies always advocate for the drug; contradictory studies are buried. Estrogen was all the rage for women and osteoporosis, until they decided it was a bad idea. The same thing happened with tobacco research decades ago. Studies favor the expensive, the patented and the new, because there’s no money to research simpler solutions. Doctors are influenced to prescribe new drugs that have only modest benefits over older, generic drugs, or fall into some weird anti-science positionthat resembles anti-vaccination.

Brian Nosek has found that many scientific studies, when repeated, yield different results. Scientists have expectations during their research, and are quick to seize on positive results, as are the journals that publish the studies. The experiments that fail get put away in a drawer. Science promises impartiality and certainty, but some are questioning the way science is done since some of the same studies yield different results.

Big data is here, but it’s a more cynical reality. Politically the data is about motivating us, micro-targeting one’s supporters to insure their vocal participation over others. Economically it’s about influencing us to buy something through various levers, showing us products and services in a certain light, or through repetition. At least the scientific studies tried to create solutions to problems. Big data is just about manipulating us.

It’s no wonder that people are confused and frustrated. But despite the contradictions and frustration, we keep going back to the studies, citing ones we like and ignoring ones we don’t. It’s a bit of a curse, and a shortcut for good thinking. It’s clear that data doesn’t help everything: would we have developed democracy and human rights based on experiments and studies? Ideas and reasoning are more vital than ever, to be able to think clearly and critically about an argument, its points, and its assumptions.

I think it’s especially worrying when decisions and leadership are solely based on studies. Here the promise of data serves the interests of power. It’s sheer ideology to oppress someone while denying responsibility: hey, it’s not my fault, the study told me to do it. If a company claims high satisfaction scores then maybe they can lay off some customer service representatives, as long as the scores don’t go down too much. Everybody and everything wants a survey, and I’m at the point where I just refuse. Why should I help you do your job? If you don’t know why I should want your product or service, or don’t know what to discuss at a leadership session, I don’t see why I should help, especially for free. It’s a weird kind of buy-in that occurs, that if we take the survey then we can’t complain because our ideas were part of the process. But that doesn’t follow at all. There’s no guarantee that my ideas were used, or that they weren’t greatly transformed in the process. If you want my ideas, I might as well lead it, after all.

We need visionaries, people who will hold the line without needing marketing data or political polling. It’s a cold world where politicians make decisions on what is palatable, instead of what is right. Reinhold Niebuhr warned us about this in 1952 in The Irony of American History, where he worried that the rise of social scientists would lead us to confidently trust social programming, to grasp the shortcuts over the hard realities of lived history that is slow and ironic,, “the temptation to become impatient and defiant of the slow and sometimes contradictory processes of history” (p. 134).