RSF Trustee Mario Luis Small on the Importance of Qualitative Research
Mario Luis Small
The 2016 presidential election was characterized by a chaotic and unpredictable news cycle, a glut of social media misinformation, and profound political polarization. As a result, our political climate today is often described as one of “post-truth.” In an effort to combat the proliferation of propaganda and partisanship, many journalists and publications have reaffirmed the importance of rigorous fact-checking, in-depth research, and accurate citation—all of which are essential practices within the social sciences. How might journalists and readers alike use the methodologies of sociologists, demographers, psychologists, economists and other scholars to better understand our world today?
In a recent lecture titled “Rhetoric and Evidence in a Polarized Society,” sociologist and RSF trustee Mario Luis Small (Harvard University) points out that over the last twenty years, there has been a marked increase in media outlets that knowledgeably draw from or rely upon quantitative social science research to interpret current events for a general audience. These range from “data journalism” sites such as 538 to verticals like The Upshot in the New York Times. As a result of the rise of quantitative literacy among journalists, Small writes, “today it is much harder than a few decades ago to publish op-eds in the Washington Post or the Times that make assertions either contrary to easily available quantitative evidence or else implausible by quantitative standards of evidence.”
However, Small argues that there has not been a corresponding increase in qualitative literacy among the media and the public. “While people understand the limits to their quantitative literacy,” he writes, “they do not necessarily know they lack qualitative literacy, because they are not necessarily exposed to the language with which to arrive at that recognition.” In a new interview with the foundation below, Small summarizes some of the points from his lecture and discusses the importance of qualitative literacy in a deeply polarized society.
Download and read his entire lecture here.
Q. How do you define qualitative literacy, and what are some common misperceptions (or misuses) of qualitative social science research in journalism today?
Small: In my lecture I define qualitative literacy as “the ability to understand, handle, and properly interpret qualitative evidence.” Qualitative evidence is the kind typically marshalled in ethnographic or interview-based projects.
Ethnographic data are the kind collected by ethnographers—people who go out into a community in the field for months or years at a time while writing down what they see or hear. If Nate Silver’s data are the results of polls and surveys, an ethnographer’s data are the field notes. Ethnographic data are the kind that Matthew Desmond reported in his book Evicted (2016).
Interview data are the kind that people collect when they go out and ask people over the course of long and open-ended interviews what they believe or feel about some issue. If an ethnographer’s data are the fieldnotes, then an interviewer’s data are the recorded transcripts. Interview data are the kind that Kathryn Edin and Maria Kefalas provided in their book Promises I Can Keep (2011).
A big misconception is that idea that while collecting and handling quantitative data requires special skills, collecting and handling qualitative data requires no such skills, or that the latter requires nothing but good critical skills. Consider the following: How would you distinguish an empirically strong ethnographic study from an empirically weak one that happens to be very well-written? If you are not sure, then that fact should tell you something about your level of qualitative literacy. It is important to be clear, though, that like any form of literacy qualitative literacy can be learned by anyone.
Q. In your view, what are the primary indicators of an empirically strong qualitative social science study, and how can journalists and readers learn to recognize them?
There are many such indicators, far more than could be summarized in a pithy statement. Moreover, mastering them, as in any form of knowledge, requires study and practice. But I can point to a few of them that are notable in qualitatively strong pieces of either scholarship or journalism.
One of them is the evidence of cognitive empathy, the ability to understand another person's predicament from their perspective. Many beginning researchers and journalists confuse empathy with sympathy, which is the feeling of sorrow or pity people have for other's conditions. Such authors often produce highly sympathetic narratives—stories that elicit pity or another emotional response—that nonetheless do not make clear why those studied understand their circumstances as they do. Accounts of disadvantaged communities often have this problem.
Another common practice among experienced journalists is their attentiveness to what psychologists have called outgroup heterogeneity bias, which is the tendency people have to think of their own group as highly diverse or heterogeneous, while thinking of other groups as homogeneous. During the 2016 presidential election, there was strong evidence of this bias, as many coastal educated journalists depicted the large number of supporters of Donald Trump under a few simplifying stereotypes.
Q. How does greater qualitative literacy help us better understand today’s deeply polarized political climate?
Qualitative literacy, like quantitative literacy, helps individuals effectively assess claims against evidence and distinguish fact from opinion. In our current era, people seem more and more likely to get their news from partisan sources already attuned to their political opinions. In such contexts, it is especially important for conscientious citizens to consistently question what they read and how they are forming their own opinions.