Do pollsters invent religion?

For a sociologist of religion like me, the cover of Robert Wuthnow’s Inventing American Religion grabs at least as much attention as would the cover of a book by a church organist promising to give the lowdown on sex in the choir loft. The title calls out those who conduct polls and surveys on American religion and accuses them of inventing what they claim to measure.

Wuthnow began his distinguished career as an analyst of survey research on religion and later served on the boards of two of the most prominent organizations in polling (Gallup) and survey research (the General Social Survey), so he is privy to much of the story. Early chapters open with portentous vignettes of important scholars doing obscure things long ago in scattered locations (Chicago 1904, Prince­ton 1941, New York 1951). For the first hundred pages, I thought I could hear timpani and rolling pianissimo in the background: I was prepared for Robert Wuth­now, holder of an endowed chair at Princeton and author of over 30 books and countless articles on religion, to set off an explosion.

Subscribe

Read our latest issue or browse back issues.

The blast never comes. The book contains valuable information on the history of surveys of religion, including portraits of George Gallup Jr. and George Barna, long the two most influential pollsters of religion, and an account of the recent rise to prominence of the Pew Research Center. Wuthnow explains why polls and surveys on religion are both necessary and problematic. Administrative rulings prohibit questions about religion in the decennial U.S. Census. And unlike political polls, where elections serve as real-world events against which predictions can be tested, polls and surveys on, say, the frequency of prayer and attitudes toward the Bible, refer to behaviors and dispositions known only to God.

It is not until chapter six that Wuth­now broaches the most serious issue for researchers: the declining rate of response to polls and surveys. Ironically, having raised it and detailed its extent, he does not zero in on what needs to be corrected in reports on polls or explain what we can reasonably rely on for our understanding of American religion.

Concerning himself with both surveys and polls, Wuthnow tries to reserve the term survey for high-quality and academic not-for-profit research, the best of which aspires to high rates of response from individuals chosen through random sampling and presented with pretested questions. Polls are typically less expensive and often fielded by for-profit organizations. In the past, polls were often based on “quota samples,” people chosen by the interviewer to fit preset categories, but in recent years polls have relied primarily on samples achieved through random-digit dialing (RDD). Polls are also more likely than surveys to be used to tap responses to items in the news. As the book proceeds, pollster becomes a term of derision.

Wuthnow seems most upset by the use of polls in the coverage of evangelicalism at the time of Jimmy Carter’s election to the presidency, when Carter was Time’s Man of the Year and Newsweek ran a cover story the week before the election titled “Born Again.” To serve the needs of the media, pollsters sought to gauge the prevalence of what was a strange brand of religion in that historical context. Wuthnow faults the polls for their widely varying estimates of evangelicals’ numbers (20 million or 50 million?), as well as for their lack of consensus on the criteria to be used to identify an evangelical respondent (having had a born-again experience, reading the Bible literally, seeking converts?). He thinks the media would have done better if they had asked advice from the National Association of Evangelicals, which at the time defined its membership largely by denominational adherence, or from social scientists, who at the time were still churning out books on the few thousand adherents of the Hare Krishna movement or the Unification Church.

Having myself stumbled on evangelicals in, of all places, a mainline Pres­byterian church in 1975, I appreciated the pollsters’ attention, and, with appropriate caution, I used their numbers. Wuthnow himself was an early contributor to the sociological literature on evangelicals, and in his Restructuring of American Religion (1988), he influentially explained why denominational labels are no longer a reliable guide to the religious leanings of people in the pews. So I find it hard to understand his ire.

Speaking for myself, studying religion among post-1965 immigrants (too proportionately scarce, geographically scattered, and unevenly fluent in English to show up in the small samples of the gold-standard surveys Wuthnow favors), I was immensely grateful for the information provided in 1990 through an RDD poll of over 100,000 respondents by Barry Kosmin’s National Survey of Religious Identifica­tion. Today the Pew Research Center assiduously tracks the religious leanings of hard-to-reach non-Christian, nonwhite, and non–English-speaking populations, largely through telephone surveys—some using RDD, others targeting specific constituencies.

To mention the Pew Center returns us to the issue of response rates. Wuthnow cites the Pew surveys’ shockingly low typical response rate, but he also draws heavily on Pew research to understand its significance. He recalls that the typical response rate for polls and surveys in the 1980s was 65–75 percent. By the late 1990s, it had fallen to 30–35 percent. Today, when Pew has become the most cited source for quantitative data on religion, its typical rate is 9 percent, and the rate rarely reaches as high as 15 percent. That information, the heart of chapter six, was excerpted in the August issue of First Things and called forth a pointed exchange between Wuthnow and Pew officials on the First Things website. Pew’s response ac­knowledges the low response rate but argues on the basis of internal checks (nonrespondents differ little from respondents) that the information provided by their polls is nonetheless valid. Wuthnow credits Pew’s professionalism but argues that they could both improve their response rate and do a better job of acknowledging the limitations of their data. He offers many suggestions to those ends.

Wuthnow’s engagement with Pew is serious and illuminating. He reanalyzes some of their data, and he seems to concede that its low-response surveys accurately reflect American demography. But he and Pew know that respondents who are easiest to reach are also more likely to be socially involved, and hence to be churchgoers. Thus low response rates bias generalizations about religious dispositions and behaviors.

I wish Wuthnow had used this opportunity to define better what can be learned about American religion from the polls and surveys, supplemented by his and others’ wide-ranging studies. American religion is not mysterious. It is complicated. Polls and surveys give us part of the story.