“Truth-seeking social policymakers and serious students of social policy and social policy research will read and reread this book. Fighting for Reliable Evidence does not provide a permanent truth but it describes a forty-five year quest to understand what works and what does not work.”
—PAUL O’NEILL, former U.S. Secretary of the Treasury
“Fighting for Reliable Evidence is a fascinating review of the development of the commitment to use randomized experiments for learning about the effectiveness of current and proposed social welfare programs. It is written from two necessary perspectives: that of a pioneering government official striving to institutionalize experiments, and that of an innovator committed to developing an organization able to implement high-quality experiments. Between them, Judy Gueron and Howard Rolston demonstrate how feasible and useful experiments can be and have been. The history they tell is riveting, and the lessons they draw are compelling. No one else from inside the community of random assignment pioneers could have told it better. One day historians of science will use different assumptions to tell their version of the history of random assignment in the social sciences. But until then, this accessibly written book is THE history.”
—THOMAS D. COOK, Joan and Sarepta Harrison Chair of Ethics and Justice and Professor of Sociology, Psychology, and Education and Social Policy, Northwestern University
Once primarily used in medical clinical trials, random assignment experimentation is now accepted among social scientists across a broad range of disciplines. The technique has been used in social experiments to evaluate a variety of programs, from microfinance and welfare reform to housing vouchers and teaching methods. How did randomized experiments move beyond medicine and into the social sciences, and can they be used effectively to evaluate complex social problems? Fighting for Reliable Evidence provides an absorbing historical account of the characters and controversies that have propelled the wider use of random assignment in social policy research over the past forty years.
Drawing from their extensive experience evaluating welfare reform programs, noted scholar practitioners Judith M. Gueron and Howard Rolston portray randomized experiments as a vital research tool to assess the impact of social policy. In a random assignment experiment, participants are sorted into either a treatment group that participates in a particular program, or a control group that does not. Because the groups are randomly selected, they do not differ from one another systematically. Therefore any subsequent differences between the groups can be attributed to the influence of the program or policy. The theory is elegant and persuasive, but many scholars worry that such an experiment is too difficult or expensive to implement in the real world. Can a control group be truly insulated from the treatment policy? Would staffers comply with the random allocation of participants? Would the findings matter?
Fighting for Reliable Evidence recounts the experiments that helped answer these questions, starting with the income maintenance experiments and the Supported Work project in the 1960s and 1970s. Gueron and Rolston argue that a crucial turning point came during the 1980s, when Congress allowed states to experiment with welfare programs and foundations, states, and the federal government funded larger randomized trials to assess the impact of these reforms. As they trace these historical shifts, Gueron and Rolston discuss the ways that strategies for resolving theoretical and practical problems were developed, and they highlight the strict conditions required to execute a randomized experiment successfully. What emerges is a nuanced portrait of the potential and limitations of social experiments to advance empirical knowledge.
Weaving history, data analysis and personal experience, Fighting for Reliable Evidence offers valuable lessons for researchers, policymakers, funders, and informed citizens interested in isolating the effect of policy initiatives. It is an essential primer on welfare policy, causal inference, and experimental designs.
JUDITH M. GUERON is scholar in residence and President Emerita at MDRC.
HOWARD ROLSTON is principal associate at Abt Associates.