Jens Ludwig, an economist at the University of Chicago, studies social policy, particularly in the areas of urban poverty, education, crime, and housing policy. A former RSF Visiting Scholar, Ludwig was the project director for a final assessment of the Moving to Opportunity for Fair Housing Demonstration Program, which was partly funded by the Russell Sage Foundation.
Q: When the interim results of the MTO program were released a few years ago, many social scientists and journalists were surprised by the modest nature of its impact. Some reporters even argued the data proved neighborhood effects don’t matter all that much. Do you think such a pessimistic outlook is warranted?
A: Some social scientists have reacted to the MTO results as providing evidence that "neighborhood effects don't matter much." A different reaction people have had to the MTO findings is to focus on the fact that the difference in average neighborhood environments for families assigned to the mobility treatment groups and the control group have been converging over time, and have dismissed the value of the MTO experiment as not being very informative because the demonstration involved a "weak treatment." I think that both of these sorts of reactions miss some of the key lessons that come out of MTO.
First, it is certainly possible that moving families from very disadvantaged inner-city neighborhoods into neighborhoods that were much more affluent than the destination areas into which most MTO treatment group families moved—very affluent places like, say, the Gold Coast in Chicago, or the Upper East Side in New York City—could have more pronounced impacts on family outcomes than what we saw in MTO. But MTO is about as intensive a housing policy intervention as one could imagine in the current American political climate—in fact, MTO is probably much more intensive than anything any elected official would ever consider really doing. So what we've learned from MTO is that the range of neighborhood conditions that could be modified by feasible social policies doesn't necessarily have dramatic impacts on every outcome we care about, at least for the sort of very disadvantaged household of the type that enrolled in MTO.
Second, it is useful to reflect on how much more neighborhood change could be induced by an even more intensive housing policy, even if we did not face political constraints. Families that moved through the MTO low-poverty voucher treatment were living in census tracts that on average contained about 20% poor families. Many people have looked at that figure and thought, "20% poor is still a lot of poor families!" But it is important to keep in mind that the latest poverty statistics for the U.S. show that around 15% of all Americans are poor right now. A common measure of residential economic segregation is based on asking, what fraction of poor families would we need to move to a new neighborhood in order to achieve perfect integration -- defined as a situation in which the fraction poor in each neighborhood was exactly the same. In a country in which 15% of all people are poor, perfect poverty integration would be defined by a world in which each neighborhood had 15% of residents below the poverty line -- not so far off from what we saw with the MTO low-poverty group families who actually moved with a voucher. With a 15% poverty rate there is just not that much room to do much better at large scale than what we got with MTO.
Third, most people have focused on the fact that MTO did not change each and every single outcome domain that was examined. But the comparison tested by MTO was to compare the effects of living in public housing versus receiving a low-poverty housing voucher. Most housing economists think that the costs of those two ways of providing housing services to poor families are about the same—some economists I talk to, like Ed Olsen at Virginia, claim that vouchers are actually much cheaper than public housing, particularly when one focuses on the costs of providing families with a given quality of housing. So we're talking about a randomized experimental test of an intervention that has very modest costs—perhaps even negative costs. One does not need to have benefits on very many outcomes in order for a policy intervention with modest costs to pass a benefit-cost test! It's true that the most important potential cost of MTO—the consequences for other families outside of the demonstration study sample, namely those living in origin and destination neighborhoods—are currently not known. But the key point is that lack of MTO impacts on some outcome domains doesn't mean that feasible ways of changing neighborhood environments for families might not generate benefits to society that are large enough to justify the costs and effort.
Q: Your recent study in the New England Journal of Medicine found some important reductions in obesity and diabetes among women who had used the MTO program to move to a less-poor neighborhood. What are the policy implications for health care design in America?
A: The main lesson from our NEJM paper for me has been that health depends on a lot more than just the health care system. There are always some difficulties comparing effects of different programs across studies, because they enroll different types of study samples and construct outcome measures in slightly different ways. But with that qualification in mind, the impacts that we see from moving to a high-poverty to low-poverty neighborhood in MTO on diabetes seems to be about the same size as what we see from medical interventions that are explicitly designed to reduce diabetes risk, such as best-practice lifestyle interventions or even medication. It is possible that investing in community environments to prevent extreme obesity and diabetes could be a very cost-effective complement to what happens within the medical care system for addressing these problems.
Q: Do you have any theories to explain the MTO’s lack of detectable impacts on education, employment and income?
A: One possibility—and this is just a conjecture—is that outcomes like education and employment may require lots of different things to be in place in order to change for people. For example, in order for children's test scores to improve, we need the child to be healthy and motivated to participate in school, we need parents to be supportive of education, we need teachers and classroom environments to be educationally productive, and we need neighborhoods to be safe enough so that children are willing to go to school regularly and are able to focus on what happens in the classroom. MTO might change some of these things, but perhaps not enough to move the needle on test scores. By contrast outcomes like obesity, we might need fewer things to be in place in order for outcomes to change.
Q: What has the experience of studying MTO taught you about evaluating social programs? In a recent article on randomized trials, you cite Peter H. Rossi’s Iron Law of Evaluation: "The expected value of any net impact assessment of any large scale social program is zero." Can we improve policy evaluation literature?
A: Despite the fact that I look like I am about 17, I have been working on MTO for the past 16 years and have found it to be a very intellectually humbling experience. The world is a much more complicated place than our social science theories would lead us to expect, and it turns out to be extremely challenging for social policy to make major changes in so many of our most important social problems. Recently Jeffrey Kling, Sendhil Mullainathan and I published a paper in the Journal of Economic Perspectives (summer 2011) arguing that one way we can make existing research dollars as useful as possible is to do more of what we call "mechanism experiments." In some cases it might be possible to carry out a relatively low-cost test to see whether some mechanism that is hypothesized to link some policy to some outcome of societal importance is really very important in practice, as a way to screen out those policies that don't seem worth testing at large scale (and great expense) and focusing our scarce resources on testing only the most promising policies. The larger idea is the view that to generate information that is useful for policy; our experiments don't always need to test things that look exactly like the policies that are of direct policy concern.
Q: Now that the final impacts report on the MTO has been released, where do you think neighborhood effects research should go next? And do you think experiments have become, as Robert Sampson once wrote, a "mantra" in the field of neighborhood effects? Should more social scientists in this area engage in descriptive research?
A: I think the optimal research portfolio for making progress on these sorts of very complicated social policy problems would have to include some mix of descriptive research and randomized experiments. Without the descriptive research, how would we ever know what to test in our experiments? On the other hand, without any experiments at all, we would have a very difficult time knowing what policies or programs could actually help make a difference. I spent the 2010-11 academic year visiting the Russell Sage Foundation as a visiting scholar and had the great treat and privilege of being in the office next to Rob Sampson's. I like to think that the way we will make progress in this area is for people like Rob to do the great work that he does alongside those of us who do experiments continuing to do our thing. I am sure that Rob would agree with me when I say that both of our efforts would surely be more likely to succeed if Russell Sage were willing to invite us back for another year in New York City to visit the foundation; we would definitely not decline such an invitation!