Editor’s note: On September 28, 2017, the U.S. Agency for International Development hosted “Evidence Day,” as part of Global Innovation Week. The Hewlett Foundation was one of several sponsors for the event, which drew approximately 1,700 participants from across government, academia, policy research organizations and social enterprises. Following are remarks offered at the closing session by Ruth Levine, our director of Global Development and Population Program. View video of her talk at the event.
Wow – what a day in Washington, D.C., in the beautiful Ronald Reagan Building: Evidence. Counterfactuals. Integrating monitoring into decision making. Investment in research. Ideas to help and heal the world’s poor. Bipartisan mandate for more accountable public policy. All this and it’s still just fiscal year 2017.
Now that we’re coming to the end of a wonderful day filled with rich discussions, I’m thinking it might be a good time to make some definitive statements about whether randomized control trials are in fact the ideal way to generate useful knowledge in international development.
Just kidding.
What I really think it might be a good idea to do is share some thoughts not about methods but about morals.
It’s an unusual topic for me to take up, since I’m neither a philosopher nor clergy person; and it’s an unusual topic to hear about in a setting like this, because the narrative around evidence-informed policymaking tends to ring the bells of rationality more than morality. But no field can be sustained and advance unless it confronts and articulates its moral core – and that’s something I haven’t yet heard today. I want to start thinking about how evidence-informed policymaking answers a call to be our best selves, individually and as a society.
If I were to ask each of you what draws you to work on ensuring that the most valid and reliable information is available for public policymaking, and used by decision makers, I’d hear a range of answers. Maybe: “I want to make sure that tax dollars are spent efficiently.” Or “I want to make sure that policymakers know what problems affect citizens, and what the impact of different policies might be.” Or “My graduate advisor was a ‘randomista’ so I am, too.”
I think if we talked a little longer, though, deeper meaning would emerge.
You might not think about it every day, with every program you monitor or regression you run, but evidence-informed policymaking is about truth, justice, equality, creativity, and love of others.
These are deep topics and we’re getting toward the end of our time together, so I’m just going to touch on a few of the strong links between the work you’ve heard about today and this moral core. I’m pretty sure you will be able to improve on these ideas in the future.
First, the value of truth. Across all cultures, and certainly in our own, the value of telling the truth is upheld over the sin of lying. One of the main tasks that parents have is instilling in their children a feeling of self-love when they tell the truth, even when it is difficult, and the pain of a guilty conscience when they tell a falsehood, even when – or especially when – it is self-serving.
Well, the work of data scientists, evaluators and researchers – the people who engage in generating the information and evidence for program design and policymaking – is fundamentally about revealing truth. Often the truths are specific, not universal: This program was supposed to improve nutritional status a lot but it only improved it a little – or maybe not at all. Or, this change in policy will leave millions of people without basic health care. They may be specific and sometimes even small, but these are intrinsically important because they are part of an aspiration that we be truthful in public affairs — that true, verifiable, reproducible information is shining a light to frighten off the purveyors of falsehoods and ignorance.
But this work is not only about truth. It’s also about justice.
The easiest link to make between the work of evidence-informed policymaking and justice falls into the domain of distributive justice – that is, the just allocation of resources in a society.
Let me tell a little story.
Once, years ago, I was on a panel with Dr. Paul Farmer. What originally had been billed as a panel global health policy rapidly became a debate about whether cost-effectiveness analysis is or is not a tool of neocolonial oppression and injustice. I was tapped to argue for cost-effectiveness analysis. Dr. Farmer took up the counterargument, saying that cost-effectiveness analysis was simply a convenient excuse to deny health care to poor people.
Though I was baffled then by the framing of “thinking about costs” versus “believing in justice,” I now think what happened in that debate is part of a larger failing in the “evidence community.” We do not state, affirmatively, clearly, and repeatedly, that what we’re doing is part and parcel of a millennia-long fight for justice – distributive justice.
We do not claim the moral high ground. It is always on the moral high ground where most people want to stand, so they will stand with us if they see us there. In fact, all too often we cede the space altogether, uncomfortable or unfamiliar with arguments centered on justice and rights.
We need to claim our space.
There are a couple of distinct elements to consider. One important part is the emergence of values – what constitutes “just allocation?” As a society we are routinely asked to make trade-offs between conditions that are not intrinsically comparable: the well-being of a baby versus an elderly person, the welfare of those of us living today versus our descendants two generations hence. We have to make these choices around which there are no objective guides or guardrails, and individuals and groups draw on profound beliefs, whether from religion, historical precedent, philosophical tenets, or other sources.
In my debate with Dr. Farmer, the social choice, or value, he was promoting was that no matter where a person lives or what health conditions exist, that person should receive health care that maximizes length and quality of life. That is a truly noble aspiration and one that I imagine many of us would associate ourselves with even though there cannot be experimental findings to back it up.
But values alone are not enough to achieve distributive justice – and that’s where the evidence comes in. Fairness can be achieved only if full and unbiased information is available about current conditions, and about the costs and benefits of one way of acting – one policy option – versus another. Yes, we could guess or assume. But when measurement is possible, guesses and assumptions are for the lazy and the irresponsible, not for the people most dedicated to a just outcome.
What many in this room, and empirical analysts around the world, do every day is provide those essential ingredients that make a quest for distributive justice real, offering the necessary information that permits those who are pursuing social choices to know what they are actually doing and how to do it better.
So, for instance, if we want to pursue the goal of good health for all, regardless of geography or wealth, we need a lot of information. We need not only the facts about what drugs safely and effectively treat what health conditions, but also how each dollar spent on treating a particular ailment, or a given patient population, can be used to obtain the maximum benefit in terms of longevity and quality of life. In other words, we need cost-effectiveness analysis – as well as the work we’ve heard about today, such as estimating the net effect of a given intervention, or testing innovative means of advancing human and environmental health and well-being – all can be used in service of this powerful aspiration around global equity.
Back again to the discussion with Dr. Farmer and others who critique cost-effectiveness analyses. I came to understand that at least some critics don’t in fact object to optimizing the use of resources, and using information to do it. The objection is to the existence of arbitrarily determined cost-effectiveness thresholds. Some health economists have stated that interventions are cost-ineffective if they cost more than three times GDP per capita per disability-adjusted life year. And it’s absolutely correct that establishing that threshold was not an empirical question, but a values-laden one.
This story has a simple point: Empirical analysis is not a substitute for the value judgments that inform a theory of justice in any society. But empirical analysis is an essential complement to those value judgments, helping to turn the “what we believe” into the “what we do.”
Beyond the values of truth and justice, evidence-informed policymaking helps to realize the value of equality. The moral value of equality is codified in many parts of U.S. constitutional and civil law – one person, one vote; equal protection under the law; equal justice under law. Whether within a legal and regulatory framework or not, there’s a widespread aspiration in the United States – if not a fully realized experience – of each person being recognized as having the same rights as each other person. Bill and Melinda Gates say that they are “guided by the belief that every life has equal value.” And, of course, the human rights conventions ratified by the member states of the United Nations affirm this concept in many ways.
Your work in the field of evidence-informed policymaking has a very specific and particular role to play in advancing this ideal. And it’s found in the work on data.
If it weren’t for efforts to collect the same type of information from each and every individual – or at least a representative sample of each and every type of individual – what would we know? We would only know about the lives, livelihoods, and opinions of the people who have the greatest access to the public square. We rarely think about it this way, but the data collected through household surveys and other means are fundamental to ensuring that public policymakers hear from poor and marginalized people, and have ways to understand what they’re experiencing, whether it’s displacement after conflict and natural disasters or lack of access to quality education or intimate partner violence. Good quality data doesn’t replace the need to create spaces for people to speak for themselves, but it contributes to a deeper understanding of how people live.
There is a tremendous opportunity before us, as we engage in today’s data revolution, to make sure that our work lives up to its potential to put each person’s experience on an equal footing with all other people. That means making sure we are including out-of-school children as well as children in school when learning is assessed. It means asking both married and unmarried women about their sexual and reproductive experiences. It means dedicating sufficient resources to count homeless and undocumented residents in the 2020 population census. It means asking women questions not just about their childbearing but about their work, and men questions not just about their work but about their fatherhood.
Finally, I want to talk about the contributions that evidence-informed policymaking can make in service of the value of human progress. On any given day, human beings have choices to make about whether we maintain the status quo, doing things as we have always done them, or we work to advance, perfect, optimize, and seek a better form of ourselves as individuals and our societies as collectives. Virtually all religions encourage or mandate ever-purer and more loving thoughts and behavior, whether for benefit in this life or the next one. The size of the self-help industry alone speaks to the impulse toward continuous improvement; businesses are judged on the basis of year-on-year growth; and nation-states are constantly trying to grow their economies, improve the health and well-being of their citizens, and sometimes expand their global power through military means. I think it’s safe to say that, in general, human beings seek to create better lives and better societies.
It is no stretch to say that this value – the value of human progress – is amplified and served by the work of people in the evidence community. We have to answer questions like, “As a society, where are we now?” That is a question that can be answered fully only by assembling facts in an organized way. Guessing will lead us astray. We have to answer questions like, “How effective are we now in solving specific problems?” Careful, thoughtful evaluation serves us, while wishful thinking does not. We have to answer questions like, “What are other approaches that might work better?” Innovation, based on reason but spiced with creativity, gives us the seed from which improvements grow. We’ve seen examples of all of this today, striving for more, better, stronger, and more lasting.
What you’ve heard here is what I hope will be a starting point for a rich conversation about the ways in which the practices and tools of evidence-informed policymaking are not separate or opposed to a moral public policy, but are essential to it. We have to lift this conversation above internecine debates about methods, or an insistence that we are hardheaded not softhearted. We have to assert, strongly, persistently that failing to use facts and evidence in decision making about matters of consequence is not only dumb, but wrong – deeply, irretrievably wrong. In contrast, championing the use of information and analysis is responsible, even righteous.
Professionals who know what it is to measure, to monitor, to estimate, to weigh – we have the responsibility each and every day to serve the aspirations of progress, equality, justice and truth.
