ACCRA, GHANA: Informal workers gather for the meeting of their association, the Makola Market Traders Union. (Photo credit: Jonathan Torgovnik/Getty Images Reportage)
In July 2017, the Global Development and Population program commissioned a formative evaluation of the Transparency, Participation, and Accountability (TPA) strategy to inform both our grantmaking practices, and related activities that extend our work beyond our grant dollars, such as grantee convenings and our engagement with peer funders. The evaluation consultants, Itad Ltd. worked with the TPA team over the past year to establish a framework for ongoing learning, exchange, and regular feedback to us and our grantees. We hope this process will help all of us to learn, adapt, and ultimately direct our time and resources towards achieving our mutual goals.
We have since produced and shared three sub-strategies and identified many questions (about 30) that we hope to answer through our grantmaking and evaluation work in the coming years. How do we plan to systematically and rigorously interrogate all those questions over the life of our strategy? First, we hope to find some answers by supporting rigorous research, evaluation, and peer learning networks and exchanges under our learning sub-strategy (also in French and Spanish). Second, each year we will prioritize a couple of these questions for Itad to answer as part of the ongoing formative evaluation of our strategy.
This update focuses on the work done between November 2017 and October 2018. We identified the following questions for the consultants to answer:
What are our grantees’ assumptions and expectations about how their interventions will lead to positive change in citizens’ use of information and engagement with duty bearers, and governments’ responsiveness in relation to improving public service delivery?
How well are various approaches to promoting transparency, citizen participation, and accountability working? Why or why not?
Are these approaches working for everyone? How effective are grantee strategies, approaches, or tactics in increasing participation of under-represented/minority groups in governance? Why or why not?
The consultants used an online survey for the first question and we have some results to share with you in this update. The second and third questions involve field visits and in-person interviews with grantees and their main stakeholders – governments and citizens.
Here’s who we hope will use our strategy evaluation results, and how:
The Hewlett Foundation’s Transparency, Participation, and Accountability team: to track whether and to what extent the underlying assumptions of our strategies are holding up or not and how we might adjust. We also want to stay tuned to our grantees experiences in carrying out this work in different contexts.
Our grantees: we hope that by sharing the results of our strategy evaluation, our grantees will gain new insights about the effectiveness of various approaches (theirs and others) and make any necessary adjustments.
Funders, researchers, practitioners, policymakers, and others in the field: by sharing our evaluation results with a broad range of actors, we hope to contribute to advancing knowledge in the field.
We realize this is an ambitious agenda and we will not be able to achieve it without your cooperation, insights, and feedback. We welcome comments and suggestions as the evaluation proceeds.
Results from Online Survey
The survey was sent by Itad to a sample of 44 out of a total of 107 grantees that were in our portfolio at the time. These grantees were selected because they are all organizations that focus on supporting citizens or citizen groups to use information and provoking some type of citizen action, including action by minority groups. Forty-one grantees successfully completed the survey. The sample included: 44 percent international NGOs; 41 percent national NGOs; and 15 percent hybrid (international NGOs with registered country affiliates).
Because the results are based on grantee responses to an online survey, they may be affected by selection and response biases. We expect that Itad’s research on the second and third questions will offer an opportunity to further validate and deepen these initial findings. Here we offer six key findings.
More than 80 percent of the grantees surveyed are implementing interventions that appear to be well-aligned with our core objective of improving public services through citizen participation and oversight. Our portfolio currently includes a mix of grantees that we have supported for many years, as well as new grantees that we’ve added since our strategy was updated almost two years ago and it is encouraging to learn that there is significant alignment.
Most grantees report that they tend to work collaboratively with governments as opposed to being confrontational. One of our key learning questions is to better understand grantees’ choice of strategies and tactics – whether collaborative or confrontational – and under what conditions collaboration or confrontation (or some combination of the two) is most effective. Thus, we will be digging deeper into this finding in future evaluation work!
More than half of our grantees promote some form of budget transparency and accountability. This is not surprising given our past and continued investments in supporting organizations working on budget transparency. We are curious to learn, over time, if organizations are adopting these approaches more often because they are particularly effective and essential to improving service delivery, or for other reasons (for example, the availability of tested tools and capable organizations that are able to use them and provide support to others to adopt them).
More than 60 percent of our grantees use global norms and standards as an important reference point in making their case. Again, this is an area which we have supported for many years and it is reassuring that grantees are able to leverage some of these norms and standards in their work. Going forward, we plan to focus more of our grantmaking on supporting country-level implementation of global norms and standards and will be seeking ways to better understand their effect at the country level.
Most of our grantees focus their effort on national-level engagement and policy change. We see this as a potential gap in our grantmaking, given that our ultimate objective is to improve public service delivery through active civic engagement; we think this requires action at the sub-national level, where services are ultimately delivered and citizens are able to interface more directly, as well as the national-level, where policy and resource decisions have an effect on priorities and how they get implemented.
The majority of our grantees are committed to and have high aspirations for learning from their work and the work of others. We are particularly delighted with this finding and will continue to explore how to effectively support grantee learning. We will welcome your feedback on how we can do this more effectively. We would also be interested to know whether grantees have an appetite to help others learn from their work.
