I was born January 1, 1961 to a working class family in a small town in the U.S. Midwest—a New Year’s baby. My parents weren’t concerned about whether I would finish high school, let alone the risk that I would graduate unable to read or do math well enough to participate fully in society. But UNESCO’s 2015 EFA Global Monitoring Report tells a different story for many children and their parents, even today. At the current rate of progress, it will be 2072 before all young people in lower and middle income countries achieve literacy.
That’s why, like so many others, I’m applauding the adoption of the Sustainable Development Goals, and especially, the ambitious fourth goal: “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.” The goals’ adoption this week makes me wonder: what will the prospects be for girls and boys born in 2016? How many will have all of the advantages of literacy come 2031? What’s clear is that a real, sustained commitment will be needed to achieve this ambitious goal, and early assessment of learning can play a critical role in ensuring success.
If governments, schools, and communities can deliver on the promise of learning for all children at all stages of their education over the next 15 years, a much higher proportion of these children should enter their adolescent and adult years as readers and critical thinkers. They’ll be capable of joining the workforce of the future, and of accessing and making sense of a variety of news, public health, economic, and scientific information to benefit their families, communities, and societies, or simply reading for leisure. Furthermore, as demonstrated through compelling evidence in UNESCO’s 2014 report “Sustainable Development Begins with Education,” improving learning at all stages of life will contribute to achieving the other sixteen Sustainable Development Goals. To take just two examples: having an educated mother saves young children’s lives, and education is, without question, a real poverty-buster.
If this goal is to be achieved, we’ll also be measuring the progress of the Sustainable Development Goals not by how many kids have attended or completed school, but rather, by how many are truly learning—and measurement of learning will start early and continue often. This means that parents, teachers, and decision-makers who are responsible for ensuring schools meet this challenge will know early on whether a child is acquiring the basic reading and math skills that serve as the foundation for all learning. And they’ll be able to provide further support to children who need it much sooner.
The People’s Action for Learning (PAL) Network—a growing movement of civil society organizations who are implementing citizen-led assessments of learning—has proven that it can be done. Their recent call for an early grade reading indicator as part of the toolkit for measuring progress against the equitable learning goal is critical to ensuring that five or ten years from now, children born in 2016 are not left behind. The citizen-led assessments have devised a methodology that is low-cost, low-stakes, and inclusive. The results can be easily understood and used by teachers, parents, communities, and local leaders to identify learning gaps early so that they can all work together to remedy them.
There are also emerging opportunities for others to learn from the PAL Network’s experience. Already, UNICEF’s Multi-Indicator Cluster Surveys (MICS) is tapping into the experience of the citizen-led assessments to develop new household survey modules that governments could adopt for measuring children’s early learning, as well capturing more data about the actions that families are taking to support learning in their homes and in their communities.
If I don’t make it to 2031 to see what happens, perhaps a 2016 New Year’s baby will open this time capsule and judge whether we’ve all measured up to our promises to her— but hopefully she’ll be too busy reading something more fun or useful to her day-to-day life, fully confident that her future children, regardless of their socio-economic background, will have the same opportunity.
There’s no question that the potential of the ‘data revolution’ first described in the U.N.’s “A World that Counts” report has captured the imagination of the international development community, especially data-wonks and donors concerned with how the Post-2015 Sustainable Development Goals will be measured. In the many discussions that have ensued, a consistent theme has been an aspiration to realize the potential of ‘non-traditional data sources.’ Non-traditional, for these revolutionaries, means not just the oft-cited Big Data, but forms of citizen-generated information that can shed light on real-world living conditions and public opinion.
Case in point—Senegal. In February, a team at Université Cheikh Anta Diop’s Laboratoire de Recherche sur les Transformations Économique et Sociales in Dakar revealed the results of their 2014 national assessment of children’s learning known as Jàngandoo (which means “learn together”)—part of a growing movement of civil society organizations who are carrying out independent, citizen-led efforts to measure basic learning of children ages six to fourteen in reading and Math in nine countries in East and West Africa, South Asia and Mexico. (Les resultats en francais).
Jàngandoo has only been up and running for three years, but policymakers in Senegal have taken notice. M. Serigne Mbaye Thiam, Senegal’s Minister of Education and no stranger to open data efforts, opened the national dissemination event, noting Jàngandoo’s contribution to providing policy makers and ordinary citizens timely data on the status of children’s learning that is easy to understand and interpret so that they can plan together actions for improving the quality of education. Since then, Jàngandoo has been rolling out regional dissemination events that have opened up a dialogue between citizen’s groups and local education officials about why children aren’t learning and what can be done about it.
Like other citizen-led assessments of learning, Jàngandoo takes place in homes rather than at school, and therefore captures data on children who do not attend regularly, have dropped out or never attended school, measuring learning earlier and more broadly than official national or regional assessments, which typically take place later in the primary cycle. Jàngandoo’s learning assessments are administered in multiple languages—French, Wolof, Pulaar and Arabic, depending on the child’s home language. According to UNESCO’s EFA Global Monitoring Report 2015, Jàngandoo is the only national-level assessment that has measured Senegalese children’s learning levels below Grade 9 in almost 10 years. The most recent PASEC, which measured children’s performance at the end of class 2 and class 5 in eight Francophone countries, including Senegal, is from 2007.
Unlike other citizen-led assessments, Jàngandoo also measures children’s knowledge of their own culture and environment, including certain aspects of sustainable development. Such locally-driven experimentation can also contribute to how UNESCO and others think about localizing how education for sustainable development gets measured in West Africa, a region where the effects of climate change and drought are an ever-present reality in nearly every child’s life.
Jàngandoo is an initiative of the Université Cheikh Anta Diop’s Laboratoire de Recherche sur les Transformations Économique et Sociales (Laboratory for Research on Economic and Social Transformation—LARTES). LARTES works with a network of twelve civil society organizations and a local technology firm that deploy a cadre of trained data gatherers armed with PDAs to reach all 45 departments of Senegal—all of whom make up what is known as Jàngandoo.
What’s striking about LARTES is that its staff is comprised mainly of young men and women representing a new breed of independent researchers in West Africa. If there is a data revolution, these are the revolutionaries. What’s their latest innovation? Ask Professor Abdou Fall, Jàngandoo’s leader, about his vision for on demand, a la carte data services for local governments who aspire to develop data-informed plans for improving their schools. Their first client: the Mayor of Dakar.
All this takes coordination and careful oversight, but also a particular vision for how data can be used to solve real development challenges. The sort of vision set out by the Africa Data Consensus for bringing different data communities together to generate “data for public good and inclusive development”.
While the global-level conversations about the data revolution are creating needed energy and networks, a cadre of data revolutionaries in civil society and tucked away in government agencies is also hard at work. Their progress often depends upon external funding that runs out before they can institutionalize their efforts. Perhaps what is needed then is a revolution in financing that will nurture the work of country-level data heroes like the LARTES team at Université Cheikh Anta Diop in Senegal. People who think differently about “data for whom,” “data for what” and the speed at which data can be made useable to change people’s lives for the better.
Despite the growing number of children in developing countries that are attending school, they are not reaching the levels of numeracy and literacy that they should. Some have called this a “learning crisis.” According to Rakesh Rajani, Head of Twaweza, previously “it [was] very hard to make that case and get policymakers to pay attention because we didn’t have data, we didn’t have evidence.” But now we do.
Now, civil society organizations across the world from East Africa to Southeast Asia are empowering communities to collect and use this data to demand better results for their children through citizen-led assessments. In eight countries, these groups are conducting household surveys to better measure all children’s learning, regardless of their school status.
The video below, produced by Made in Africa TV in Tanzania, describes the program and illustrates both the need for collecting household-level data on children’s learning, as well as the opportunities for better policies and action presented by this innovative approach.
The Hewlett Foundation announced a pledge of $18.5 million to the Global Partnership for Education’s 2015-18 replenishmenttoday. The pledge, which will be fulfilled through the Foundation’s grantmaking and technical support to civil society organizations and other key actors in global education, will support capacity building for better systems for learning assessment and the use of assessment data to inform planning and improve learning outcomes. It will also support civil society organizations that are using citizen-led, household assessments of learning to raise awareness about children’s learning status and to motivate action at the national, sub-national and community level for improved learning.
Last month, my colleague Dana Schmidt wrote a blog post about what the Hewlett Foundation and its grantees have learned about improving children’s early learning from the Quality Education in Developing Countries Initiative. Under this Initiative, our grantees implemented a variety of instructional models both within the school day and after school hours, with children enrolled and with those who were not. Many of these were evaluated using randomized control studies or other quasi-experimental evaluation designs to determine the impact of these interventions on children’s learning.
At the Hewlett Foundation, we encourage our grantees and also try ourselves to talk about and learn from failure, or at least from those things that don’t always go according to plan! As the (relatively) new kid on the block with the Global Development & Population team, I thought I would offer a few observations about what we learned about the evaluation process itself, recognizing that hindsight is 20-20 (or some approximation thereof). Most of this wisdom comes directly from conversations with our grantees and colleagues since I’ve joined the Foundation.
We underestimated the time that was needed for some of the instructional models to be more fully developed, and in hindsight, should have allowed our grantees more time to work the kinks out before carrying out some randomized control evaluations. In some cases, randomized control study designs were just not possible, and so we had to be flexible about evaluation methods. One of the smartest things we did was to encourage grantees and evaluators to work closely together—so evaluators could better understand the instructional model and the context, and grantees had input in framing the questions and helping determine how best to measure learning outcomes.
While the randomized control studies that we commissioned were able to measure the impacts of these instructional models on learning, they were not able to sufficiently unpack the effect or most essential elements of each instructional model. Practically speaking, this has meant that it has been difficult to tell which elements contribute most to learning improvements and thus are highest priority for scaling up, and which could be dropped or emphasized less, depending on resource availability. Despite these challenges, the Meta-Analysis by Patrick McEwan of Wellesley College goes a long way towards unpacking what is known about improving learning based on a review of dozens of randomized control studies conducted over the past 20 years or so. We think it is essential reading. We will continue to work with our grantees to help them better unpack the elements of their instructional models wherever possible; we encourage others to take this essential step before initiating randomized control studies in the future.
We also underestimated the challenges associated with completing cost analyses of these instructional models. But where we did succeed in doing so, it has produced valuable information that our grantees and policy makers can use for identifying possible areas for streamlining or improving efficiencies that will enable scale-up (e.g. rethinking or restructuring teacher training, mentoring and support, and options for getting instructional and reading materials into the hands of teachers and children most affordably).
Finally, we learned that it is especially important to spend more time and energy figuring out possible delivery channels and constraints to scale-up from the outset. We assumed that we and our grantees would be able to build ownership and political will for scaling up based on evaluation results coupled with brokering new partnerships and financing relationships. We did not fully appreciate all of the systemic barriers to change in environments where incentives and accountability are not currently structured around learning outcomes. We did not sufficiently plan how to manage the discontinuity in political will when reform-minded leaders and other allies left office.
We also could have done a better job of structuring our support to some grantees to enable them to work with policy makers and other key stakeholders to answer vital questions about scaling up. Such as: when to scale and whether this meant deepening services and impact in existing locations, viral spread of innovations, or scaling programs vertically through government or other key providers to reach more children with a basic package of improvements? How to do this without losing the basic integrity of these instructional models? Who would be responsible for quality assurance? Who pays for what and how to achieve the necessary commitment and clarify the roles of various key actors?
So what does this mean for the remaining nine months or so of this time-bound initiative that we started several years ago? The Hewlett Foundation is not funding the full scale-up costs of these programs—we don’t see that as our role in getting sustainable solutions in place, and simply don’t have the resources to do so. Rather, we are supporting our grantees to consolidate and expand the most promising instructional models, and also to use the results of evaluations and cost analyses to their best advantage. We are providing more intensive technical support and capacity building to a few of our existing grantees to assist them in pursuing opportunities that have emerged for scaling up. We also hope to document and share some of our experiences through our participation and support of the Brookings Institution Center for Universal Education’s “Millions Learning” work. Finally, we will be intensifying our support for household-based assessments of student learning, like ASER in India, Uwezo in East Africa and Beekunko and Jangandoo in Mali and Senegal, in order to better capture data on all children’s learning and inspire both communities and policy-makers to take notice and take action to improve children’s learning.