Building a demographic data platform for openness, learning and transparency: Q&A with Center for Effective Philanthropy

The Hewlett Foundation is publicly releasing a new data platform that describes the racial, ethnic, and gender makeup of its U.S. grantees—a platform developed in partnership with the Center for Effective Philanthropy (CEP). In this interview, Kevin Bolduc, Vice President of Assessment and Advisory Services, and Della Menhaj, Associate Manager of Assessment and Advisory Services, describe the platform, as well as the data collection and design process with Hewlett’s Grantmaking Operations, Insight, and Learning team, to explain how tools like this can support learning to advance diversity, equity, and inclusion.

Q: Tell us a little bit about how you approached this need and how it’s different from what others in the field have done.

A: From the outset of this work, Hewlett made clear to us that the foundation intended to use grantee demographic information for reflection, learning, and change—and that it was committed to this work for the long term. This values-centered approach freed CEP to think differently in two important ways. First, we could create a more robust process and tool because we could be confident that the resulting infrastructure would be useful over time—not only to Hewlett, but potentially to other funders as well. Second, we could focus on creating an interactive tool for exploration and learning, rather than simply the kind of static report that is the end result of many demographic data collection efforts.

This interactivity creates an ability to easily review overall organizational results alongside a mechanism to help individual teams test assumptions and hypotheses about differences that may not be visible in the overall data. For example, a simple flip of a switch allows a user easily to view different relationships present in the data for their team, such as the extent to which demographic characteristics of grantees’ organizations are associated with other factors like grantee location, grant size, grantee budget, etc.

Finally, Hewlett and CEP’s commitment to transparency and field-wide learning steered this work to what strikes us as an uncommon level of sharing and openness. While we’ve seen quite a few reports showing the aggregate demographic data foundations have collected, we’re not aware of any other instances in which a funder has created the opportunity for those outside the foundation to explore this kind of data, in-depth, on their own. We hope this approach invites a broader conversation, the learning from which can enrich the approach we all use to collect and understand this important information.

Q: What were the key choices you needed to make during the design process?

A: In addition to a deep commitment to building safeguards that protect the confidentiality of individual respondents, the most important choice we made in the design process was to create an approach that would allow for iteration and flexibility wherever possible. As our own understanding of demographic information grows—both as a field and as individual institutions—we know its uses will also change and evolve. With this in mind, we focused on building a system that would allow for new questions to be added, new hypotheses or cuts of data to be explored in future iterations, and new insights to shape future functionality.

Building on the good work Harder & Company had done for Hewlett in a pilot version of this survey, we also knew that different visual presentations of the information resonated more or less effectively with people depending on their needs. We maintained a variety of different approaches to showing this data, each of which help answer a different set of questions.

Q: How did you minimize burden for grantees?

A: This was an important goal for Hewlett’s Grantmaking Operations, Insight, and Learning team, with whom we worked on the survey design. Among the design choices made to minimize grantee burden were to allow grantees to understand what demographic information would be collected before they entered the survey, to ensure that grantees had a chance early in the survey to opt-out (and describe their reasons for doing so), and to design the survey with widely used categories so that grantees could reuse data they may have collected for another purpose. The team also decided to collect the data on a rolling basis such that any individual grantee is approached only every three years. And transparency is helpful here, too: Making the survey instrument itself available to other funders can help spur other funders’ learning and minimize the number of approaches in the field.

Q: Looking back at the process, what did you learn? Were there any surprises?

A: This is not so much a surprise, but it is an important reminder: Even a relatively small number of demographic questions—in this case, about race and ethnicity and gender identity—can create a very powerful dataset when linked thoughtfully to other information in a funder’s grants database. Having deeply engaged partners in Sara Davis and Sarah Yun [from Hewlett’s Grantmaking Operations, Insight, and Learning team] made all the difference when it came to creating the most useful links. Sara and Sarah helped us translate questions and ideas from the rest of the foundation into the data fields and visualizations, which, in turn, resulted in deeper learning and action from staff and new questions about how to use this information. They also helped us spot where the data didn’t make sense and how to fix it. Nearly two years into our partnership, we’re still learning from them every day—and we hope the feeling is mutual!

Q: How are you working with other funders?

A: Thanks to Hewlett’s funding for the development of this platform, CEP has been able to use it as a baseline to visualize results of demographic data for a variety of other funders. In many of those cases, we’ve worked with funders to develop demographic surveys specific to their programmatic needs, to field those surveys on their behalf, and to visualize the results in ways that lead to rich reflection and learning. That work is customized for each funder we’ve had the privilege of working with in order to meet their goals. But, wherever possible, we’re seeking to use our engagement with funders to minimize the number of different approaches used in the field. This will ultimately minimize the burden on grantees and maximize the likelihood the funders will be able to learn from each other’s experiences.