Key Questions About Philanthropy, Part 3: Making and Evaluating Grants

Throughout the post, “we” refers to GiveWell and Good Ventures, who work as partners on the Open Philanthropy Project.

This post is third in a series on fundamental (and under-discussed) questions about philanthropy that we’ve grappled with in starting a grantmaking organization (see previous link for the series intro, and this link for the second installment). This post covers the following questions:

When making a grant, should we focus most on evaluating the strategy/intervention, the leadership, or something else?

We've received a fair amount of advice to focus more on supporting the best people than on supporting the best ideas, since execution is extremely important and ideas tend to evolve greatly as a project progresses. On the flip side, we believe it is often very difficult to evaluate people with any reliability, especially when working with people who don't have significant track records (as may be necessary when funding novel projects and nascent fields).

I note that I see a couple of disanalogies between philanthropy and for-profit investing that point to a relatively larger role for evaluating strategies/interventions, relative to people:

In general, for relatively small "seed" grants, we've had the attitude that we want to be excited about the idea or the people, but not necessarily both, since it can be very difficult to evaluate either with confidence. For the biggest bets, we believe it is important to do what we can to evaluate both.

For a given budget, is it best to make fewer and larger grants or more numerous and smaller grants?

We know relatively little about other funders’ views on this question, other than that the Sandler Foundation seems to favor fewer and larger grants compared to most foundations we've seen.

Our current thinking is based on a couple of conceptual points:

What sort of paperwork should accompany a grant?

Based on GiveWell's experience as a grantee, as well as grant proposals we've seen from nonprofits we've worked with, it appears that funders often require grantees to complete lengthy writeups about their plans, strengths, weaknesses, and alignment with funder goals. Such writeups also often include setting out specific goals that later progress can be measured against.

So far, we've tried to take a different approach: we've tried to write up the case for each grant ourselves, and we've tried to get information from grantees in the form of conversations, other informal communications, and already-existing documents rather than requesting that new materials be created. We believe this approach allows us to maintain quality control over our writeups and avoid excessively burdening grantees. Ultimately, the purpose of our writeups is to assess grants according to our own values and strategies, so it seems logical for us to develop the capacity for creating such writeups, rather than asking our grantees to do so. When it comes to setting out specific goals, we generally include a section on this, but often stick to long-term and/or highly general goals, because we think this is appropriate for many sorts of grants (more below).

We generally ask grantees to sign a grant agreement covering topics such as confidentiality, our expectations around checking in and sharing public writeups, our preferences that any original research share its data and create a pre-analysis plan, and legal issues.

What should the relationship be between different funders? How strongly should we seek collaboration, versus seeking to fund what others won't?

It seems to us that many major funders greatly value collaboration, and often pursue multi-funder partnerships. We don't fully understand the reasons for this, and would like to understand them better. Our instincts tend to run the other way: we seek to fund the best work that we believe would have trouble getting funded otherwise. All else equal, seeing that other funders are interested in an approach makes us less interested in that approach.

Sometimes, when a project fails to get interest from other funders, this can be a warning sign. Talking to funders who could have supported a project, and didn't, can help us learn about possible shortcomings and reservations we would not have thought of otherwise. On the other hand, sometimes a lack of interest from other funders simply means that we've found a neglected opportunity, which is exactly what we're looking for. For this reason, we generally seek to think about the funders who seem like the best fit to support a given project, then try to understand why they aren't supporting it. Sometimes the reasons turn out to be compelling to us, and sometimes they end up seeming like reasons we are happy to discount (particularly when the reasons are "structural," i.e., a project is considered to be out of another funder’s focus areas rather than having anything wrong with it).

We seek to maintain contact with other funders who work on similar causes to us. One benefit of doing so is that we can learn from them. Another is that we can avoid doing work that is redundant with theirs. Having a practice of trying to understand why they don't support the things we support, as described in the above paragraph, accomplishes some of each.

Another benefit is that we can sometimes pitch promising giving opportunities to them, which can result in the projects we find most exciting getting more support. In some cases, another funder might be excited to commit all the funding that is needed, making our own support for a project unnecessary. We've also heard informally that many funders value "getting in on the ground floor," so involving another funder early on may make a big difference to a project's or organization's long-term funding situation.

But the kind of informal contact described above – checking in, discussing potential grants, pitching giving opportunities – falls well short of formal, large-scale, or long-term collaborations. Our feeling is that a collaboration of this sort with a staffed foundation – possessing its own focus areas, strategy and style – would likely introduce a significant amount of time cost and coordination challenges, and we haven't yet come across a situation in which that seemed like the best approach.

How should we evaluate the results of our grants?

Out of all the questions we've listed, this may be the one that has been most extensively discussed in writing:

We believe the best approach to evaluation depends very much on the specific case. When evaluating GiveWell top charities – which are defined by relatively linear, quantifiable, evidence-backed uses of money – we set clear expectations and assess progress regularly, along "strategic philanthropy" lines. But we don't think this approach is appropriate in all cases, particularly when working on very long time horizons and on high-risk goals, such that the points raised in "The Elusive Craft of Evaluating Advocacy" become more relevant. In general, our writeups on each grant include discussion of our plans for follow-up.