What can we take away from this extensive policy effort? Foremost, it was indeed extensive. From the time Moss and Leo first proposed a consolidated DFC to when the BUILD Act was finally passed, nearly seven and a half years had passed. Such a lengthy time horizon for policy impact is not atypical for think tanks. Secondly, Moss, Leo, and CGD did not operate in isolation. They partnered closely with Board member and the ONE Campaign to provide much-needed advocacy support, built influential relationships with key congressional staffers at the right time to capitalize on political momentum, and rallied other think tanks like CSIS and the Brookings Institution to the cause. In essence, policymaking of this scale and nature rarely happens through single-player activation. 

The example and factors above highlight the unique challenges that think tanks face in measuring impact and influence. Typically, the purpose of a think tank is to advance evidence-based policy, or to broker policy knowledge and incubate new ideas. These organizations focus their time on research and debate to generate ideas that tackle pressing problems ranging from economic inequality to climate change to global development.² The presumption, therefore, is that a successful think tank is one whose policy ideas are adopted by decisionmakers and put into action. However, the reality is rarely that simple. Policymaking is a complex process of setting agendas, passing proposals, and implementing solutions. It is a lengthy, unpredictable process with a consistently evolving set of stakeholders and Overton windows.  

In this paper, we delve into the historical context and trends affecting think tank impact, the tradeoffs that must be navigated in measuring impact, and our recommendations to reimagine impact measurement moving forward. Our recommendations are based off Camber Collective’s years of strategic support in this sector with organizations like the Center for Global Development, Urban Institute, Center for Strategic and International Studies, Freedom House, the Center for American Progress, as well as interviews with a range of think tanks, funders, and policymakers. 

Content and Trends

According to the 2020 Global Go To Think Tank Index Report, there are more than 11,175 think tanks in the world, defined broadly. These organizations operate in the “ideas industry,” a term coined by Daniel Drezner to describe a marketplace where academics, researchers, and political pundits attempt to “sell” their ideas to policymakers in the US and beyond.³ While all think tanks have some role in generating and communicating ideas, their actual placement in the policy, research, and governance ecosystems varies widely. Think tanks may fall anywhere across a variety of spectrums: research-driven vs advocacy-focused, single vs multi-issue, partisan vs non-partisan, global vs local. For example, some institutions describe themselves as “universities without students,” producing top-quality research that serves as the evidence-base for policy choices, while others emphasize advocacy using strong communications and messaging campaigns.⁴ Audience targets also vary, from elected politicians (e.g., members of Congress) to influential bureaucrats (e.g., USAID administrators) to global leaders (e.g., UN officers). Funding is another key differentiator, and think tanks can be fully autonomous and independent, quasi- or fully government funded, or associated with universities, corporations, and/or political parties.  

Regardless of their specific target audience or overall approach, think tanks play an important role in today’s world. They provide a critical bridge between policy ideas, research and practical implementation, and they  often operate as the rare actor that is able to broker what is desired and what is possible for policy change. Think tanks ability to continue to play this role, however, is growing increasingly complicated.  

Today, this diverse field faces evolving trends that pose challenges to their operating environment and raise the stakes for institutions to define and measure their impact. First, political polarization (especially in the US) has made it difficult for think tanks to be perceived as non-partisan (most think tanks operate as 501(c)(3) in the United States and must operate with independence to maintain tax status). Through interviews with key think tanks and funders, we heard that “it appears that policymakers only read reports… from think tanks with which they agree politically.”⁵ Secondly, the proliferation of organizations attempting to drive public discourse and influence policy has increased competition for funding and decisionmaker attention. This takes the form of an increasingly fractured set of competitors, as the lines between think tanks, media, investigative journalism, and consulting firms blur, and the rise of social and online media change how audiences digest information. In the words of one interviewee, “the policy landscape has shifted and interest in traditional think tank product is waning… unless you’re a policy wonk, you don’t sit and read anymore.”⁶ 

Think tank funders, traditionally a combination of philanthropies and high net worth individuals (HNWIs) as well as government agencies, have greater influence given the competition. This increases the expectations of strategic alignment and proof-of-impact, sometimes resulting in overly siloed or funder-driven research programs. Amid this polarized and competitive environment, concerns over think tank independence are on the rise. The New York Times published a series of investigations between 2014 and 2017 that highlight how the struggle for funding has led some funders and think tanks to be unduly influenced by corporate and foreign donors. For example, between 2007 and 2015, the Heritage Foundation received at least $5.8 million from a Korean weapons manufacturer whose autonomous weapons system was touted by Heritage experts.⁷ While many think tanks fully disclose donors, maintaining independence -both actual and perceived- is a priority and challenge throughout the sector. The problematic choices of a few has led some to distrust think tanks, or as one pointedly named Foreign Policy article Why Everyone Hates Think Tanks describes, “it is time we confront the truth that think tanks have a serious, and perhaps also a deserved, reputation problem.”⁸ Ultimately, it falls on both think tanks and funders to be aware of the pressures they apply on each other and to realize the undue influences that can sometimes can lead to poor decisions. Think tanks must do what they can to maintain transparency and independence, and funders, in parallel, must recognize the deep value of non-partisan research so they can resist the urge to apply unwarranted and sometimes harmful indirect influence. 

Measuring Impact and Influence

Given the environmental pressures and context in which think tanks operate, the impetus for measuring (and disclosing) impact and influence is really two-fold. Externally, being able to clearly articulate one’s activities and what effects they had on the world provides a level of transparency and accountability for think tanks to their funders, stakeholders, and society at large. Internally, creating a model for defining and measuring impact allows think tanks to be more strategic and thoughtful, resulting in better stewarding of resources. Unfortunately, doing so has been a persistent and well-documented challenge for think tanks. After all, what is the impact of an idea? 

The first challenge that many think tanks face in measuring impact and influence is related to attribution or how to connect the research, expertise, or convenings to tangible policymaking or changes in perception. Arthur C. Brooks, President of the American Enterprise Institute, touched upon this issue in the Harvard Business Review: “[Think tank] output is pretty straightforward: books, research articles, op-eds, media appearances, public events, and so on. These products effectively constitute our supply curve. But nobody contends that simply writing an op-ed, publishing a peer-reviewed paper, or booking a scholar on television automatically guarantees a change in how leaders think and act.”⁹ These outputs are proxy measures, and proxies by their nature have limitations. They tend to be snapshots of a confined time period, are subject to the volatility of current events, and do not always link directly to “impact.” Moreover, policymaking is just complex, as the passage of the BUILD Act demonstrated. It’s nearly impossible to measure the impact of any one output, action, or organization if the qualifier is solely defined as policy change. To start, think tanks need a logic model that clearly states how their activities and outputs connect to short-term and long-term outcomes, and ultimately, to the impacts they wish to foster. 

The second hurdle that think tanks face in measuring impact and influence revolves around infrastructure: building the capacity, technology, and processes necessary to track impact long-term. More often than not, funder reporting requirements dictate what datapoints teams track and measure, given limited capacity to dedicate towards overall impact measurement. Hence, organizations lack clarity or efficiency in evaluating impact, whether this be ad-hoc requests for impact data, divergent definitions of what “impact” actually means, or a dearth of senior leadership to propel the work forward. The level of think tank investment in self-evaluation varies from one group to another (sometimes from one internal team to another), but in general, teams struggle with either securing the funds to build meaningful capabilities or generating enough organization-wide buy-in to invest—or both. For think tanks that operate like academic institutions, a lack of “buy-in” can be a barrier to developing a culture of consistent evaluation. As one interviewee states, “There are fellows at [our think tank] who believe that their research and work are valuable in itself… that the process of measuring impact is inherently flawed, and therefore, a hard task to prioritize.”¹⁰ To address this, think tanks need to collaborate closely with their funders to not only make appropriate investments, but also collectively agree on what is most important to measure.  

Tensions and Tradeoffs

Given the issues and trends that think tanks face, there are various tradeoffs and tensions in tracking impact and influence. In our experience working with think tanks, some of the key questions and considerations are: 

1. How do you determine a research agenda?  Research agendas can be driven by policymaker demand, researcher experience, and/or funder interest. They can also be determined based on gaps in the field or an estimation of potential impact for a given project. With the proliferation of organizations, competition for both funding and public attention increases, making it difficult to decide how to choose an agenda. One interviewee reflected on this challenge, asking “If everyone is chasing after the same new shiny thing, what differentiates you? Sticking with what you do and doing it better may be more effective than doing twenty new things… You may also be stuck with things that were interesting ten years ago but may not be impactful today.”¹¹ 
   
   Building an agenda that ignores the current policy discourse is a road to irrelevance. At the same time, a research agenda too broad can also lead to pitfalls. Another individual we spoke to commented, “Most think tanks are thin on an issue-by-issue basis. There are real benefits in the long run of working on fewer issues and having larger teams.”¹² However, scholars typically have evolving interests and may wish to set a dynamic agenda that diverges from funder or even policymaker interest. Any of these approaches to determining a research agenda require tradeoffs, necessitating clarity about the choice and its consequences. 

2. What is the best way to navigate the funder landscape? Think tanks not affiliated with a university or another primary funding entity typically rely on a combination of private foundations, HNWIs, mixed-vehicle philanthropies, and U.S. government agencies for their operating capital—and for some think tanks, corporations and foreign governments as well. The need to deploy varied development strategies hampers think tanks’ abilities to make decisions without considering funding sources. They also must wrangle with complicated determinations around whether to pursue organization wide or program, issue, or project-specific funding. These decisions are difficult, yet essential, as one interviewee reflected, many “think tanks are all running after the same rich people. It makes [them] very hand to mouth.”¹³  Siloed, short-term funding may fuel programs, but it also engenders significant administrative burden and the potential for undue funder influence.
   
   As we heard, “If you’re selling your research there is always the question of objectivity. Is the buyer really going to pay for something that comes out against them?” Think tanks and philanthropy alike should focus on long-term, unrestricted, or flexible funding, but it runs counter to the current project-specific and short-term grantmaking strategies of many foundations. Further, if the funding framework were to change, success is not a given.  

3. Is success defined in terms of policy change or policy outcomes? Balancing the idea of impact as policy change (evidence-based policy suggestion is passed or implemented) versus impact as policy outcomes (the extent to which a given policy had the desired effect) creates tension. Funders may express interest in the actual outcomes–for example, does a given think tank’s research about education ultimately lead to better educational outcomes for a target population? Indeed, given the variability of policy outcomes and the difficulty of assigning any causal attribution between outcomes and a think tank’s work, defining success in terms of policy outcomes can be impossible. All the same, it is dangerous to ignore policy outcomes. Doing so risks disconnecting the work from the mission and neglecting all the ways in which lives are actually affected, especially already-marginalized communities. Think tanks and their funders should make conscious decisions about the tradeoffs in defining success and create a logic model that most thoroughly captures the impact being made. Funders must also recognize that changing the conversation, or the facts that constitute the basis of developing policy, can also be a form of policy impact independent of outcomes in a given population.  

Recommendations

Despite the challenges and tensions, measuring impact and influence is still of vital importance for think tanks, not only because of external accountability and internal alignment, but also to compel think tanks to evolve and meet grassroots demands. Coming out of a world-defining pandemic and a new era of intolerance for racial injustice, think tanks need to build the infrastructure necessary to consistently evaluate their work and become better stewards of influence for all stakeholders, particularly impacted communities. This is not possible without measuring impact and influence. The era of think tank research speaking for itself is over.  

As Emma Vadehra, senior fellow at The Century Foundation, states: “As the broader policy ecosystem adjusts to a post-2020 world, think tanks that aim to provide the intellectual backbone to policy movements—through research, data analysis, and evidence-based recommendation—need to change their approach as well.”¹⁴ In essence, the very measurement of think tank impact and influence needs to be reimagined, steering away from counting productivity outputs and obligatory funder reporting to meaningful evaluation and learning with a clearly defined theory of influence. Think tanks can achieve this through a series of strategic shifts: 

1. Moving from policy agenda to theories of influence (TOI). Before setting policy agendas, program teams need to first define a theory of impact or influence towards achieving their missions. The TOI should then be used to define what to do and not do in terms of research, policy, and advocacy. This includes a logic model that outlines how inputs (e.g., staff time, resources), activities (e.g., conduct a pilot study, host a public forum), and outputs (e.g., published journal article, congressional testimony) lead to outcomes (e.g., issue salience, policy approval) and ultimate impact goals. Another core component of a TOI is clarity around who the audience is and why–a TOI should be driven by who needs to be influenced and which communities are being impacted, as well as why they are important. TOIs also have value at both the organization and program level, as they provide a structured narrative to describe the what, how, and why of the work. It is a central framework for any organization that wants to better define and understand impact.  

2. Moving from “bean-counting” to impact narration. Many think tanks fixate on the easily quantifiable outputs of a TOI when all parts of the logic model from inputs to impact are important in telling the full narrative of impact. Instead of focusing on fluctuations in outputs or activities to articulate productivity or effectiveness, organizations should use narratives to humanize the data and capture the full arc of their impact. This can be in the form of short vignettes or full case studies, as long as the stories recognize the complexity of policy influence, the contributions of various stakeholders, the lengthy time horizons for impact, and the results on impacted communities. Sarah Lucas, formerly of the Hewlett Foundation, a major funder in the industry, echoes this: “My view is that moving beyond numbers — and talking instead about how they are positioned, what they decide to work on, and who they work with — can help think tanks overcome their angst about impact.”¹⁵ These vignettes can also be developed and shared during key moments on a long-term pathway to impact, as organizations do not need to wait until policy change to share their learning journeys. 

3. Moving from funder reporting to evaluation and learning. The best TOI or impact framework is meaningless unless the organization buys into impact measurement for internal growth. Too often, program teams and fellows feel obligated to report impact for funding purposes only, but tracking impact is as much for developing team strategy, tracking progress, and learning from past successes and failures. Building this infrastructure will require collaboration between think tanks and their funders, prioritizing investments in senior leadership, technology, and staff capacity to measure impact and influence. The end goal is a learning organization that can adapt to the challenging environment in which think tanks operate. Furthermore, if this learning can happen transparently, it creates an opportunity for the public to contribute to the growth of think tanks as policy “expertise” becomes more collective and communal. 

Conclusion

In 2021, as Todd Moss and Ben Leo reflected upon their years-long efforts to pass the BUILD Act, they landed upon some clear takeaways. For starters, building relationships with key congressional staffers, grassroots lobbying groups, and other stakeholders in the development community was vital. Moss and Leo’s experience in government, as well as CGD’s board relationships, contributed to such relationships. Furthermore, the initial policy proposal for a DFC had arguments that appealed to both Republicans and Democrats. Proponents could adjust the benefits depending on who the audience was, and that flexibility was powerful. While no one at CGD contends that Moss and Leo contributed solely to the BUILD Act, many in the development community do credit their leadership and persistence in seeing it through. Many also recognize that the fight is not over—in its second year of operation, the USDFC has yet to make large-scale investments in low and middle-income countries, and CGD scholars continue to shine a light on these challenges. On reflection, there is a clear narrative with through lines and learnings that CGD and the global development community can apply to future advocacy efforts. With a clear theory of change, the right infrastructure investments and partnerships, and long-term commitment, all think tanks can and should measure their impact to better themselves and the communities they seek to affect.   

---

Notes

¹ George Ingram, Dan Runde, Homi Kara, Ben Leo, “Strengthening U.S. Government Development Finance Institutions.” Brookings Institute, 2013. 

² John de Boer and Rohinton Medhora, “What Are Think Tanks Good For?” United Nations University Center for Policy Research, 2015. 

³ Daniel Drezner, “The Ideas Industry: How Pessimists, Partisans, and Plutocrats Are Transforming the Marketplace of Ideas,” Oxford University Press, 2017. 

⁴ Alek Chance, “Think Tanks in the United States: Activities, Agendas, and Influence,” ICAS, 2016. 

⁵ Camber Collective project interviews/survey 

⁶ Camber Collective project interviews/survey 

⁷ Eli Clifton and Ben Freeman, “Restoring Trust in the Think Tank Sector,” Quincy Institute for Responsible Statecraft, 2021. 

⁸ Matthew Rojansky and Jeremy Shapiro, “Why Everyone Hates Think Tanks,” Foreign Policy Magazine, 2021. 

⁹ Arthur Brooks, “AEI’s President on Measuring the Impact of Ideas,” Harvard Business Review, 2018. 

¹⁰ Camber Collective project interviews 

¹¹ Camber Collective project interviews/survey 

¹² Camber Collective project interviews/survey 

¹³ Camber Collective project interviews/survey 

¹⁴ Emma Vadehra, “We Need to Reimagine the Modern Think Tank,” SSIR, 2021

¹⁵ Sarah Lucas, “6 ways think tanks can overcome angst about impact,” Hewlett Foundation, 2017. 

Thanks to Amanda Glassman and the Center for Global Development for their contributions to this article.

As we near a year of coronavirus and lockdowns, what started with debilitating fear of infection, peaked at pandemic fatigue, and now is beginning to curtail with the chance of vaccine-induced safety, one of the most pressing questions of development presents itself yet again: How does one distribute scarce resources?

Decision making in the face of scarce resources is undoubtedly difficult, and in many ways often presents truly impossible choices. In such situations, being strictly utilitarian (where ‘future, largely economic, benefit to society’ is the unit of utility measurement) often feels like one way in which to simply move forward, to break free from the debilitating anxiety of having to make a decision in impossible circumstances, and of at least doing something. Yet, always choosing utility as our sole compass leaves us in another bind. If we always choose to give to those people who have the highest potential return on investment (ROI), then when will those who truly need the resource get a turn?

In many ways, our systems, our societies, and even our philanthropic communities are very often not set up to actually elevate the needs of the people whose circumstances are truly desperate. As a result, true need is very rarely addressed in situations of scarcity. Instead, time and time again, across numerous contexts, societal ROI trumps true need as a decision-making criterion, and ultimately holds back our collective ability to progress.  

Take for example, the calculus that poor parents make every day when they choose to send sons to school over daughters. While a daughter’s education-less future might be far bleaker (uneducated women, as compared to educated ones, tend to have more children, earlier, with riskier pregnancies, and less healthy newborns, and are at greater risk of gender-based violence^[1]) and while the argument could be made that she ‘needed’ it more, poor parents often find themselves too lacking in resources to think this way. They must make calculated choices about how to spend their limited means and, to them, boys have a far better chance of future employment hence boys get sent to school.

This utilitarian decision-making is not just limited to other parts of the world. A well-known example can be found in the case of the Admissions and Policies Committee of the Seattle Artificial Kidney Center at Swedish Hospital, more commonly known as the “God committee.”  Formed in 1961, it was this committee’s (made up of seven citizens selected by the King County Medical Society) job to choose which patients would get hooked up twice a week to a one of a kind “artificial kidney” that could offer them a chance at life.  Ultimately the committee relied on criteria based on ‘social worth^[2]’ which itself was heavily weighted toward economic status that reflected the committee’s own values and biases (excerpt from the committee: “If we are still looking for the men with the highest potential of service to society, then I think we must consider that the chemist and the accountant have the finest educational backgrounds of all five candidates….”). Were patients more at-risk given priority, or were patients who had more ‘social worth’ given precedence? Once again, in the face of scarce resources, the availability of only one machine, there was no room to prioritize those who ‘needed it’ the most.

Similarly, in the philanthropic community, how many grants require visible results in the near term? And in turn, how many local NGOs must carry out their programing not in the districts where it is most needed, but, rather, in the districts in which their programs will show impact most quickly, guaranteeing them future funding and an ability to continue to exist to serve additional people? Districts which show impact quickly often do so because there are other factors in place – infrastructure, norms, expertise – that support programming in being successful. Conversely, very needy districts are the districts lacking in those things – they will likely have limited or non-existent programming, infrastructure, or supportive norms, and to put it candidly, non-existent hope too, for those that truly need it.

The fundamental issue with focusing on utility in the form of societal ROI is that it leaves us, somewhat ironically, as a society, no better than before. A few families or individuals may benefit in the short-term, but as a collective, instead of overall progress, we see deepening inequality. Conversely, by truly focusing on the neediest, we may see less progress in the short term, but we fundamentally improve long term possibilities. Enabling this paradigm change, however, will require a transformative shift in how the global development sector does business.

At the outset, we must all play the long game. The philanthropic community must, as it is beginning to do, begin moving to longer terms grants with less strings attached. By allowing people in communities to no longer perceive their grants to be ultra-scarce, red-taped resources, we begin to give them breathing room so they can bring creativity and flexibility into their programming, and so they can focus where they know there is need, rather than on where they think their own survival as an organization depends.

Secondly, we must challenge norms and the basic premises of utilitarian thinking. Perhaps it seems a boy is more employable after schooling, but what does he do once he is employed? Numerous studies show that when women have access and control over the household income, they are more likely than men to invest in the health and welfare of their families. Maternal income for example has been shown to increase family nutrition by 4-7 times more than the income of fathers^[3]. When women have the education and capacity to work, they can end a vicious cycle that prevents them from earning and saving the money they need to achieve a brighter future for themselves and for their families. This is powerful. This, and the whole body of research like this, must be talked about more.

And thirdly, we must be willing to take a stand. To give to the needy because that is the right thing to do, and because that will spur the innovation required to no longer have them be the neediest anymore. At the Swedish Hospital, when the inventing doctor finally received a refusal that he simply could not fathom (a 16-year-old girl in the prime of her life), do you know what he did? He spurred the invention of a much smaller, more manageable dialysis machine that changed the stakes. His innovation led to much needier patients having access and to the eventual dissolution of the God Committee.

May we reach a point too, in philanthropy, and in the global development sector at large, where we manage to fund and support the kinds of initiatives that truly focus on the needy. May we ask the questions that help us identify the people who need resources the most, and may we do everything in our power to ensure that we get them the resources that they need. Ultimately, this is the only path forward that will help to raise people, collectively, out of poverty and that will make it so they too can finally take care of their most needy without having to always calculate the opportunity cost of doing so. 

 

[1] https://www.concernusa.org/story/girls-education/

[2] Social worth based on a combination of factors, including age, sex, marital status, number of dependents, income, net worth, emotional stability, educational background, occupation, past performance and future potential

[3] Thomas, D. (1990). Intra-Household Resource Allocation: An Inferential Approach. The Journal of Human Resources, 25(4), 635-664. doi:10.2307/145670