Making Outcome-Based Approaches Work: Insights from the UK's Government Outcomes Lab

ETHOS Issue 25, April 2023

Introduction

The Government Outcomes Lab is a research programme based at Oxford’s Blavatnik School of Government.¹ It started in 2016 as a partnership between the UK Government and the University of Oxford, to look at ways to tackle complex social and environmental issues by working collaboratively across sectors. In today’s hybrid environment, public services are no longer just delivered by the public sector. Instead, we now see a multiplicity of actors, including the private sector, impact investors, non-profit organisations, voluntary groups and social enterprises. This is enriching, but also challenging: how do you bring them together to work effectively towards shared goals?

When the GO Lab was set up, the UK Government was about to launch a very large programme called the Life Chances Fund. This was aimed at exploring how a particular instrument—social impact bonds—could contribute to the portfolio of what government was doing. At the time, the evidence about the effectiveness of such instruments was still very weak. Since then, we have been working alongside government to investigate this, gathering evidence from all over the world on these instruments and how they work.

This means the government’s share of GDP is expected to rise. In the private sector, you’re paying for these better services. But in the public sector, there’s a limit to how much you can meet these higher expectations purely through tax-based public funding. In countries like the UK, for instance, it’s taboo to even think about market pricing for healthcare services because of perceived inequalities in affordability.

So, increasing demands on the public sector are not going to go away, but there is a limit to how much public funds we can raise through taxation. The question is how we are going to manage both the demand for and supply of government services. We will have to get much more efficient in what we do and find better ways of doing the same things, because we cannot just keep throwing money at problems.

In the beginning, the government wanted to know if social impact bonds "work". Over time, we unpacked this question—a social impact bond may or may not work, in the same way a simple grant may or may not work. So the question is more about the circumstances under which instruments like social impact bonds are more likely to be effective. We have found that there is no one model: the term "social impact bond" or "social impact partnership" is in fact an umbrella term that includes very many practices and types of agreements between the public sector and other sectors, including impact investors and those from the non-profit world.

The core idea is what we call an outcome-based approach: we come to an agreement for the provision of a particular activity or service, but we will pay only when a certain level of predefined outcomes has been achieved. A social impact bond is thus a kind of outcome-based financing model based on the presence of an impact investor who is willing to provide the upfront capital to provide the service, which will only be paid ex-post, and only if outcomes have been delivered.

Outcome-based approaches are most successfully deployed as part of attempts to do things differently because the status quo has not been satisfactory.

The idea is that the impact investor will bear the risk in case the pre-agreed outcomes have not been achieved. Given the resource constraints of the public sector, outcome-based contracting is hence, potentially, a good way to make more efficient use of resources, because the government only has to pay when it works. If a programme is successful, and governments pay for the achieved outcomes, the originally invested capital could even be reinvested in the same or other programmes.

In my view, having looked at these approaches for several years, outcome-based approaches are most successfully deployed as part of attempts to do things differently because the status quo has not been satisfactory. However, it can be tricky to distinguish between whether a programme has worked because of a particular intervention, or the governance arrangement (i.e., the outcome-based approach) under which it was implemented. Sometimes, what works is the combination and interaction of the two.

Nevertheless, in our research, we have found that outcome-based contracts tend to be most effective when designed and used intentionally to improve collaboration, prevention or innovation.

Three Logics of Outcome-Based Contracting

1   Logic of Collaboration

With complex social issues, there is often fragmentation: a person who is homeless, for instance, requires support from several different services. They may have a problem with finding a job, with the housing they have, alongside problems of substance abuse and mental health or family issues.

No single agency can offer support in all these dimensions. Typically, what happens is that there are separate contracts: one to support housing, a separate one to support employment, and yet another to support mental health. And the person who needs these has to report to separate agencies and retell their story; the respective service providers don’t collaborate.

Here, an outcome-based approach seems to be promising because it is not premised on who delivers what service or how it is organised: what you want to see is the outcome, which is a person finding their way out of an unsustainable situation. Usually, an organisation emerges as the main contractual counterpart who works with a portfolio of other organisations offering different services. Often, a link worker, who knows the individual client’s situation and needs, then offers wraparound care, guiding them through the different services, making the connections and offering a holistic package that makes sense for the person needing help.

An outcome-based contract makes this arrangement easier to operate, because you have one coordinating organisation responsible for the overall outcome, with the freedom to organise personalised packages of different services to make an overall difference.

2   Logic of Prevention

The public sector is often firefighting and having to intervene in one crisis after another. With greater demand for public services and increasingly constrained resources, the pressure is to cut provision in some areas. The investments that bring benefits only in the longer term, such as preventative services, are often the ones that get dropped first.

This is where an outcome-based approach with a logic of prevention has been promising. A programme might have access to funds to help reduce the number of fires that need to be fought in future—and this pays back the investment that is made at the beginning, through cost savings. Outcomes to prevent long-term unemployment reduce demand for support services down the road.

In such projects, you can have a case history and track record to see where trends are going, allowing you to make an informed guess on whether the intervention has made a difference in changing the trend and reducing the cost of the service. The cost savings can then be shared between the public sector and those who have made it happen.

3   Logic of Innovation

The public sector is often considered conservative, risk averse, and not particularly innovative. We tend to see interventions being delivered over and over, without checking if they actually work. Doing the same thing avoids blame, compared to trying something courageous and new where something might go wrong.

An outcome-based approach with a view to trying things out could help break through this risk-averse culture, by allowing discretionary space for organisations to try something new that hasn’t been done before, particularly when the status quo is not working.

Even if the logics are clear and in place, there are many different ways to carry out outcome-based contracts. Outcome-based approaches are complex instruments. Not all projects will succeed. Transaction costs can be high, and without clarity of intention and commitment, people can lose patience and then little benefit is gained afterwards. We see that these instruments have been most effective when they are used to adapt, change and build relationships in the field, in order to create a working ecosystem and a learning culture.

Outcome-based approaches are complex instruments. Not all projects will succeed.

Giving Outcome-Based Approaches the Best Chance to Work

Clarify Intentions and End Goals

It is important to clarify what one is trying to achieve through the outcome-based intervention, so that it can be carried out more intentionally, and evaluated appropriately. An important priority is thinking about the exit strategy: what happens at the end of the project. We have found this to be a revealing and provocative question in practice.

Do you want to trial an intervention to deploy elsewhere once if the project ends, even outside an outcome-based programme? Or do you want to explore ways to better collaborate with partners? Or your goal might be prevention, in which case you might want to keep issuing similar contracts if they work by continuing to access patient capital. Once you know what you are really after, you can work backwards to set up an appropriate way to design and also evaluate the programme.

Asking people to look forward to the future also obliges them to think about why they are embarking on this project. It prompts them to be more mission- and purpose-driven, and helps bring out the motivation of why people started on this work in the first place.

Active Performance Management

Much goes on during the delivery of a service that can make a difference to whether an intervention works. In typical public sector procurement, most attention is paid to a contract before it is issued, but afterwards little is paid to the job until the contract is due to be renewed. However, when an impact investor is involved in an outcome-based contract, they have a stake in making sure the project succeeds: otherwise, they lose their capital.

A good investor, typically in close collaboration with the other stakeholders involved in the programme, will practise active performance management: they will come to the table to check that things are on track, and that the outcomes are being delivered as intended. While there are transaction costs to meetings and handholding, there could be benefits from this hands-on approach that outweigh the additional cost as delivery problems can be tackled promptly.

Evaluate Outcomes Thoughtfully

How impact is measured in outcome-based contracts is crucial. Metrics and performance indicators are meant to be proxies for what the outcomes are, but they exist on a continuum and not all of them will be directly indicative of the outcome. While it is rare for payment in outcome-based contracts to be 100% based on purely outcome-driven indicators, how these proxy measures are used seems to make a difference to the success of a project. They shape the motivation and focus of the contracted organisation.

Organisations that are willing to work in a relational fashion seem to make the best use of flexible contractual arrangements.

In one example we have seen, two organisations were given the same contract to tackle homelessness in the UK. One used the outcome-based approach to motivate staff to do things differently, and to redefine what good looked like. The other organisation, which was smaller and more dependent on the contract’s paid value, put emphasis on meeting targets. The metrics were the same, but the way they were used to manage staff was different. In one organisation, the emphasis was on learning how to achieve outcomes. In the other, staff seemed more pressured to deliver on targets and demotivated. The same contract was deployed by one organisation to build a culture of learning and to focus on the long term, while the other used it transactionally. This experience teaches us that contractual clauses and technical features of these instruments are only half of te story.

Relationships of Trust

Organisations that are purpose-driven and that are willing to work in a relational fashion seem to make the best use of flexible contractual arrangements.

The basic theory of transaction cost economics suggests that if you can specify everything in a task, you can contract it out, but if it is too complex and you cannot, you need to do it in-house. But the world is not so black-and-white. What do you do with elements you cannot specify in a contract? If the counterpart is someone you can trust, the contract works better—but how then do you select a trustworthy partner?

In places like Japan, there are traditional long-term relationships between organisations and between the public and private sectors that liberal economic ideology might traditionally have considered suboptimal. But new forms of relational contracting are now becoming more common, because finding a good supplier is very costly. Private sector firms are now attempting to nurture a meaningful, open long-term relationship with counterparts, and not only stop at a reductionist interpretation of contractual clauses.

There is value in establishing an environment of trust, where you allow variation in contracts within a range without needing to issue contractual amendments each time. However, we also want to avoid being captured by a supplier who can dictate terms unduly and become complacent.

One of the success factors in outcome-based contracts is finding an area where you can build trust between stakeholders and different sectors, without being naïve about it. Finding and codifying this balance, such as through legal mechanisms, allows trust to become an instrument for improvement.

The key lesson here is that we cannot rely on a legal contract to do all the work, so civil servants will still need to continually educate themselves, stay abreast of what is happening, and exercise judgement.

INDIGO

International Network on Data Impacting Government Outcomes

INDIGO is a collaborative initiative of the Government Outcomes Lab. It gathers data—made available either publicly or through our stakeholders—related to impact bonds and outcome-based contracts from all over the world into an online database for like-minded individuals to learn from how these approaches have been used in the past.

For example, anyone who wants to launch a contract in education can go to INDIGO and find similar contracts or data on related projects. They can look up or compare projects in a particular sector or region and reach out to people that were involved in past projects.

INDIGO seeks to digitise and make accessible all empirical evidence on outcome-based contracts, as well as papers, reports and evaluations that provide credible empirical insights on how the outcome-based approach works. The aim is to bring evidence to bear in public sector decision-making, as well as to capture and promote institutional learning about this approach globally.

For more information, see: https://golab.bsg.ox.ac.uk/knowledge-bank/indigo/

A Culture of Learning

For the civil servant looking to implement good outcome-based approaches, we have found that it is important to have a mindset of wanting to learn and being open to being wrong. In public sector work, the narrative is often about success and failure. That is the wrong narrative. Whether or not a programme delivers what is expected, we should ask: why did it or why did it not do so?

This also implies that when government agencies contract out for a service, they should also always acquire the resultant data. This is where the real insights are, but all too often these stay with the organisations providing the service. When we buy a service, we should also buy the learning, and that learning should be shared.

We need to bring back learning as a key objective, because the world is not static, and neither are societal problems. We need to have an incurably curious mind that continually asks: If this intervention works, I wonder how long it will work for? And if it doesn’t, why not? Centring learning also allows us to open up and say: Something that has worked elsewhere or worked in the past didn’t work in this situation—why is that?

So often in the public sector there is a culture of blame, and of flying low. We need to invest in creating a professional norm that emphasises learning and being open to sharing the insights from both what has worked well and what has not. We could begin by anonymising the case examples, while emphasising the ethical need to share the learning.

In making space for discretionary experimentation and cultivating this appetite for learning, we can begin to create trust across an organisation. The alternative is a transactionally oriented culture which tends to promote a race to doing the minimum and hiding mistakes, rather than innovating and growing. We should rise up to the level of our best peers. In the public sector, this is countercultural, but it is fundamental.

ABOUT THE AUTHOR

Mara Airoldi is the inaugural Academic Director of the Government Outcomes Lab at the Blavatnik School of Government. She is an Economist and Decision Analyst by background and holds degrees from Bocconi University in Milan and the London School of Economics and Political Science. Her research is motivated by a desire to improve decision making in government, with a special interest and extensive expertise in the field of healthcare.

NOTE

1. Government Outcomes Lab, accessed February 23, 2023, https://golab.bsg.ox.ac.uk/.

Back to ETHOS Issue page