Iterate, if you can

The long struggle to create shorter feedback cycles in government

For around 15 years in the UK, there has been a concerted effort to introduce shorter feedback cycles in government. The goal is to reduce the time lag between having an idea, seeing if this works in the real world, learning, and trying something better.

A central focus of this work has been introducing user-centred methods and iterative management practices into the design and delivery of public services.

This effort is very important because working in quicker feedback cycles can help us to:

• Progressively improve outcomes, so that learning accretes, rather than dissipating

• Reduce risk

• Be more responsive to public expectations

• Learn how to put new technologies to use

• Make much better use of public money

This week I had a fascinating conversation with teams trying to work iteratively in a big public service. I thought it gave some usefully concrete insights into why this push for shorter feedback cycles is difficult. So although this will get pretty nerdy, I thought I’d share some thoughts here.

I’ll start with the context, then I’ll describe some real world examples, and then I’ll wrap up with some reflections and some tentative thoughts on where we go next.

Where we are now

If you go back to 2010, the UK public sector was still largely reliant on pre-digital-era management practices. This meant managing work in a linear way, e.g. trying to reduce risk by planning upfront. We treated work as monolithic projects, like building a big bridge. This made feedback cycles dangerously slow — sometimes it took literally years (and millions of pounds) before knowing if a programme or initiative was working. This contrasts with the days or hours it takes to get feedback, and adapt, in a contemporary / agile organisation.

This was all proving especially problematic when developing software (e.g. see Horizon, the NHS Spine, etc), which is nothing like building a bridge. But these methods were also failing wherever work was complex, which it turns out accounts for most of the work public institutions do. This was driving risk, waste, and inertia across the state, which in turn was stoking public frustrations with government.

From the early 2010s, there was a push to develop and spread a more user-centred and agile framework for delivery. This originated in the Government Digital Service (GDS) because those linear industrial methods were most obviously disastrous in the context of digital work. However, because of the far broader need for methods that work in complexity, these efforts soon broadened. A key vehicle for them became the Service Standard and an associated architecture of Service Assessments (Alpha, private Beta, public Beta, Live) and the spread of new disciplines and methods, most notably design.

This more iterative approach to delivery has since been spreading across the public sector, reducing risk and improving outcomes and saving money, albeit slowly and unevenly.

10–15 years on, this leaves a very different picture. I would say things are much better, but also much messier. In simple terms:

• Mature parts of the UK public sector — at least those that are seen as doing digital service delivery (and the most advanced teams leading services more generally) — now use a version of the Service Standard’s user-centred, iterative methods. But there is a lot of variation within this.

• Large parts of the public sector still haven’t made it to 2010. i.e. most teams still don’t apply contemporary methods (e.g. they have no/little input from design/user research, they don’t iterate their work in the real world, they use old school PMO practices, etc.)

• A new version of iterative working is now spreading, which is called Test & Learn. This is similar to the Service Standard and its associated practices, but differs in a few respects. It is less closely associated with digital, and so has a broader scope. It is, I would say, more explicit about the need to continually iterate in a live environment. And it also comes as part of a more ambitious agenda for public service reform, in which we modernise the system by working iteratively at the edges. As we hit barriers, this acts as a discovery mechanism, and we drive efforts to remove those barriers up and through the system.

So I guess the simple summary is that the push for iterative approaches is more important than ever.

Some real world examples

This is all quite abstract, so let me make it concrete by sharing five examples of real teams trying to work in iterative ways.

I won’t name the teams because this isn’t about individuals and it’s not particular to any one domain of policy (and to be clear I think all of these teams are doing great work). What I’m interested in is the way iterative delivery practices and philosophies are evolving and spreading.

NB: This is where things will get nerdy and a bit insider baseball.

• Team A is frustrated. They have been told to use the Alpha and Beta stages of the Service Assessment in a strictly demarcated way. Alpha is seen as a phase in which they have to test all of their riskiest assumptions, so during Alpha they cannot test with live users. They are asked to ‘prove’ their service works in order to pass through the Alpha assessment to Beta. After this they are allowed to ‘test with live users’ / ‘go into production’. Nothing is ‘built’ in Alpha. They struggle to experiment in any case, because their team is separate from the ‘policy’ team that works in the same area. The policy teams see even small tweaks to delivery as changes to policy, so they want to sign-off any changes in advance.

• Team B has more freedom to experiment. This is strange because they work in the same policy domain as Team A. However, they sit in a local part of the system. This means they don’t use the Service Assessments. Also, because they are in a local part of the system, their work is not seen as encroaching on ‘policy’, so they are less constrained. They can experiment more and learn more than Team A.

• Team C sees the Alpha and Beta stages of the Service Assessment differently. They are developing a new service and trialling it with live users, from the beginning. They are managing risk by starting very small — they are working in one narrow domain, with a tiny cohort of people, in one Local Authority. They will trial a minimal version of a live service, learn from this, and scale as they move through Beta into Live. (This is very similar to the approach that saved Universal Credit.) For them, scaling means expanding the cohort, first in the local area, and then by adding other local areas. Then they will build out sideways into other domains. This is working well and has helped them spot and correct issues early.

• Team D works as part of a wider platform. The platform itself went through the Service Assessments a few years ago, and was signed off as a Live service (even though it is not really a ‘service’). This means the sub-teams working on the platform no longer need any central permission to run experiments. Given the scale of the platform and the numbers of users involved, the team has sensibly developed its own assurance processes, which can work at pace. This is going well and they are managing risk while driving improvements to outcomes.

• Team E is seen as a non-digital team, although their service still relies on digital components/infrastructure, as pretty much all services do. This means they sit outside of the culture/processes of the Service Standard and the Service Assessments. However, they are excited by the potential of Test & Learn methods, so they are working iteratively in a local area to improve their service, trying out small changes with real users. This is driving quick and cumulative improvements in outcomes.

What do we make of all this? We see again that it is a messy picture. Indeed it’s a nice reminder of the uneven, unpredictable path of public service reform, even when it goes well. (Bear in mind that GDS/the Service Standard is rightly seen, globally, as a success story — the change it has driven has been unusually quick by the standards of public service reform).

Still, despite the messiness, are there any observations we can usefully make? I guess it feels fair to say that:

• In some cases, linear/project planning mentalities seem to have snuck their way back into the new frameworks (e.g. Service Assessments), at least in the way these processes are sometimes applied. For example, some teams feel they have to do a lot of upfront work before they learn from real people in a live environment.

• Although design and user research have led the way in making public services work better for users, these disciplines sometimes add to this sense of work being linear. Work often starts with a long upfront design/research phase, without any ‘build’ or real world testing. This is especially tricky in cases where it’s hard to learn under lab-like conditions (e.g. when the outcome is all about real world behaviour change, such as with weightloss services). (I know some designers/user researchers will dispute this, and I am not saying this is intentional. I am just saying that, in practice, in an overall system that is nervous about real-world testing, the Discovery phase can sometimes end up functioning like a better version of the old ‘project planning’ period.)

• Often, when teams struggle to work iteratively, it is because the operating environment around them is still stuck in pre-2010 industrial structures — most of all because delivery teams are separate from policy teams (or because delivery teams are separate from teams that hold decisions about risk (e.g. assurance or clinical/technical expertise)). This makes it close to impossible to achieve the rapid feedback loops of an agile organisation.

• Good teams are infinitely creative. People who understand the need to work iteratively will often find ways to do so, regardless of where they sit in the system (although it is still often very hard to work in this way). This is a useful reminder that it would be easy to make things worse by insisting on more standardised processes. What matters at heart is that people understand that iterative working makes things better, and some fundamentals of the operating model, e.g. combining policy and delivery into mixed discipline teams.

• Finally, we should not lose sight of the fact that I am focused on the parts of the public sector that are furthest ahead, and not the broad swathes that have still not arrived at the world of the Service Standard (i.e. where there is essentially no design work done, and no iterative testing). A big reason for this is that much of Whitehall still sees itself as doing ‘policy’ work, as opposed to ‘delivery’ work. And despite ongoing work to modernise the policy profession, and some nice examples of policy being integrated into multifunctional teams, policy is still largely an industrial-era discipline (i.e. it relies mainly on linear planning, rarely speaks to the public (except in transactional consultations), etc.)[1] (Again, I am not saying there are no good examples — there are some great examples of agile policy-making! I am just describing the default approach at the median.)

What should we do next?

So, what about recommendations? Is there anything we could do to improve this picture? Here are some suggestions, all loosely held:

• Because linear mentalities have crept back in some places, it would be worth a big new push to restate the basic case for iterative and user-centred methods, and to insist on the associated operating model (e.g. mixed discipline teams). Clarity is key: assert the basic principles of iterative working, explain why it reduces risk and makes better use of public money, be insistent on the model, etc. Test & Learn might be the best framing/vehicle for this, but it will need strong support from the highest levels of government if contemporary management practices and operating models are to become non-negotiable.

• It’s worth doing some dedicated work on how to apply Test & Learn to the gnarliest areas. For example, in areas where testing in a live environment is seen as especially risky (e.g. clinical services). We need to be clearer about the operating model that is needed to do this well (e.g. in these cases, we don’t just need to integrate policy and delivery, we also need to bring in clinical and assurance expertise, so that risk management happens in the team, in rapid cycles). We also need better risk management frameworks. In particular we need to get much better at capturing the risk of not experimenting, since we often allow services to atrophy because of the fallacious idea that doing nothing is risk free. This might mean moving away from a language of ‘experimenting’ (which will always sound risky) to just talk about learning.

• It would be valuable to curate the best examples of teams doing proper Test & Learn work with live users (as per Team C, and perhaps Teams D and E). And, having curated these examples of Test & Learn, maybe we could codify these practices and use them to update the processes and guidance around Service Assessments. The goal is to make sure we are supporting true iterative working, including in the trickiest areas. Perhaps we could also be clearer about the policy domains in which real world iteration is needed. As Richard Pope writes here, sometimes user research will be good enough (e.g. when developing a transactional digital service), but sometimes there is no alternative to real world testing (e.g. when we are trying to help someone lose weight, or move into work after a long period of unemployment).

• It might be useful if the disciplines of design and user research really owned the issue of Discoveryitis. i.e. getting curious about cases in which upfront design work has inhibited iteration, and about why people sometimes feel (not always fairly) that designers/user researchers are saying people should delay trying things out in the real world. Maybe this could lead to some useful guides and professional development — ways for design and user research to fit as well as possible with Test & Learn.

• Working in the open is of course a key part of all of this — it’s central to the way complex systems learn. So the government should obviously scrap its silly guidance that says civil servants shouldn’t speak in public. They should replace this with the opposite approach — a big drive to help reformers work in the open. Why not create a new platform — public.gov.uk — as a place for reformers across government to blog about the wonderful work they are doing? This is one of the most powerful ways we can cut through inertia.

Finally, there is an awkward question, which I keep putting off, about the role of the Treasury in all of this. The short version is that HMT was a drag on the efforts I have described over the last 15 years, because the processes HMT owns (business cases, etc.) and the mentalities it embodies, inhibit contemporary management practices. This means that, ironically, HMT has been a big driver of risk and waste across government.[2]

This is starting to change — in part thanks to a good relationship between HMT and DSIT — but very slowly. (Again, I am not saying that there are no good examples of digital-mentalities being driven by HMT, I am just describing the default at the median.)

One of the most powerful things we could do over this next 15 years is reverse this dynamic, so that HMT actively pulls Whitehall into the future rather than being a drag (e.g. getting back to the kind of reforming zeal we saw during the movement of Organisation and Methods in the late 1960s). But that’s a wider debate, with lots of interesting history and sociology to it, so let’s save that for another day.

That got longer and more catty than I’d intended. As ever, this is shared in a spirit of thinking in the open and I’d be interested in whether this resonates or jars with other people’s experiences. For more in a similar spirit, you could try my recent post, How to save bureaucracy from itself.

---

This was a personal post, but these themes are in keeping with an initiative I’m convening called Kinship Works. We support the shift away from bureaucratic approaches to something more human, which often means learning in more agile and adaptable ways. You can read about our work here and you can follow Kinship Works on Blue Sky or LinkedIn, or me on Blue Sky and Medium.

Footnotes

1. There are some half-good reasons that policy-making is still a very linear process, e.g. the legislative process is itself linear. But this is not really a proper defense because (a) there are still ways to incorporate a linear legislative process into a mixed-discipline iterative team (e.g. see the excellent work that has been done by the teams modernising Lasting Power of Attorney, which did precisely this). And (b) there are ways to legislate that are themselves more adaptive, and that allow iteration (e.g. we can make primary legislation less specific and more principles-based). And in any case, if we use our failure to modernise one part of the system to justify not modernising another, we will never get anywhere.

2. One reason the Treasury has lagged behind other departments, is that HMT is not a delivery department, so it hasn’t had the influx of design and digital staff which has modernised the culture of other departments. This has contributed to a situation in which HMT has less direct experience of contemporary delivery practices and the operating environment they need. This is one reason I think building up a cadre of digitally-native people in HMT, and ultimately moving these people quickly into leadership positions, is so important.