Advanced Introduction to Public Innovation, Edward Elgar, 2026

I’ve written a book trying to put in one place lessons about what works in public innovation, from structuring teams to finance, experiments to evidence, citizen engagement to technology, systems to adoption.  It draws on examples from across the world and across history and aims to help practitioners as well as researchers.  It will be published next year by Edward Elgar.

This is an extract from the introduction:

‘500 years ago innovation was not always welcome.  The English King Edward VI even issued a proclamation ‘Against Those That Doeth Innovate’.  But in the modern world innovation is seen as the fuel of progress, and it is encouraged everywhere: from business to the arts, education to architecture.   Government is no exception.   It’s long been obvious that innovation drives progress – albeit often with side-effects, and rarely in straight lines. But every improvement in our lives ultimately comes from innovation of some kind.

I’ve had the good fortune to observe public innovation both close up and from a distance.  I’ve had roles in the heart of governments, seeing the world top-down, and also working bottom up, with innovators in community organisations, startups and public agencies.  I’m lucky to have had quite a global perspective, working not just in the old democracies like the UK, US, Canada, Germany, France, India and Scandinavia but also in China, Latin America, the Gulf, and south Korea, where governments have often leapt ahead in the boldness of their initiatives. I’ve worked in local and national government, the European Commission and UN, and as an academic overseeing research programmes. And I’ve met hundreds if not thousands of impressive innovators, some within bureaucracies and others very much outside.

As a result, it seems ever more odd to me that governments were, and still are, so casual about innovation. They might have an occasional event, festival or project, and the best have programmes, labs and teams.  But none have anything as systematic as is normal in parts of business, or science, much of it funded by governments (in 1960 a third of all global R&D was funded by the US Department of Defense).  No wonder so many governments experience stagnant productivity and a widening gulf between public expectations and what they do.

Yet the public want governments to be involved: they don’t trust either business or scientists alone to handle innovations well.  Indeed, globally the public are more likely to believe innovation is mismanaged rather than well managed by a margin of two to one.  They want technologies to be assessed with an ethical lens, and they want businesses to partner with governments.[i]

In early 2000s, when I was running the UK Strategy Unit I co-wrote a report for the Cabinet Office surveying the options for doing innovation more systematically and I’ve subsequently written many reports, set up and run labs, funds, programmes and teams, and hosted big gatherings of hundreds of innovation labs.

This short book aims to synthesise a large body of action, experience, observation and analysis so as to reduce the need to reinvent wheels.  It describes the many methods, structures and tools that can be used, from labs to funding devices, experiments to scaling, including how to cultivate mindsets that are more conducive to innovation, and how to ensure innovation is aligned with political priorities.  Towards the end it makes a broader argument about ‘generative shared intelligence’ as a guiding philosophy for 21st century government in which innovation would fit more naturally.

Innovation can’t be turned into a precise menu.  It’s bound to be more fluid, open and unpredictable than that.  But the great insight of bureaucracies at their best is that if you make any task part of people’s jobs, whether it’s cutting child mortality or building roads, reducing online fraud or educating engineers, and give them resources to do it, it’s far more likely that you will succeed.  Exactly the same applies to innovation.

Why innovation matters now

A 2024 survey by Edelman in many countries showed that the public sees governments as neither very competent nor very ethical by comparison with business and NGOs.   This negative verdict points to the twin challenge facing governments: how to become both more efficient and more worthy of trust.  It’s a challenge that requires innovation on many fronts – from how they work to how they communicate, and from how they fix problems to how they engage the public.

But the challenge is intensified by the acute pressures many face.  The most obvious pressures are fiscal, as rising costs for services like care for the elderly and public goods like defense clash with reluctance to pay more in tax. The result for many is unsustainable growth of public debt.[ii]   But pressure also come from public expectations, particularly as commercial services become ever more efficient.  Add to these the broader pressures to tackle challenges such as climate change and migration and it’s not surprising that governments often feel stressed, struggling to keep up and adapt fast enough.  The widening gap between what’s expected of governments and their ability to deliver in a complex world, is one factor driving cycles of disappointment and distrust that easily become toxic.

According to conventional wisdom governments are bound to fail. They are congenitally incapable of changing fast enough. The novelist Joseph Conrad called bureaucracy the killer of “anything that breathes the air of human endeavour” and there’s a preconception that bureaucracies lack the competitive spur that drives businesses to create new products and services; that their rules squeeze out anything creative or original.  Their staff are penalised for mistakes but never rewarded for taking successful risks.   Their cultures are more like snails than gazelles, and they love tying things up in red tape, accumulating rules and procedures that make daily life, let alone creativity, all but impossible.  So, while businesses develop new chips, AI tools, aeroplanes and wonder drugs, the slow and stagnant public sector is perceived as acting as a drag on everything else.

This account is commonplace.  At times it’s accurate. But it is at odds with the history of innovation. Two of the most profound innovations of the last 60 years were the internet and the World Wide Web. Both came out of public organisations: the US Defense Advanced Research Projects Agency (DARPA) in the first instance and CERN, the European Organization for Nuclear Research, in the second.

Contrary to popular opinion, business was not particularly innovative until at least the late 19th century. Instead, for most of human history the most important innovations in communications, materials or energy typically came from wealthy patrons, governments or from the military.[iii]

Indeed, many of the most important innovations that underpin human civilization originated with governments.  Writing and counting were invented in ancient Sumeria not by creative entrepreneurs but by the world’s first ever governments that needed to track movements of grain, or the work done (mainly by slaves) in textile production. 

In the 19th and 20th centuries much innovation was driven by the needs of the military. It was essentially focused on the priorities of defense and attack.  All of the methods that were later characterized as ‘mission-oriented’ innovation became common before WW1, when the UK, Germany and France, in particular, competed in a furious arms race around ships, tanks and later aircraft, and again during the extraordinary surge of innovation in the US and elsewhere during WW2, which created nuclear bombs, radar and jet planes, and then again during the Cold War.

Even today, the caricature of public agencies as stagnant enemies of creativity is disproven by the innovation of thousands of public servants around the world who have discovered novel ways of combating AIDS, promoting fitness, educating children, vaccinating vast populations or implementing new methods like intelligence–led policing or auctions for radio spectrum.

The COVID-19 pandemic was a good example: prompting governments to innovate rapidly and radically, from moving all education online to providing welfare support, introducing mass testing programmes as well as mass vaccination, to new models of procurement such as Costa Rica’s online platform to ensure transparency (and avoid the scandals that hit countries such as the UK, where members of Parliament were seen to be issuing contracts worth millions of taxpayers money to their personal contacts). The invasion of Ukraine in 2022 was another prompt for innovation – with the country’s digital platform for public services, Diia, quickly re-purposed to allow citizens to report on the location of Russian tanks.

All of these examples disprove the lazy prejudice that sees governments as incapable of innovation. But there are good reasons to doubt the public sector’s ability to innovate enough. Talk to innovators and they usually say that they have succeeded despite, not because of, dominant structures and systems. Too many good ideas are blocked, filed away or simply forgotten. Public services remain poor at learning from better models – even on their doorstep – and only a handful of governments have any roles, budgets or teams devoted to innovation in their main areas of activity: welfare, security, health or the environment.

In contrast, over the last century mature innovation systems have taken shape in science and business. In science, both the public and private sector invest billions.  The difficult task of turning scientific insights into useful products was long ago taken away from lone inventors in garden sheds and placed at the heart of great corporations and great public laboratories. Most developed countries spend between 2 and 5% of GDP on R&D, primarily because they are convinced of the benefits, both to their prosperity and their security.

Meanwhile in business the 20th century brought the creation of in–house labs and Research and Development (R&D) teams, and in recent years more widespread use of open innovation, user innovation, service innovation, design methods and agile methods for software. These all benefited from significant financial investment – with investors fully aware that any company worth its salt in biotech, pharmaceuticals or AI needs to invest a high proportion of turnover in innovation, and there is now a vast body of research and data on the state of innovation globally[iv].

But in the public sector there are few comparable models.  The good news is that there are far more examples of public innovation globally than ever before, with pioneers in every continent, and arguably more creative energy now in Asia, sub-Saharan Africa and Latin America than in the parts of the world that used to think of themselves as the frontiers that others had to copy.   

But the less good news is that innovation remains under-appreciated.  A 2025 OECD survey of over 50,000 public employees showed that although ‘innovation climate is the strongest driver of perceived organisational performance’ in the public sector, ‘less than half of employees view their organisations’ innovation climate favourably’. [v]   It remains rare for a public organisation to be able to give a coherent account of how it innovates, and even rarer for the media to ever ask. The idea that the leaders of an organization should be held to account as much for what they are doing for the future as for what they are doing in the present, is unusual in most democracies.   So perhaps it’s not surprising that few agencies or ministries have roles, budgeting or assessment methods focused on innovation.  

Instead, new ways of doing things tend to be managed in a much more arbitrary fashion, which leads to a messy mix of the top–down imposition of unproven new ideas, creative but dis-organised local innovation, and reliance on quasi–markets without the R&D necessary for radical innovation.

Perhaps the fundamental reason why innovation is hard in the public sector is also why it’s hard in any other sector. There are big returns to be gained from innovation for whole systems, and these drive most productivity growth, and indeed most progress. Yet the rewards for any individual unit or organisation – such as a school or hospital – are much lower, while the costs of running experiments many of which are likely to fail, is bound to be high, making it more rational to focus on adopting the ideas developed by others.

This imbalance between system-wide benefits and organization-level risks is of course why governments subsidise innovation in science and business – through a wide array of funds, tax credits and procurement devices. It is also why investors – like venture capital – spread their risks, accepting that most investments will fail.

What’s odd is how rare it is for the same logic to be applied to the public sector itself. Yet here too the cost of experiments and trials need to be pooled, and the benefits need to be pooled too. The design challenge for the public sector is how to share both risks and rewards, and, like other sectors, to organize innovation through portfolios, since some will succeed but many won’t[vi].   How that can be done is the focus of the next few chapters.  In them I try to answer some of the recurring questions that I have heard repeatedly over many decades working in and around governments:

·      How to stop government and political environments from crushing imaginative risk taking?

·      What structures work best for accelerating innovation – labs, teams or nothing?

·      How should governments identify ideas that work – and then spread or scale them?

·      How can governments align their diverse ministries and agencies around common goals?

·      How should governments remove the barriers that often seem to stop imaginative people from doing innovation?

·      How can innovation teams avoid becoming ever more detached from the ‘mother ships’ of the wider government?

·      How should the public, or frontline staff, be involved in the work of innovation?

·      When do markets and competition have a role – and when don’t they?

·      How similar or different is innovation in different parts of the state – from health services to policing, the military to tax collection?

·      How useful are the fashions in government which often claim to be universal answers – from new public management and markets to missions, behavioural insights and design thinking?  Are they relevant to 5% or 50% of what government does?

·      What is the right scale for innovation? When it is best done locally, at a national level, or through collaborations between many cities or nations?

·      And finally, how should we judge if any government is innovating enough (or sometimes whether it is innovating too much?)

[i] Edelman (2024) ‘Innovation: Destroyer or Saviour? Edelman Trust Barometer’. Available at: https://www.edelman.co.uk/insights/innovation-destroyer-or-saviour-worlds (Accessed: 17 July 2025).

[ii] Robinson, M. (2020) Bigger Government: The Future of Government Expenditure in Advanced Economies. London: Arolla Press.

[iii] For a good recent reminder of just how big a role governments played in driving innovation see Frey, C (2025) How Progress Ends: Technology, Innovation, and the Fate of Nations, Princeton University Press 

[iv] For example, in the annual Global Innovation Index:  https://www.wipo.int/web-publications/global-innovation-index-2024.  It’s notable that the GII has struggled to ever cover public innovation with anything like the data it has for business, science and technology. 

[v] OECD (2025), Workforce Insights from Central Governments: Findings of the 2024 OECD/EU Survey of Public Servants, OECD Publishing, Paris, https://doi.org/10.1787/2f9080b1-en. 

[vi] In my experience most public innovation is organized through portfolios. But this is one of many truths which is repeatedly rediscovered, see for example: https://gds.blog.gov.uk/2025/08/18/the-prime-ministers-ai-exemplars-programme-a-portfolio-approach-to-public-sector-innovation/