The Brampton Factor: Why big government IT projects fail

A £500m prisoner-tracking project failure barely causes a ripple these days. We're so hardened to government IT problems we no longer see the underlying causes, says Martin Brampton.

I've been feeling sorry for the government. Well, perhaps not very sorry. But it is easy to criticise government attempts at implementing major IT systems. If anything, things seem to be getting worse, with frequent reports of disasters. So it seems worth thinking again about the problems facing government IT.

First, a quick review of the state of things. The Guardian recently totted up abandoned projects and came up with a figure of £2bn since 2000. That could well be an underestimate. More likely it is a fraction of the money wasted.

After all, in every sector people go to great lengths to avoid admitting that a project is in such a bad state that it should be abandoned. How many times has the argument been made: "It has cost so much, it must be worth something"? So the projects that failed to deliver benefits commensurate with costs are likely to have added substantially to fruitless expenditure.

In fact this was pretty much confirmed by Joe Harley, programme and systems delivery officer at the Department for Work and Pensions, who was quoted saying only 30 per cent of projects are successful.

With government IT expenditure running at about £14bn, that means about £10bn is spent every year on unsuccessful projects. With figures like that, the £1.5bn spent on maintaining old systems begins to look pretty good value.

So why feel sorry for the government? Partly because all these costs eventually come out of our pockets but also because the government seems to be the prisoner of a number of apparently sacred beliefs and some commonplace errors.

Actually solving the problem may well be impossible within those constraints, simple though they are. Let's look at three of the issues:

1. Fact - big decisions get less attention than small ones.
2. Belief - managing IT doesn't require specialist skills.
3. Belief - specialised functions like IT should be outsourced.

These are widely held views but I contend that they have a particularly damaging effect on government IT.

The first item is one of those evident truths everyone knows, yet no one seems able to tackle. Anyone who has sat on a committee is aware that major investment decisions are often made with minimal discussion, while questions about replacing the office carpet provoke interminable debate.

Sometimes this phenomenon is associated with people who have a vision for the organisation, which renders detailed discussion unnecessary. It particularly affects government because quite a few of the decisions on IT have been very big indeed.

A prime example is the revamping of NHS IT. The project of centralising the entire record system seems to have been started on little more than a moment's thought by Tony Blair.

His inspiration was something like "wouldn't it be good if we could bring up patient records at the press of a button". Anyone who has been in IT for any length of time will be familiar with the press-of-a-button syndrome and will cringe every time it crops up.

Naturally, people who make decisions such as this prefer to describe it as bold innovation that breaks with the constraints of a fusty past. In a word, modernisation.

But all too often, the result is an addition to the statistics of abandoned or inadequate projects that eat through money without yielding comparable benefits.

The second belief is widely promoted nowadays. Naturally, it is true senior managers need to have skills that have nothing to do with IT, such as the ability to organise and to motivate people.

Unfortunately, though, this principle all too easily translates into a situation where actually knowing about the challenges involved in building IT systems is regarded as something trifling that can be safely ignored by true leaders.

This is encouraged by the half-true belief that IT people just raise all kinds of objections to innovation and stand in the way of progress. But this is natural and to be encouraged. In 1733 Alexander Pope wrote that "hope springs eternal in the human breast" but qualified it by implying optimistic beliefs were forever projected into the future.

People who have to build and deliver IT systems naturally start to think about the difficulties faced by a new project, many of which have been predicted from the moment an ambitious project starts.

So while there may be a case for managers of vision and skill who lack IT awareness, real problems arise if they become disconnected from the knowledge and experience needed for actually implementing successful systems.

Given the current spate of failures, one might well feel justified in concluding that the problem is not so much a lack of knowledge of the business so much as a lack of ability to assess the feasibility of projects and then to deliver those that offer big returns for small outlays.

And then there is outsourcing. Not so long ago industrial conglomerates such as the Hanson Group were greatly admired and rode high on stock exchanges.

Nowadays, fashion has swung in the opposite direction, and companies are expected to concentrate on a narrower range of core activities.

Everything else is put out to specialists - that includes IT. This creates huge problems for government, which is obliged to put all large purchases through a complex regime of competitive tendering.

While the principle of fair competition for government work is sound, the practical result is that it can easily take two years to start a project.

Sometimes, people ask why government keeps going back to the same providers that were involved in past failures. Often, the answer is simple.

To speed up purchasing, government bodies are obliged to set up framework agreements that allow a whole series of projects to be bought, without specifying the projects at the time of tendering.

So if a new project is to be started speedily, it often has to be done by a contractor that is already in a relationship with government.

Results certainly seem to justify scepticism about whether outsourcing is really a practical way to secure good government IT. Especially as there is no hard evidence for the widely held belief that government organisations are necessarily less competent than their private sector counterparts.

But this principle is so entrenched that it seems no more likely to be abandoned than the belief in non-specialist IT managers or the practice of taking huge decisions on a whim.

It seems that government is the prisoner of impenetrable delusions and the best we can hope for from government IT is very expensive and rather cynical amusement.