When ROI isn't worth the spreadsheet it's written on

Statistics are awkward things. Consider the following: when does 36 per cent stop being a small fraction and become a significant proportion?

That's the question left hanging from the most recent NOP/silicon.com Technology Confidence Barometer. Among all the findings the most intriguing fact is that just - perhaps that should be as many as - 36 per cent of UK companies conduct return on investment (ROI) measures following major IT projects.

For some, like NOP's own Michael Freedman, the figure represents an abrogation of duty. "You would have thought given the state of the economy many more [IT directors] would be spending their companies money more wisely," Freedman said.

For others the failure, or rather the active choice not to conduct these measures post-project, makes perfect sense.

"Most people conduct analysis to get projects off the ground," said David Rippon, chairman of IT director organisation Elite and former IT chief at Landsecurities. "No one I know will go back and say 'Did we make it?' No one. Once they've made the commitment they don't want to look back. Frankly, they don't want to be embarrassed."

On the outside, however, ROI seems a compelling model. It tells you how much your current systems cost to run on a monthly or quarterly basis, how much the new technology will cost to buy, how much the new system running on the new technology will cost you on a monthly or quarterly basis and at what date in the future savings on running costs will outweigh the money you spent on the new technology.

Simple. Not so, says Rippon. "At least a year will have passed between a project beginning and ending. By that time the business environment will have changed. The key assumptions will have changed. The original ROI story no longer makes sense. You're in a new status quo."

According to Dale Vile, service director at analyst house Quocirca, ROI measures do make sense but only for specific projects, what he calls "part of the furniture, infrastructure" projects. "Here it's about making sure you're getting the right return, so ROI is very firmly the right way of approaching it."

However, Vile says, companies shouldn't get caught up in the ROI trap when planning "top line, speculative" roll outs. These are the projects where IT is being deployed to gain competitive advantage. And despite the ravaged economies across the continent these roll outs roll on. "If you speak to CIOs around Europe, this kind of visionary project still exists despite the downturn in the economy. Yes, it's being challenged but it doesn't need to be quantified," he said.

None of this will stop big vendors marketing their products with ROI at the fore. Cisco, IBM, Microsoft et al continue to bang the drum and are acknowledged for doing so according to the NOP research. It's their way of appealing directly to company bosses who have seen technology as a drain on resources and will always be ready to hear the cost reduction argument.

David Rippon is having none of it. He added: "In my opinion vendors shouldn't be allowed to put forward ROI cases. IT should be left to the in-house people, the IT department and the bean counters. If vendors do want to offer ROI cases they should be made to do it contractually. Let's get it in writing."

Now what are the chances of that happening? Pick a percentage - but make sure it's a low one.