• Senior network engineer...
• BUSINESS SYSTEMS ADMINISTRATOR
• SERVICE MANAGER - GLOBAL IT...
• DESKTOP SUPPORT TEAM LEAD
• Multiple Project Management...

Outsourcers: Do they measure up?

Thomas Hoffman, Information Age

12/08/2003 15:58:00

Information is power, and that's especially true in IT outsourcing deals. So a growing number of IT executives are demanding that contracts include clauses that enable them to compare the contractor's costs and service levels with industry benchmarks a few years into the contract.

A bank, for example, might invoke its benchmarking clause three years into a data centre outsourcing deal to make sure that the vendor is providing mainframe CPU cycles at a reasonable cost.

The benchmarks might also compare the amount of downtime the bank is experiencing to industry averages, the size and duration of the batch processing runs, and other measurements to gauge the efficiency of the service provider.

If the benchmarking tests reveal that the outsourcer is charging considerably higher costs for its services compared with industry averages or is failing to meet other service-level agreements, the bank has several options. It could elect to renegotiate its contract with the outsourcer, invoke the financial penalties in the contract, bring the function back in-house or bid it out to other vendors.

"If we're going to sign long-term agreements with our supplier partners, we certainly have been asking for benchmarking tests to be implemented into the contracts so at points in time we can test the marketplace," says Cathy Brune, chief technology officer at Allstate Insurance.

One benefit is that if there's a sudden drop in market prices for a particular IT function, such as help desk support, "we're not saddled with contracts where we're being charged too much," Brune says. In addition, benchmarking agreements make her more comfortable in signing longer-term outsourcing deals. That way, "we're not in constant churn or constantly reworking contracts", she adds.

Brune's not alone. The Hackett Group, an IT advisory service that has a database of nearly 2000 benchmark participants, estimates that it has seen a 25 to 33 per cent increase in the number of benchmarking clauses written into outsourcing contracts over the past year, says director Scott Holland. "It's on the tip of everyone's tongue," says Holland.

The outsourcing contractors don't like benchmarking clauses, though they often feel forced to accept them. The problem is that an outsourcing contract is typically front-loaded, meaning the price is low in the first two years to give customers the upfront cost savings they're seeking, but then climbs in the third and fourth years so the contractor can make a profit.

Outsourcing vendors complain that benchmarking is unfair to them because the tests single out a period where they've built in a short-term "uplift" in prices to cover their costs, says Rod Hall, vice president of consulting at Compass North America. Moreover, the tests don't adequately examine costs and service levels over the lifetime of a contract, he adds.

A new wrinkle is the emergence of base-lining services. Whereas benchmarking takes a snapshot of service metrics, base-lining compares the average costs and service levels through the life of a contract.

Base-lining can also be used to compare the year-over-year costs and the efficiency of an internal IT department.

Vendors obviously see base-lining as the fairer approach. And some customers like base-lining because it gives them a better idea of the average costs they're paying over the life of a contract and how those compare with peer and vendor averages.

So if a vendor's costs and service levels are in line with industry averages, most customers would be content with that rather than going through the expense and headaches of negotiating a new contract. With base-lining tests, says Hall, "a customer can determine from start to finish if this is a reasonable contract or not".

The offshore component

Interest in base-lining has also heated up as more companies have started to send software development and maintenance work offshore to take advantage of lower labour rates.

But companies aren't simply using base-lining services for cost comparisons, says Michael Mah, a senior consultant at the Cutter Consortium. The metrics can also be used to keep an eye on offshore practices.

Because of the abundance of low-cost programmers overseas, some offshore providers that get paid based on the number of programmers they use will try to place extra bodies on a project to pump up their revenues, according to Mah.

"But if you double the labour, you won't double your productivity, and sometimes you end up introducing more software errors and running into higher costs" to fix them, he says.

That's why it's important for outsourcing customers to examine various metrics, including the cost per function point to produce or support code, the amount of functionality being delivered over a period of time, and the productivity of programmers and development teams, says Mah. The trick, he says, is knowing which metrics are really important.

Tips on conducting base-line tests

Keep accurate records on IT metrics -- beyond the cost of producing code. For example, calculate the rate at which problems are resolved, the defect rates of new systems and the amount of functionality that's being maintained. Benchmarking and base-lining metrics should be holistic and look at more than one or two dimensions of an IT function.

Seek out external sources of benchmarking or base-lining data, including companies that specialise in metrics research.

Remember that the metrics about your own IT shop are even more valuable than the benchmarking data about other firms. Internal measurements can suggest areas for improvement and identify trends in end-user demographics or systems utilisation, for example.