But the problem with such claims is that there's very little hard data to conclusively support them. Admirers of computerization rely primarily on anecdotal case studies as evidence of astounding productivity gains. But such stories rarely hold up as independently verifiable proof. Such claims are almost always authored by public relations departments of vendors, consultants or firms boasting about their high-tech prowess based on well-written anecdotes.

Some academics have generated a small number of studies trying to show that spending on computers has been extremely profitable. But the studies are flawed because they base their findings on questionable government statistics [ Fuzzy Math, Jan. 8] and rely on arcane econometric formulas.

Published financial statements from commercial banks offer perhaps the best data for tracking recent changes in productivity. Only banking firms publish detailed data about overhead costs, payrolls, employment and equipment purchases. Banks are also unique in that their statistics reflect their total economic value added. Because of tight regulatory controls, their reporting practices are more uniform than those in other industries. Banks also spend the most of any industry on computers per employee: more than 20% to 30% of payroll. Such high spending rates should give computers every opportunity to demonstrate improved productivity results, since increased computerization would figure as a critical influence in operating results. And showing such a favorable impact of computerization on productivity may indicate that IT could be profitable for other industries.

To find out if banking productivity has made recent gains, I collected financial statistics for 497 U.S. commercial banks. I excluded 36 that had drastically reduced employees through mergers, leaving 461 firms employing more than 1.2 million people.

The principal indicator that would signal productivity gains would be the ratio of staff payroll to revenue. Effective IT would make it possible to generate more revenue for the same amount of payroll dollars. A decline in this ratio would confirm the widely believed assumption that capital spending on computers is always more profitable than paying staff. But the ratio didn't decline - in fact, it showed a slight increase, from 19.9% in 1995 to 20.1% in 1999.

Then there's the ratio of noninterest costs to revenue, which reflects the banks' median overhead cost ratio - the sum of administrative, data processing and managerial expenses - divided by total revenue from fees and interest. After a decline from 1995 through 1997 (39.1% to 37.2%), it rose to 38.3% in 1999, showing only an unimpressive improvement over five years.

The ratio of equipment cost to staff payroll (down from 16.3% in 1995 to 15.9% in 1999) represents only an approximation of IT hardware spending. This indicates that banks continued to spend heavily on computer equipment and suggests that policies of devoting exceptionally high levels of capital investment on hardware were necessary.

This confirms that the much-debated "computer paradox" that MIT professor Robert Solow wrote in 1987 ("You see computers everywhere except in the economic statistics.") hasn't been banished. None of that should be discouraging, since there are quite a few banks that deliver staff payroll-to-revenue improvements materially better than the median values. Some 93 banks delivered better than 10% improvements in that ratio. The extent to which such superior performance can be directly attributed to computers can be understood only by studying the quality of leadership, especially in cases where superior management - and not technology - made the difference.

Be very skeptical of general pronouncements about universal benefits of computerization.