This is the second in a series of postings about my thoughts from sessions that I attended at the Information Technology Alliance’s Fall Collaborative (<—I love that word) held in Palm Springs.

Presented by Jeanne Urich of SPI Research. The highlight of my ITA experience occurred during this session when Jeanne acknowledged that a conversation that the two of us had at the spring ITA meeting has influenced her thinking and that she now believes that fixed price agreements are better for customers and consultants. Now to convince her to give up on these dozens of benchmarks! I think this will be a much harder task.

The study consists of 175 metrics, 20 of them deemed key performance indicators. The problem in my mind is that all, save one of the 20 are inwardly focused on the firm, even those in the “Service execution” and “Client relationship” areas are firm-based.

The problem as I see it with most benchmarking is that it focuses us on all the wrong things: the past rather than the future; internal rather than the customer; efficiency rather than effectiveness.

For example, one “customer” metric was about project completion success. The question was asked of the providers, “What percentage of your projects are completed on time and on budget?” The average answer was 74 percent. This more than doubles the number (35 percent) according to a study done by The Standish Group who asked the same question of customers. Needless to say, I side with customers on this one. Again, to her credit Jeanne acknowledged this flaw.

Many of the other metrics are based on the false premise that value delivered is equal to rate times hours, aka, the labor theory of value. This theory is demonstrably false and belief in it has been proven to cause harm.

I have sat in on countless benchmarking session and the reactions of the attendees is always the same: a) if they are doing better than the benchmark, they think they are OK and do nothing, and b) if they are worse than the benchmark, they dispute the data and still do nothing. Path (b) actually happened twice during the session!

My beef with all benchmarking in business is that while it attempts to appear scientific, it is not even close. To her great credit Jeanne is very careful about saying that these metrics are correlative not causal. Unfortunately, most people do not understand this distinction and are lulled into the illusion of control via data.

The findings are always similar and in many cases are nothing more than a bunch of truisms:

• Firms who market well have higher revenue. (Yet, marketing spend, even among top firms, is less than average across all industries.)
• Firms who close more business (win to bid ratio) are more successful. (The question is, what do they do differently that allows them to have a higher win to bid ratio? Win more or bid less?)
• Few firms grew revenue in 2009.
• Clear vision and strategy and taking care of your people are important in professional firms. Firms that focus on culture are rated as better places to work. (What kind of culture?)

Conclusions are almost always the same - “Increase revenue, lower discretionary spending.” Always a good idea.