SUBSCRIBE BY RSS
There is an active major-league pitcher with a career win-loss record of 59-68, and he is a mortal lock for the Hall of Fame. Now, if you're a real baseball fan, you immediately realized that I must be talking about a relief pitcher, and probably a closer. And you'd be right. Trevor Hoffman, the closer for the Milwaukee Brewers, has 591 saves to go with that seemingly unimpressive W-L record, more than anyone in major-league history.
You see, wins are a useless statistic for a relief pitcher. Whether a relief pitcher wins or loses the game often depends on what other players (on either team) do. In fact, it's possible for a relief pitcher to win a game without throwing a single pitch. (This has happened 15 times since 1957, according to the hardball geniuses at Baseball-Reference.com. The scenario involves throwing out a baserunner to end an inning, having your team take the lead immediately after, and then getting relieved by someone else.)
The win-loss record for starters is a more-useful statistic, but it is still seriously flawed. For example, on October 2, Adam Wainwright was denied a chance to be the Majors' only 20-game winner in 2009 when the St. Louis bullpen imploded. He pitched six strong innings, striking out eight and allowing just one run. Then the first two batters reached in the seventh. The playoffs were around the corner and Wainwright had already thrown 90 pitches. So manager Tony La Russa (a lawyer, by the way) replaced him with Kyle McClellan, who promptly allowed Wainwright's two runners to score and let in four more of his own. As a result, Wainwright finished with 19 wins, which probably cost him the National League Cy Young Award. It wasn't his fault that he didn't get that twentieth win, still considered a magic number.
What is unusual is that both Cy Young winners announced last week had even fewer wins than Wainwright. Kansas City's Zack Greinke won with just 16 wins, and Wainwright's teammate Tim Lincecum won with 15 wins, the lowest win total ever (for a starter in a nonstrike year). In Sunday's New York Times, Tyler Kepner discusses this novelty in a piece called "A New Generation of Statistics Redefines Baseball." (Actually, the article appears online with a different title: "Not Your Grandfather’s Stats: Baseball Redefined.") Kepner makes the point that the sportswriters who vote for the awards are becoming savvier about which statistics are better indicators of a player's performance. Win totals have long dominated the Cy Young voting, and Kepner and ESPN.com's Rob Neyer point out that Wainwright would have beaten Lincecum if the Cardinals bullpen had held on to get him his twentieth.
Kepner compares the 1990 seasons of Roger Clemens and Bob Welch. Clemens pitched substantially better according to the more-meaningful statistics: many more strikeouts, fewer walks, far fewer home runs, fewer baserunners per inning pitched, much lower OPS against (that's opponents' on-base percentage plus slugging percentage, for you nonseamheads). But Welch won 27 games, six more than Clemens, and thus won the American League Cy Young Award.
The point here is that if you're going to measure performance, measure the things that matter most. (Brace yourself for the segue to law firms.)
So how do law firms measure associates? By the number of hours they bill, of course. Oh, sure, they have evaluations, and 360-degree reviews (whatever that means), and other methods of assessing performance. But the one number that matters — the win totals for associates, if you will — is hours billed for the year. (If this were baseball, we'd call it "HB." Of course the Times would call it "H.B." with the periods, just like they write "E.R.A." and "R.B.I." That bugs me.) This metric is so important that many BigLaw firms have a threshold: if you don't bill 1,850 hours, no bonus for you.
[It's worth noting that unlike baseball teams, law firms usually don't pay their associates based on performance. At least at the larger firms, they pay their associates (in lockstep) based on years of service. Under that system, Alex Rodriguez (who debuted in 1994 — call him a 16th-year associate) should get less than Cliff Floyd (who debuted a year earlier). Of course, the Yankees paid A-Rod $33 million for 2009, while the Padres paid Floyd a mere $750,000.]
And what's the most important measurement for the firms themselves? Profits per partner. This is how they keep score, with the numbers voluntarily reported and shared with The American Lawyer. (Why law firms, which are all privately held, would share their profit numbers is beyond me.)
But how do these "win totals" — HB and PPP — measure real performance? In other words, the performance that the clients care about. I submit to you that like relief wins, they don't.
It's difficult to measure performance when you're talking about professional services. Lawyers aren't playing baseball. But it's not impossible. Value-pricing guru Ron Baker has an entire book devoted to key predictive indicators, or KPIs: Measure What Matters to Customers: Using Key Predictive Indicators. In it, Ron talks about the sort of KPIs that law firms could use, such as turnaround time (which is basically the opposite of billable hours, if you think about it), innovation sales (selling new services), customer loyalty (retention rates), share of customer wallet, and others. You could learn a lot about how to run your law firm better — buy it here.
Lawyers: stop worrying about relief wins, and start measuring what actually matters to your customers.
[Update: Ed Kless, who comments below, has a great sidebar in Ron's book that discusses the topic of using the right stats in baseball. A must read.]