Defensive Regression Analysis

Editor’s Note: Last year, Michael Humphreys introduced a revolutionary new fielding statistic, called Defensive Regression Analysis (DRA), which represents an entirely new way of thinking about fielding stats. DRA uses stats that are available throughout baseball history so it can be used to evaluate fielders of any era. We consider it a significant improvement over fielding Win Shares.

The original DRA article (pdf) submitted to Baseball Primer is now available. Also, Web Archive has the original Primer articles, with the correct formatting — Parts One, Two, and Three.

In this series of three articles, Michael will explain DRA, use it to evaluate major league fielders from 2001-2003, and compare it to zone-based systems such as Zone Rating and Ultimate Zone Rating in order to verify its accuracy.

The second article will be published tomorrow, with the third published the day after.

Introduction and Summary of Results

A. General Introduction to Fielding Stats

Alan Schwarz says in his wonderful new book, The Numbers Game: Baseball’s Lifelong Fascination with Statistics, that “n some ways, fielding is baseball statistics’ holy grail.”

Baseball analysts have been rising to the challenge, and the quality of fielding information available to fans has never been higher, particularly for the 2001-03 seasons.

Back in the 1980s, Dick Cramer helped create the first and most widely used proprietary record of the number of batted balls hit reasonably close to each fielding position, and the percentage rate at which individual fielders turn such batted balls into outs. The resulting Zone Ratings (ZR) eventually found their way to the public (though not the underlying data, except at a price and with licensing restrictions); during 2004, ESPN posted 2001-03 ZR on the Internet for all major league players.

Beginning in 2001, Mitchel Lichtman purchased zone data even more detailed than what is used for publicly posted ZR, converted it into runs saved or allowed (“runs saved”) ratings, and published his results (Ultimate Zone Ratings, or UZR) for the 2001 through 2003 seasons (as well as the 1999 and 2000 seasons) at Baseball Think Factory (BTF), as well as Tangotiger’s site.

Tom Tippett at Diamond Mind (DM) has posted “Gold Glove reviews” for the 2001-2003 seasons, in which he uses high-quality zone data, traditional data and even videotapes of performance to provide thoughtful verbal evaluations of the best fielders and a few notably bad ones.

In 2002, Bill James published his latest fielding evaluation system (Fielding Win Shares, or FWS), which, when introduced, was the best publicly disclosed and reproducible method for rating fielders throughout major league history. Last year, Studes here at The Hardball Times began posting FWS, and Total Baseball updated Pete Palmer’s Fielding Linear Weights (FLW) to reflect many of the ideas of FWS. At this point, FLW is probably more accurate than FWS at most positions, but only because it has incorporated Bill’s ideas.

In 2003, David Pinto introduced a new system (a Probabilistic Model of Range, or PMR) based on proprietary play-by-play data similar to (yet clearly different from) zone data, and has published ratings for 2003 and 2004.

Sometime in the past few years (I’m not sure when), Baseball Prospectus, on its website, posted individual runs saved ratings throughout major league history, based on Clay Davenport’s Davenport Fielding Translation (DFT) system, which converts traditional fielding statistics into runs saved ratings, but which was never fully disclosed (at least in a reproducible way), and which is currently described in only general terms. As I will explain below, I believe DFT currently provides the most accurate fielding evaluations throughout major league history. I also believe that DFT has probably been improved by incorporating Bill James’ ideas.

B. Introduction of DRA in Late 2003

In November 2003, I published a 3-part article (Parts One, Two, and Three) at BTF (then called Baseball Primer) introducing a new pitching and fielding evaluation system I had developed. I called the system Defensive Regression Analysis (DRA) — I apologize for proliferating acronyms and abbreviations; but they do save space and time — because it is based on the statistical technique known as “regression analysis.” Regression analysis has long been used to generate equations for evaluating batters similar to Pete Palmer’s Batting Linear Weights equations, but has never been used before (or at least so comprehensively) to evaluate pitching and fielding. Fans who might be put off by statistical jargon can think of it as Defensive Runs Analysis. The acronym rhymes with ERA, so that should make it easier to remember.

DRA is the first pitching and fielding evaluation model that systematically works through and determines the statistically significant relationships between traditional, publicly available pitching and fielding statistics and the actual number of runs allowed by a team. DRA yields formulas just as simple as the well-known one-line FLW equations in Total Baseball, that enable us to estimate the number of runs saved or allowed by pitchers and fielders (a) relative to the league average and (b) independently of each other. DRA is designed to be fundamentally accountable—the pitcher and fielder ratings add up to a team DRA rating (i.e., an estimate of the number of runs the team should have allowed). Such estimates are as or more accurate than Batting Linear Weights or Runs Created estimates for the number of runs a team should score.

This article will try to show, through a careful consideration of the published 2001-03 results of a multitude of fielding evaluation systems using proprietary zone-type data, that DRA has essentially solved the problem of evaluating fielding using traditional fielding statistics. The two simplest criteria for determining the accuracy of a fielding evaluation system that uses only traditional statistics are (a) its correlation with the best zone-based systems (i.e., how well it estimates the relative quality of fielders) and (b) how close the standard deviation in its runs-saved ratings is to the standard deviation of the best zone-based ratings (i.e., how well it estimates the absolute impact of fielders). DRA ratings at all positions (other than first base, for reasons which I’ll explain) have an overall 0.8 correlation with — and almost exactly the same standard deviation as — ratings derived from the best proprietary zone-based data.

A Hardball Times Update
Goodbye for now.

Based on its correlation and standard deviation output, DRA is significantly more accurate than FWS and FLW, and meaningfully (not terrifically, but meaningfully) more accurate than ZR (which is based on proprietary data) and DFT (which is based on a proprietary methodology). The DRA methodology is proprietary now as well, but the general principles and most of the techniques used in DRA are disclosed in the November 2003 article, and sometime this summer, I plan to complete a draft of a book that will reveal the method and formulas in complete detail. (This article is a first draft of one of the technical appendices; I fully intend to make the main part of the book more accessible, but I trust that The Hardball Times readership will appreciate the detail here. I hope you will find this the most careful assessment yet published of any fielding systems.)

When DRA becomes “open source,” baseball fans will not only have a way of generating good historical ratings for themselves, but also a tool that, combined with ZR output posted on the ESPN website, will enable them to produce with minimal effort contemporary fielder runs-saved ratings having close to a 0.9 correlation with proprietary zone-based ratings. Fans will be back to having information nearly as good as what the teams use. (In the meantime, this article actually provides the first objective evidence that DFT ratings available for free online are usually quite good, particularly at the most important positions: shortstop, second base, and centerfield.)

DRA will be (as DFT is today) an especially valuable resource for fans as alternative sources of information disappear. Two of the best sources of fielding information will apparently not be publicly available for seasons after 2003. Mitchel Lichtman has been hired by the St. Louis Cardinals and will no longer be publishing UZR. It appears that Tom Tippett won’t be publishing a Gold Glove essay for 2004; he used to get them out by the December after each season, but no 2004 essay has appeared as of February 2005.

C. Summary of Results of DRA Test for 2001-03

This article is being published in three parts. So that you won’t be in complete suspense, I’ll now post the summary results of the “test” of DRA against UZR, as well as corresponding results for ZR and DFT. Part 2 of this article will provide background as to the whys and hows of the UZR, DRA, DFT and ZR test. Part 3 will provide the complete results and some brief further explanations regarding individual players.

The chart below shows the average UZR, DRA, ZR and DFT runs-saved ratings over the 2001-03 time period for players who played at least 130 games at a single position for at least two of those seasons (without splitting seasons between teams) (a “Full Season”). Catchers are not included because I did not have a complete sample of UZR ratings for them. The DFT ratings are the RAA2 ratings available online for the player when playing his main position. ZR has been converted into runs saved by calculating marginal plays made and multiplying that value by the sum of the approximate value of the out created (about 0.3) and the hit saved (about 0.5 to 0.6, depending on the position). If a player played only two Full Seasons, the average rating shown is for only those two Full Seasons. The “Yrs” column indicates if the rating is based on three Full Seasons. As will be explained later, the sample does not include players whose DRA rating differs from UZR by more than one UZR standard deviation if non-UZR zone-type sources (DM, PMR, ZR) and other reliable sources of information agree more with DRA than UZR. I’ll explain each of these cases in Parts 3 and 4.

Also included is “DRAZR”, a weighted average of .67*DRA + .67*ZR, which shows a very, very good match with audited UZR. Once DRA is open source, fans will easily be able to reproduce “DRAZRs” (rhymes with “razors,” as they’re so sharp; and “lasers,” because they’re so precise) and evaluate contemporary fielders with confidence. DRAZRs work because DRA and ZR measure different things with different data, so they complement each other.

Average, standard deviation, and correlation numbers are provided at the bottom of the chart, broken down into three categories: (a) all positions except catcher (“3456789”), (b) all positions excluding first base (“456789”), and (c) all positions excluding first and right field (“45678”).

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 6        Aurilia         1   -9    5    5   -3
 6    3   Cabrera        11   13    5   13   12
 6        Cruz           -5  -10    3   -4   -4
 6        Furcal          7    5   -5   -4    0
 6        Garciaparra     9   12   -3   -7    6
 6        Gonzalez, A.    2    2    3   -6    4
 6    3   Gonzalez, A.S.  8    8    5   10    9
 6        Guillen         3   -3    1  -11   -1
 6        Hernandez      16   10    8    7   12
 6        Jeter         -25  -22  -19  -20  -28
 6        Ordonez        -1   -1    4    6    2
 6    3   Renteria        7    7    5   -3    8
 6    3   Rodriguez       9   -5   12   10    4
 6    3   Tejada         -1    6   -8  -11   -1
 6        Vizquel         8   -4   -4    9   -5
 6        Wilson         -8    3   -1    2    2

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 4        Alomar        -13  -13   -9    2  -15
 4    3   Anderson       -4    2    8   -9    7
 4    3   Boone          14    9    4    7    9
 4    3   Castillo        0    1    6   -2    5
 4        Grudzielanek   11    4    5   -2    6
 4    3   Kennedy        21   15   13   11   19
 4        Kent            7    4    2   12    4
 4        Rivas         -20  -22  -14  -19  -24
 4    3   Soriano        -4  -15   -8  -11  -15
 4        Vidro           1    0   -5   -5   -3
 4        Vina            4  -10    4   -6   -5
 4        Walker         -9   -9   -5  -12   -9
 4        Young         -11   -8    5   -3   -1

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 5        Alfonzo        -3    2    2    0    3
 5        Batista        -3    3   -4   10    0
 5        Beltre         16    1    5  -10    4
 5        Castilla        1   -4    2   -8   -1
 5    3   Chavez         17   12    8   12   13
 5        Glaus         -10   -2   -1   -4   -1
 5    3   Koskie         11    9    7   11   11
 5        Lowell         -6   -3   -2   14   -3
 5        Rolen          18   13    6   10   13
 5        Ventura        19   10    7    3   12

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 8    3   Beltran         6    6    8    9    9
 8    3   Cameron        28   24   10   12   23
 8        Damon          13   -1    5    4    3
 8        Edmonds        -4    5    4   10    6
 8        Erstad         42   36   13   20   32
 8    3   Hunter          8    0    4    3    2
 8    3   Jones          15   24   -1   17   15
 8        Wells          -2  -14    5    0   -6
 8        Williams      -20  -11  -14  -11  -17

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 7    3   Anderson       -2    4    6    6    6
 7        Bonds          -8   -6   -7   -4   -8
 7    3   Burrell       -11  -12    2    0   -7
 7    3   Gonzalez       11    3    5    5    6
 7        Jones, C.       2   -5    2   -6   -2
 7        Jones, J.      14   12    9    5   13
 7    3   Lee             4    0   10   -3    7

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 9    3   Abreu          -7   -4    5   -6    0
 9    3   Green         -13    9    5    6    9
 9        Guerrero       16    0    4   -6    3
 9    3   Ordonez        -7    1    5    2    4
 9    3   Sosa           -6   -2   -1   -4   -2
 9    3   Suzuki          7   15   -2   12    9

Pos  Yrs  Last          UZR  DRA   ZR  DFT  DRAZR
 3    3   Bagwell         7    0   -6   -5   -3
 3        Casey           5  -11   14   -3    2
 3    3   Delgado        -2    1   -1   -3    0
 3    3   Helton         22   11    7   16   12
 3        Konerko       -10   -6   -3  -11   -6
 3    3   Lee, D.         9    8   12    7   13 
 3    3   Lee, T.        11    8   13    8   14
 3        Martinez       14    5    8    9    8
 3    3   Mientkiewicz   10    6    7   12    8
 3    3   Olerud          0    8    2    2    7
 3    3   Sexson         -5    9    1   15    6
 3        Thome         -14  -10    5   -6   -3
 3        Young          10    0   12    5    8

                    UZR   DRA    ZR   DFT   DRAZR
    Avg 3456789       3     2     3     2     3
    Std              12    10     7     9    10
    Correl w/ UZR  1.00  0.76  0.64  0.60  0.82

                    UZR   DRA    ZR   DFT   DRAZR
    Avg 456789        3     2     2     1     2
    Std              12    11     7     9    10
    Correl w/ UZR  1.00  0.79  0.67  0.59  0.84

                    UZR   DRA    ZR   DFT   DRAZR
    Avg 45678         4     1     2     1     2
    Std              12    11     7     9    11
    Correl w/ UZR  1.00  0.84  0.72  0.64  0.89

It’s pretty clear that DRA is very, very accurate at all positions except first base and right field. Perhaps larger samples will improve the situation in right field. My only concern is that the DRA average is lower than the UZR average, though DRA, ZR and DFT are all closer to each other than to UZR. This reflects the effect of outliers; the median ratings are closer together:

    Medians         UZR   DRA    ZR   DFT   DRAZR
    med 3456789       3     2     4     2     4
    med 456789        2     1     4     2     3
    med 45678         3     2     4     2     3

During the 2001-03 period, the best fielders at each position, taking into account per-season ratings and the ability to play three Full Seasons:

Shortstop:   Orlando Cabrera (whom saber-saavy Boston picked up for defense).
Second:      Adam Kennedy.
Third:       Eric Chavez; perhaps Scott Rolen, who had a great 2004 after an OK 2003. 
Center:	     Mike Cameron, perhaps Andruw Jones.
Right:       Ichiro! (DM commentary is closer to the DRA +15 rating).
Left:        Jacques Jones.
First Base:  Todd Helton.

And their DRA ratings are all fundamentally in agreement with the UZR/DM consensus.

References & Resources
I’d like to thank Dick Cramer for his support in the past, Mitchel Lichtman for creating UZR, and baseball analyst, Tangotiger, for making detailed UZR output available in a convenient form. I’d especially like to thank the folks at Retrosheet:

The information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at 20 Sunset Rd., Newark, DE 19711.

There is one more absolutely necessary acknowledgement: my own fallibility. In creating DRA and tracking the results of other fielding systems, I had to do a tremendous amount of cutting and pasting and hand-coding of data. I have done my best, but I’m sure there are some errors, though I don’t believe any of them are significant.

I look forward to hearing from you. Don’t hesitate to e-mail with questions, criticisms and corrections.


Comments are closed.