How Do We Assign Credit for Catching Base-Stealers?

Who was more influential on the basepaths, Chris Carpenter or Yadier Molina? (via Keith Allison)

Who was more influential on the basepaths, Chris Carpenter or Yadier Molina? (via Keith Allison)

So, who’s in charge here? In 2004, Yadier Molina made his major league debut alongside Chris Carpenter’s Cardinals debut, the beginning of one of the most successful batteries in recent baseball history. In the decade since, Carpenter has owned a 65 percent caught stealing rate; during the same time, Molina has caught roughly 40 percent of would-be base-stealers. While Carpenter had a quick move to the plate and excellent control, many would argue that it was Molina’s presence behind the plate that led to his success (in all facets of the game). But could it be the other way around? Did Molina benefit from Carpenter’s presence on the mound? If so how much?

Like most interactions in baseball analysis, there is no clear way to distribute credit for the outcome. It is much debated how the value of a caught stealing should be divided among the three parties involved (the pitcher, catcher and baserunner). How much of a caught stealing should we grant to the pitcher on the mound? Does the credit lean toward the pitcher or the catcher? However unclear the picture remains on the surface level, the answer depends on numerous factors that are very much related.

Say, Carpenter (if he were still active) is on the mound and the Cardinals manage to put away Billy Hamilton on an attempted steal of second. In this situation, we assume that Molina is the main cause, because his talent at throwing runners out is well-known, tested, and consistent among many different battery mates. In singular cases, like this, it is easy to generalize which party is at fault or deserves credit — Molina is one of the few guys with the ability to throw out Hamilton. Whether through scouting information (pop-times, move-to-mitt, the runner’s jump) or play-by-play data, we know Molina has the tangibles and the record to pull it off.

The Molina-Hamilton showdown is the most extreme example I could think up. But imagine the normal spectrum where average pitcher meets average catcher meets average baserunner. There, it isn’t quite as easy to determine credit objectively. This is usually where we use modeling to make sense of it

Where Baseball Meets Basketball Analytics

Most of my research revolves around the battery dynamic — as I call it “The Battery Effect” — or the idea that there are so many confounding variables in a single battery interaction (a wild pitch or a stolen base attempt) that it’s wrong to assign all credit to one party.

For some reason, baseball decided to reward all caught steals and passed balls to the catcher and all wild pitches to the pitcher, instead of divvying responsibility “n-times” among “n-parties”. My previous research and the correlations suggest that the pitchers “control” most every battery outcome, but as you’ve heard before, correlation does not always equal causation. So I’ve been searching for a better way to adjust for “The Battery Effect” and assign credit more responsibly among the pitcher and catcher.

My attention was turned to a popular Basketball analytics model in Regularized Adjusted Plus Minus (RAPM) by Nik Oza — founder of the the Georgetown Sports Analysis, Business and Research Group. RAPM adjusts each player’s plus-minus (the accumulative sum of score margin while the player is on the court) to reflect the quality of his teammates and opposition, while accounting for home court advantage. Without RAPM, plus-minus (+/-) suffers from many of the same problems we observe in the battery setting — like co-linearity between teammates and lineups. These problems are inherent in any team sport, where the line between one player’s contributions and another’s influence is blurred. While baseball for the most part is a collection of individual contributions, perhaps no other situation is more similar to that of a team sport than the exchanges among the pitcher, the catcher and the baserunner — so a RAPM framework in this setting makes sense. The question is who affects whom? What are the player’s true contributions independent of his teammates?

Note: Feel free to skip this section ahead if you’re not interested in the mathematical details.

Introducing Regularized Adjusted Plus-Minus For Battery Related Outcomes

The idea behind any RAPM framework, like the one suggested by Nik Oza, is a ridge regression which penalizes coefficients and regresses them towards zero. The penalty factor is known as lambda, and the lambda that minimizes the error of the model is chosen and can be computed through a program like R. The ridge regression can adjust for co-linearity of variables and decrease the variability of the prediction while sacrificing some validity (introducing bias).

Say that pitcher A has pitched only with catcher B behind the plate or a majority of pitcher C’s innings have come with catcher D — the ridge regression does a good job at adjusting for the relationship between the two battery mates. With that in mind, each pitcher, catcher and baserunner becomes an independent variable in our model — with their respective coefficients reflecting the impact of their presence on an battery outcome. However, because the ridge regresses towards zero, in the future I would like the model to take on a more Bayesian approach by regressing with some prior (determined by pitcher handedness) in mind and a minor league prior for rookies. This can be achieved without a RAPM framework: In the past, Jared Cross, of Steamer Projections, has used a Mixed Effects Model to assign credit among battery mates while adjusting for batting order.

Many other variables could affect a stolen base attempt, in addition to the battery or baserunner:

  1. Outs
  2. Score margin
  3. Base-state
  4. Inning
  5. Wild pitch
  6. Passed ball
  7. Pickoff
  8. Pitcher handedness
  9. Hitter handedness

With those factors in place I added them to the model and pulled all opportunities of the last three years from Retrosheet play-by-play. I grouped each line of my data by each opportunity where a runner was on first or second with no lead runner ahead. Later, I created dummy variables for every baserunner, pitcher and catcher possible. With the other factors above I plugged and chugged these numbers through the R package “glmnet”, and adopted and amended Jacob Frankel’s code used to calculate RAPM found here. (My code can be found here for replication’s sake).

The predictor I sought was caught steals per opportunity (what we will call adjCS+/-). I ran through and applied a similar framework to stolen base attempts, to get adjSBA+/-. We don’t want stolen base attempts in the denominator of our response variable, just because we don’t want to assume the same frequency of attempts for all parties involved. Instead, we know that attempting a stolen base against one catcher/pitcher tandem is much different than another. For this reason, I grouped by opportunity — or the number of times a baserunner was in a selected base state against the battery. I included only base-states where the runner could steal the base ahead of him, given no one was directly in front of him on the basepaths.

Back to the Carpenter dilemma. As we mentioned, Carpenter did have immense success keeping runners from stealing, with a 0.25 caught steals per stolen base attempt above average. However, when converting his plus-minus into an expected caught stealing percentage (xCS%) we see a large differential between his observed success and his adjusted performance. The difference between his CS% and his xCS% is nearly 20 percentage points (65 percent versus 48 percent). So when adjusting for the quality of his backstop (mostly Molina) his adjusted performance nets three runs less than his career track record suggests. (Note: to derive xCS% I used caught stealings divided by attempts as the response variable; the rest of the article uses caught stealings divided by opportunities to keep it consistent with SBA+/-).

A Hardball Times Update
Goodbye for now.

Oh yeah, that’s three runs over a decade. So even when observing the most extreme drop-off between unadjusted CS% and expected CS% for a pitcher, we don’t even see even half a win as a consequence. Simply put, the pitcher does not accumulate enough stolen base attempts against him (unless his reputation is awful) for his expected performance to mean much. While a pitcher may “control” most battery outcomes, it’s not what they are sought out or selected to do, and for good reason. Meanwhile, some catchers and their pitchers are not as far off in contribution as the eye would suggest.

However, the opposite is true of a catcher. He accumulates many more attempts against him by virtue of his job — to sit behind the plate for a thousand-plus innings a year. However, a catcher’s value is not only in his arm but in his ability to reduce attempts through his reputation — the same is true for certain feared pitchers. So it is also important to introduce a statistic that objectively defines a player’s reputation, or the effective amount of stolen base attempts added or subtracted per opportunity (adjSBA+/-).

Overall, the predictability of both adjSBA+/- and adjCS+/- was weaker than previous seasons’ caught stealing percentage. I didn’t expect this method to be predictive in the first place. My reasoning is that if this were an independent measure of pitcher/catcher defensive contribution, then year-to-year caught stealing percentage would not reflect their “defensive skill” — since the remaining influences of last year’s similar battery and similar environment will remain.

However, I expected that when evaluating team switchers, we will see that their independent evaluation metrics will overtake caught stealing percentage in predictability. To test this out, I took all team-switchers from 2003-2013 and ran a regression of year one adjCS+/- and adjSBA+/- on year two caught stealing percentage. Only 40 catchers had switched teams from 2003-2013 with at least 500 opportunities with runners on the base paths. Their year-to-year relationship between caught stealing percentages was a mere 2 percent compared to 8 percent between adjCS+/- and CS%. So, in honesty this is not a predictive tool to begin with. Descriptively, adjSBA+/- could explain 70 percent of the variation in a pitcher’s CSruns in the same year, but there were little other relationships of interest. Instead I find interest in its use as a descriptive tool to tell us who affected the battery’s performance overall, which we will come back to later.

Key

  • CSadded: Number of CS added/subtracted by pitcher/catcher based off of adjCS+/- * Opportunities
  • SBAadded:  Number of SBA added/subtracted pitcher/catcher based off of adjSBA+/- * Opportunities
  • Runsadded: CSadded*Value of CS + (SBAadded-CSadded)*Value of SB
  • CS: # of Caught Steals
  • SBA: # of Stolen Base Attempts
  • OPP:  Number of opportunities, in selected base states (no runner ahead of prospective baserunner)
  • CS%: CS/SBA, pickoffs removed

Results

Below are the 2011-2013 numbers (Note: Retrosheet removes pickoffs from caught stealings, so caught stealing percentage is without pickoffs while the regression included it. See Mark Buehrle for discrepancy between these two.*)

Of course, the “Runsadded” is just an estimate, mostly because of a rough estimation of SBA value — which can be debated in the comment section.

Here I took adjCS+/- and adjSBA+/- and factored in the number of opportunities each catcher/pitcher had (this means runner in our selected base states). Multiplying these metrics by opportunities gives us a rough aggregate of their contribution (CSadded and SBAadded). The rough assumption here in turning this into runs is the value of a SBA added or subtracted. In general, the fewer stolen bases the better, since we are not testing the probability of a success or failure. So I took the difference between SBAadded and CSadded and multiplied by the value of a stolen base. Say that a player added two caught stealings and -50 stolen base attempts. Then, technically, he added -52 stolen bases. I know this is a rough way of assigning a run value but it will do for now until we can objectively define the value of a stolen base attempts added or subtracted — which I would like to be related to individual catcher/pitcher break even points, much like this framework.

Catchers 2003-2013
Name CSAdded SBAAdded RunsAdded CS SBA Opp CS%*
Yadier Molina 1 -196 49.8 160 399 9866 40%
Ivan Rodriguez 10 -116 35.8 122 397 8299 31%
Matt Wieters -16 -145 25.8 92 322 5318 29%
Gerald Laird 37 3 23.4 116 327 4202 35%
Miguel Montero 14 -53 22.1 81 303 5320 27%
Ramon Hernandez 22 -25 20.6 121 516 7464 23%
Salvador Perez 14 -42 19.3 33 93 1672 35%
Jose Molina 31 3 19.1 79 248 3143 32%
Joe Mauer -10 -102 18.6 96 376 6979 26%
Kenji Johjima 18 -19 16.9 67 204 3781 33%
————- ————– ————– ————– ————– ————– ————– ————–
Paul Lo Duca 14 87 -12.8 108 468 4980 23%
Alex Avila 0 54 -13.2 79 329 3684 24%
Chris Iannetta -26 -10 -14.3 60 330 4566 18%
Geovany Soto -10 34 -14.9 81 380 4597 21%
Victor Martinez -5 57 -17.5 101 515 6135 20%
Mike Piazza -7 66 -21.5 32 241 1969 13%
Brian McCann -15 56 -23.9 132 655 8016 20%
A.J. Pierzynski -37 9 -26.0 135 832 11788 16%
Jason Kendall -11 81 -27.6 149 665 9882 22%
Michael Barrett -28 48 -30.2 54 324 4205 17%

Pitchers 2003-2013
Name CSAdded SBAAdded RunsAdded CS SBA Opp CS%*
Carlos Zambrano  0 -73 18.4 27 65 1667 42%
Mark Buehrle -10 -98 17.7 11 53 2260 21%
Jon Garland 7 -46 15.9 33 65 1634 51%
Zack Greinke 7 -37 13.7 33 75 1544 44%
Livan Hernandez 11 -19 11.7 46 124 1890 37%
Chris Carpenter 5 -33 11.3 28 42 1218 67%
James Shields 2 -33 9.9 29 74 1536 39%
Justin Verlander 9 -17 9.8 39 120 1604 33%
Bartolo Colon  0 -38 9.5 17 34 1015 50%
Rich Harden 7 -18 9.2 19 43 628 44%
————– ————– ————– ————– ————– ————– ————– ————–
John Lackey 9 68 -11.4 44 217 1976 20%
Ted Lilly -7 33 -12.6 9 128 1440 7%
Chris Young 1 55 -13.3 10 106 561 9%
Jose Contreras 11 82 -13.7 25 163 1011 15%
Brandon Webb 9 81 -14.1 32 166 1298 19%
Carl Pavano 58 -14.3 26 140 1157 19%
Cole Hamels -5 47 -15.0 16 134 1389 12%
Tim Lincecum -9 46 -17.2 17 145 1341 12%
Tim Wakefield 21 132 -19.3 44 248 1569 18%
A.J. Burnett 2 113 -26.7 36 264 1874 14%

These numbers are on a counting basis, so what about on a pure rate statistic level?

Below are the leader boards from 2011-2012:

Best and Worst Pitcher Reputation
Name ADJSBA+/-
Clayton Richard -0.139
Johnny Cueto -0.086
Nathan Eovaldi -0.08
Josh Tomlin -0.077
Doug Fister -0.065
——— ———
Alex White 0.073
Jeff Niemann 0.08
Tommy Hanson 0.091
Scott Feldman 0.096
John Lackey 0.097

Best and Worst Pitcher Caught Steals Added
Name ADJCS+/-
Alfredo Simon 0.039
Josh Collmenter 0.037
Jair Jurrjens 0.036
Michael Pineda 0.035
Rodrigo Lopez 0.030
——— ———
Chris Archer -0.020
Brandon Morrow -0.021
Nathan Eovaldi -0.024
Alfredo Aceves -0.030
Clayton Richard -0.034

Best and Worst Catcher Reputation
Name ADJSBA+/-
Salvador Perez -0.033
Chris Stewart -0.029
Matt Wieters -0.028
Humberto Quintero -0.025
Yadier Molina -0.024
——— ———
Hank Conger 0.017
Eli Whiteside 0.019
Jose Lobaton 0.020
Alex Avila 0.025
Rob Brantly 0.039

Best and Worst Catcher Caught Steals Added
Name ADJCS+/-
Yan Gomes 0.020
Salvador Perez 0.010
Josh Phegley 0.008
Yorvit Torrealba 0.007
Nick Hundley 0.007
——— ———
Martin Maldonado -0.007
Derek Norris -0.007
Ryan Doumit -0.008
Chris Stewart -0.010
Joe Mauer -0.010

In the Scope of the Battery

Like the previous research I have conducted has shown, a pitcher’s adjusted CS performance correlates with the battery CS performance at a 1.5/1 ratio with catcher past CS performance. With this in mind, I also used our proxy of reputation to see which battery mate had more of an impact on the amount of stolen base attempts that took place under their watch.

When comparing CS+/- and SBA+/- with the battery numbers the following was found:

Comparing CS+/- & SBA+/- With Battery Numbers
Regression Position R^2
CS% Battery~adjCS+/- Catcher 0.002
CS% Battery~adjCS+/- Pitcher 0.021
SBA% Battery~adjSBA+/- Catcher 0.034
SBA% Battery~adjSBA+/- Pitcher 0.566

The interesting note here is that it is the pitcher “reputation proxy” (adjSBA+/-) that correlates best with the actual SBA% of the battery — and the margin is not even close. For anyone who thinks that a base runner steals off the pitcher, this is more evidence in their favor.

Next Steps

When I return to this topic, I’d like to see Bayesian priors for both pitchers and catchers. These can be based off pitcher handedness and we have plenty of minor league catching data to build from. I think the inclusion of the above will improve the predictive value of this framework.

Meanwhile, I’d like to see dummy variables connected to pitch location, and swing/no swing. These factors have an effect on caught stealing percentage.  On this front, pickoffs would need to be removed — which would require a link between PITCHf/x and Retrosheet.

I attached all the code here (R code, SQL code), and the data is in this google doc. So build on it if you like. I hope my next article focuses on what this means for the base runners involved.

References & Resources


Max Weinstein is a baseball analyst. He has written for Fangraphs, The Hardball Times, and Beyond the Box Score. Connect with him on Twitter @MaxWeinstein21 or email him here.
5 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Peter Jensen
9 years ago

I don’t see pitchouts included in your list of other factors. Did you consider them at all?

Max Weinstein
9 years ago
Reply to  Peter Jensen

I did not consider them, yet. I understand the problems that presents and I am working towards adding PITCHfx data to incorporate pitch type, location, swing/no swing, removing pitchouts. Before I would return if like to study if pitch location/pitch type is solely a function of the pitcher. In the meantime, SQL and r-code is attached if anyone wants to take up including pitch variables.

Peter Jensen
9 years ago
Reply to  Max Weinstein

Max – Retrosheet shows pitchouts in the Pitch Sequence list. It also shows pickoff throws which is another variable you might want to consider.

jerry weinstein
9 years ago

Max
I know you mentioned it & just wanted to reinforce the effect pitch location has on the SB/CS process.Pitches down & out of the zone are more difficult. Also pitch type also is a very important factor.Most off speed pitches are at least 1/10 of a second slower than fastballs + their movement is less predictable. You did not say anything about “break time.”It would be interesting to measure SB/CS by the time it takes the pitcher to deliver the pitch. You would need to cross match it with the base runner’s running time from 1st move until he arrives at second base. Another factor would be the type of jump the base runner gets relative to the pitcher’s 1st move.( When you break down SB’s it’s about pitcher’s release time,catcher’s glove to glove time,accuracy of throw/tag time,& base runner’s jump relative to pitcher’s 1st move) You talked about swing & non swings which certainly factor in. You also need to look at the type of check swing or fake bunt that is intended to cover the ball so the catcher can’t see the ball until the last instant. That is the #1 most difficult thing for catchers to over come because it destroys their rhythm & tempo. The run prevention value of an efficient stretch delivery coupled with tempo variation & a + move go much further than just stopping the SB. More DP’s, less 1st to 3rd,fewer score from 2B on base hits, & fewer overall stolen base attempts which reduce runs scoring on extra base hits with the runner in motion.

victoria secret uk
9 years ago

fewer overall stolen base attempts which reduce runs scoring on extra base hits with the runner in motion.