Evaluating catchers: Framing pitches – part 3

In the past couple of articles catchers’ framing has been analyzed here: (part onepart two).

Before diving into part three, let me explain what I considered framing in those articles (and in this one, as well). I should have made this premise clear at the beginning of the first article of this series, but I neglected to do that, probably believing that the definition would emerge from the text. Some of the comments made me realize that was not the case, so here I’m trying to redeem myself.

What has been considered framing in this series?

The way a catcher receives a pitch can slightly change the likelihood that said pitch is declared a strike by the umpire. I suspect there are several ways to alter the odds, and sometimes even the catcher himself is not aware he is doing something on that effect.

So, when talking about framing in the past couple of articles, I did not have any specific action in my mind: Bringing the glove inside the zone (that, in my opinion, would indeed reduce the chances of a called strike), positioning the body so that the pitch arrives right to a motionless glove, catching the slider closer to the plate so that the breaking action had not brought the ball too far away from the plate, and so on.

What I had in mind is the following: A catcher receives 100 pitches. Given their characteristics (pitch type, count, location, pitcher throwing it, batter facing it, umpire judging it), 60 of them should have been called for strikes. If 75 of them are actually declared strikes, then the catcher has converted 15 balls into strikes by virtue of his framing skills, no matter how he physically achieved the result.

If you feel that “framing” is a misnomer for what I’m measuring, feel free to suggest something else in the comments section below.

From rate to count

Until part two, the runs saved by framing were calculated by prorating the increase (decrease) in probability of a called strike due to the catcher to an estimated number of borderline pitches.

For the purposes of this article, the catcher effect has been calculated on every single pitch. Also, while the average run value of turning a ball into a strike was previously used, now the count-specific differential is adopted. Finally, framing for high and low pitches delivered to the middle part of the plate has also been thrown into the mix.

Unfortunately there is a portion of the PITCHf/x database that does not get into this analysis yet, as the ultimate statistical model would combine a spatial component with the multilevel structure (this stuff is not available on the stat software I use). The pitches falling in the green area in the diagram shown below are part of the analysis.


Let’s go with an example.

According to the model employed, a 2-1 fastball from Roy Halladay to Albert Pujols delivered two inches outside the middle of the plate one foot and eight inches from the ground has a 44 percent chance of being declared a strike if Dana DeMuth is calling the game.
If such a pitch ends up being called for a strike, we assign the catcher (1 – .44) * .181 = .10 runs; otherwise he gets (0 – .44) * .181 = -.08 runs.

The first number inside the parenthesis is either zero (ball) or one (strike). Then you have the predicted probability of a called strikes given all the considered factors (.44 in our example). Finally, you multiply for the run value of converting a ball into a strike on the given count (.181 on a 2-1 count).

Seasonal rankings

A Hardball Times Update
Goodbye for now.
-------2008 top ten-------
           player   G   RV
     Brian McCann 138 23.6
   Russell Martin 149 22.5
      Jose Molina  97 20.3
    Yadier Molina 119 19.2
        Paul Bako  96 13.8
      Chris Coste  78 10.8
        Joe Mauer 139 10.6
       David Ross  54  8.0
      Brad Ausmus  77  7.2
 Yorvit Torrealba  67  6.7

------2009 top ten------
         player   G   RV
   Brian McCann 127 17.7
   Ryan Hanigan  88 13.5
    Jose Molina  49 12.7
     Gregg Zaun  83 12.2
 Miguel Montero 111 12.1
     David Ross  52 10.1
 Russell Martin 137  9.9
  Yadier Molina 138  9.3
  Bengie Molina 123  8.4
      Joe Mauer 109  8.1

------2010 top ten-------
          player   G   RV
     Jose Molina  56 16.9
 Jonathan Lucroy  75 16.9
    Brian McCann 136 15.9
   Yadier Molina 135 15.8
  Russell Martin  93 12.0
    Ryan Hanigan  68 11.0
    Matt Wieters 126 10.2
  Miguel Montero  79  9.8
    Geovany Soto 104  9.6
    Buster Posey  76  8.8

Brian McCann tops both the 2008 and 2009 lists and comes up third in 2010. Several other names recur in those short lists: Various Molinas, Russell Martin, Joe Mauer, David Ross, Miguel Montero and Ryan Hanigan.

As you can see, the best catchers are contributing over one win with their framing skill. Jose Molina unbelievably tops the 2010 list despite having played just 56 games. (Remember, we are now dealing with a counting stat!) While a player can put up extreme numbers in a small sample, the fact that Jose recorded similar numbers both in 2008 and 2009, should leave out the possibility that he got lucky and had all the calls come his way in 2010.

Sure, the prorated three to four wins he would produce in 130 games behind the dish would need to be regressed somewhat, but it’s hard to believe he does not countribute an extra win per season thanks to his framing ability.

-----2008 bottom ten-----
         player   G    RV
    J.R. Towles  53  -4.7
   Brandon Inge  60  -4.8
    Mike Rabelo  32  -5.6
      Toby Hall  37  -6.0
   Gerald Laird  88  -6.2
  Kenji Johjima 100  -8.3
 Chris Iannetta 100  -9.3
 Dioner Navarro 117 -10.9
   Nick Hundley  59 -11.8
    Ryan Doumit 106 -29.8

-----2009 bottom ten-----
         player   G    RV
    Omir Santos  91  -6.9
     Koyie Hill  79  -8.2
    Mike Napoli  96  -8.4
 Dioner Navarro 113  -8.6
   Jorge Posada 100  -9.3
    Rob Johnson  80  -9.4
    Kurt Suzuki 135 -11.9
  Kenji Johjima  70 -15.9
    Ryan Doumit  71 -16.2
   Gerald Laird 135 -16.2

----2010 bottom ten-----
        player   G    RV
  Nick Hundley  76  -4.4
   John Hester  33  -5.6
     John Jaso  96  -5.8
    Lou Marson  87  -6.8
    Adam Moore  59  -7.4
 Jason Kendall 118  -7.7
  Gerald Laird  87 -10.8
   Ryan Doumit 100 -10.9
   Rob Johnson  61 -11.8
  Jorge Posada  83 -14.5

The bottom lists also display recurring names. The 2008 run values have Pearson’s correlation coefficients of .69 and .66 with the 2009 and 2010 values, respectively, while 2009 and 2010 have a coefficient of .82.

Park effects

The proposed model does not take park effects into consideration. Considering the really small sample of catchers that changed teams during the 2008-09 and 2009-10 offseasons, there does not seem to be a substantial variation in their ratings.

----------Rating of players changing uniforms-----------
			     Run Value	 Run Value/130 G
           player   seasons seas1 seas2   seas1    seas2
      Rod Barajas 2009-2010  4.7   3.4      5.1      4.6
   Kelly Shoppach 2009-2010 -2.2  -0.2     -3.5     -0.5
       David Ross 2008-2009  8.0  10.1     19.3     25.2
  Ramon Hernandez 2008-2009 -2.3   0.1     -2.4      0.2
    Bengie Molina 2009-2010  8.4   5.8      8.9      6.7
     Chris Snyder 2009-2010  4.7   8.5     10.9     10.9
  Victor Martinez 2009-2010 -5.7  -1.7     -8.7     -2.0
    Jason Kendall 2009-2010 -3.2  -7.7     -3.1     -8.5
     Miguel Olivo 2009-2010 -2.2   2.4     -2.8      2.8
 Yorvit Torrealba 2009-2010  0.2   4.8      0.4      6.8
       Gregg Zaun 2008-2009  5.4  12.1      8.9     19.0
  Victor Martinez 2008-2009  2.6  -5.7      6.1     -8.7
     Gerald Laird 2008-2009 -6.2 -16.2     -9.2    -15.6
   Ivan Rodriguez 2009-2010 -2.2   8.6     -2.5     11.0

Note: Catchers above are listed from lowest to highest difference in Run Value. Fourteen catchers who changed uniforms and played more than 50 games in consecutive seasons sport a .67 correlation. For comparison, 53 catchers who didn’t move to another ballclub record a .82 correlation.

Also, catchers’ performances at home and on the road are highly correlated, sporting a .82 Pearson’s correlation coefficient. However, it should be noted that home Run Values are generally higher than away Run Values.

Given all the above, one would be tempted to dismiss the park as an important effect and state that somehow catchers are more adept at putting their framing skills into play at home. However, when digging deeper into this issue, the following cases emerged.

During the full period considered (2008 to May 2011), Brian McCann totaled 47.5 runs at home and 30.7 on the road. Is that all due to the possible familiarity effect mentioned above? During the same time span, the visiting teams recorded a composite +12.2 runs at Turner Field.

Meanwhile, Ryan Doumit has been -22.7 at home and -39.9 on the road, while opposite teams have scored +20.8 runs at PNC Park.

While the bottom line is that McCann is great and Doumit is poor at framing pitches, both have played half of their games in parks that seem to favor strike calls. Thus, adding the park to the model will likely improve the final ratings.

Final remarks

In the past few weeks we have tried to estimate the runs a catcher may contribute by inducing the umpire to call more strikes. We progressively refined our estimation, but the final value has not moved by much. A catcher can contribute more than one win (probably even two wins) just by his ability to frame pitches.

As we mentioned, there is room for further improvement of the model. The ballpark should be part of the model, and pitches at the four corners are still out of the equation. Thus, we might still be on the conservative side when we state one or two wins as the ceiling.

Several of the top catchers in the proposed lists are backups. This makes sense, since it’s so hard to find one good-hitting catcher, let alone two. So, when looking at replacements, teams hunt for some other assets, like defense and ability to handle pitchers. Teams have probably figured that out for a while, but this analysis show that if you pay attention, you can get one extra win just by selecting the right backup catcher.

So just how good is Yadier Molina on defense? FanGraphs credits him with five to eight fielding runs each year from 2005 to 2010. For the last three seasons, we can add another 19, nine and 16 runs on top of that due to his ability at framing pitches.

References & Resources
Table 2 of John Walsh’s Searching for the game’s best pitch is the source for the value (by count) of turning a ball into a strike.

Newest Most Voted
Inline Feedbacks
View all comments
Harry Pavlidis
12 years ago

RJ Anderson tweeted this link
relevant quote:
Cobb talked about Chirinos’s and Lobaton’s shared ability to “stick” low and/or sinking pitches, of which Cobb throws plenty; that is, they get the thumb of their gloves underneath the ball and keep the arm moving up as they catch it, framing the pitch within the strike zone. This has something to do with pure arm strength—imagine catching a 91-mph two-seam fastball without the downward force of the pitch knocking your arm down—but also with technique: They actually practice this.

Peter Jensen
12 years ago

Max – Great article again.  Could you break out how much of the catcher’s run value in framing comes on inside, outside, low and high. I would guess that the majority is on outside, next would be either inside or low, and last would be high, but it would be interesting to know the overall proportions and whether any specific catchers have any special areas where they perform better.

It will also be interesting when we have enough Pitch Fx years where we can see how catchers develop over time.  My guess would be that framing ability is something a young catcher could improve with experience and that older catchers would not lose with age.  But only more years of data would tell for sure.

12 years ago

Very interesting work, Max.  Couple of questions.

How is location built into your model?  Are horiz and vertical distances used, or do you divide the zone up into a grid and make each “box” a unique variable?

Are there interaction variables between pitcher and pitch type, and/or pitcher X location?  Or is each pitcher assumed to have a single fixed impact on the probability that a pitch will be called a strike?

12 years ago

Max, when you say that 60 out of 100 pitches should be strikes, is that a base model you use for all pitchers or is that just an example number?

12 years ago

Max, since most pitchers use the same catcher for most of their starts in any given year or even several years in succession, how do you control for the pitchers?  In other words, and for example, how do we know that Yadier Molina’s framing skill is not partially or mostly due to his pitchers since many if not most of his pitchers from 2008-2010 only threw to him most of the time?

And yes, great work.

I also want to point out the obvious – that the value of a catcher’s framing is already included in his pitchers’ stats, such that in order to adjust or account for catcher framing, first you have to remove it from the pitchers that he caught and then give the catcher credit (positive or negative).  In other words, we shift the credit, unlike defense and FIP.

12 years ago

If you total up all runs saved and allowed by “framing”, they should add up to zero. Is this the case?

This is great work, regardless. I am looking forward to reading the first two parts of the series (and any future additions).

Peter Jensen
12 years ago

Max – I think the next step would be to identify particular pitches where some of the catchers that have high ratings get the call but average catchers do not, and look at the video to see if we can see why a catcher might be getting the strike calls consistently.  Do they initially set up in a better position to catch the ball without moving the glove?  Are they better at catching the ball in different areas of the glove so as to minimize glove movement?  Do they use different wrist orientations as indicated in Harry’s Cobb quote?  Do they catch the ball closer in to their bodies as Craig Wright noted?  If we can pin the “why” down it would strengthen the premise that these numbers represent an actual catcher skill rather than some undiscovered bias in the data.  And perhaps it is a catcher skill than be taught which would indeed be valuable information.

Max Marchi
12 years ago

I have used horizontal and vertical distances, but no interaction effects—so, for example, the count has the same effect on everybody. And 60 out of 100 was just an example number.

It seems that multilevel models can do a good job even for small-sample combinations; I can’t tell for sure about Yadier, but even a small number of pitchers who have worked with other catchers should contribute to make his numbers reliable. Probably a simulation would be useful here to test if that’s indeed the case.
Also, regarding your second point, it may be obvious, but it was worth pointing it out.

The way I calculated the run values, they should not add up to zero.
I assign to each pitch a “delta probability” value, i.e. the increase/decrease in probability of a called strike due to the catcher. If framing happened equally on the various pitch counts (or if I assigned an equal run value, no matter the pitch count) they should add up to zero (and they do if I assign an equal run value).
On the contrary, using count-specific run values the sum comes out way greater than zero. Thus I would say catchers know when the art is needed the most.

I would have guessed as you did, but it turns out that both low and high (in that order) show the highest range of variation,  followed by outside then inside (no big difference between RHBs and LHBs).
Mauer comes out as the best on high pitches, McCann and Yadier on low ones, Martin on outside pitches to righties, Jose Molina on outside pitches to lefties, I-Rod and Varitek inside to righties and finally Mathis inside to lefties.
The ratings by zone are consistent season-to-season as well.

Harry (and RJ by extension),
Thank you for the link.

Max Marchi
12 years ago

that’s a pretty interesting path to follow, but quite an effort!

I run separate regressions for inside, outside, vs LHB and vs RHB, thus your example on Halladay should be treated correctly.
On the other hand, I suspect that given a pitcher who exploits the outside corner and one who exploits the inside one, the same umpire won’t have an equal effect on both.

I tried to isolate pitches where the vertical component should not be a factor and pitches where the horizontal component should not be a factor, and see the effect of the other component—that’s the reason for the missing pitches at the corners.
I’m not fully convinced about what you propose, because pitches at the same distance from the plate have differente probabilities of being called for strike, depending on which direction they are from center.

Finally, the previous articles were based on the full PITCHf/x database.
Also, for this one, every effect but the catcher comes from a model run on the full PITCHf/x database.

12 years ago

I’m sure including interaction variables poses potential computing challenges.  At the same time, assuming each variable has a single effect on strike probability seems pretty problematic.  To use your example, do we really think Roy Halladay necessarily has the same effect on strike% on outide pitches as on inside pitches?  The same against RHH as LHH?  The same on his curves as his FBs?  If it’s possible, I think it would improve your model to at least use two variables for each pitcher – “X facing LHH” and “X facing RHH”. 

Another interaction that is probably very important is H and V distance.  The oval shape of the effective zone (rather than rectangle) tells us there’s a big interaction.  So it might improve the model to include distance from center of zone as a variable—sqrt (H^2 + V^2).

Final thought:  why not run 3-4 seasons simultaneously?  That would address a lot of MGL’s concerns, as pitchers and catchers would be less correlated.  (Again, assuming computing power isn’t an obstacle.)

12 years ago

Max:  Thanks for explanation.  I didn’t realize you were running multiple regressions.

I hear what you’re saying on the exclusion of pitches to the corners.  It seems to me that a total distance variable might still improve the model—as an addition to H and V distance, not replacing them—but perhaps not.