Pitch Framing Was Doomed From the Start

Catchers have been boning up on their pitch framing skills. (via Lisa Suender)

Catchers have been boning up on their pitch framing skills. (via Lisa Suender)

Editor’s Note: This article was originally printed in The Hardball Times Baseball Annual 2017. You can purchase a copy of it here.

Here’s a little exercise for you. Which teams would you consider to have forward-thinking, stat-friendly front offices? Go ahead, list them off. I’ll even play along.

I don’t know where you start. The Dodgers, obviously. And the Cubs, of course. You have to make sure to fold in the Rays and the A’s. The Yankees have a massive analytics department. The Mariners took a step forward with Jerry Dipoto. The Phillies have changed under Matt Klentak. The Twins are looking to get more analytical under their new management. Cleveland has to count, and so do the Astros, and the Cardinals, and…

It goes on. Every front office has some sort of analytics department. Even the Royals! Especially the Royals. Major league baseball, overall, has gotten smarter. It’s no longer enough to just hire some smart guy in a suit. Smart guys in suits are everywhere. Any hiring has to be considered in context.

In a sense, analytics doomed analytics. At first, they seemed powerful. You could find market inefficiencies everywhere. And so early adopters were able to benefit. But then came the later adopters, and then came the last adopters. Numbers are no less powerful than they used to be, but there’s less relative power to be wielded when everyone’s trying. A given front office now would have a massive advantage over an organization run by a bunch of stubborn throwbacks, but such organizations are so near to extinct they’re almost hypothetical. So the comparison doesn’t have any meaning.

That’s the larger trend. That might be the trend, as major league baseball goes. The sabermetric movement has been fully embraced. But I mostly want to talk to you about a similar trend on a much smaller scale. It’s not the easiest thing to prove, but it’s plenty easy to discuss, and I feel it works as a legitimate theory. As analytics arguably doomed analytics, pitch framing seems likely to doom pitch framing.

Pitch-framing analysis has been moved to the back burner. It’s a product of the PITCHf/x system, and it was deeply exciting, but many have moved on from PITCHf/x to Statcast, because it’s the shiny new thing, and it’s impossibly informative. “Exit velocity” and “launch angle” have turned into commonplace terms overnight. We’ve already learned so much about hitters. We’ve already learned so much about pitchers. And, importantly, we’re beginning to learn about defenders. Statcast is going to get us places.

But pitch framing didn’t go away when the attention did. And, if I can give you some background: Pitch-framing statistics represented a major breakthrough. They were made possible by PITCHf/x around 2008 and 2009, and the greatest public efforts were made by Mike Fast, Max Marchi, Matthew Carruth and Dan Turkenkopf. Maybe you’ve been reading baseball analysis for a few years. Maybe longer. But, maybe not so long. It was groundbreaking stuff. What the numbers indicated was that pitch framing, or pitch receiving, could be worth dozens of runs in a season, by preserving or stealing strikes. More, the different methods mostly agreed with one another, and they tended to show year-to-year sustainability. There was signal, which meant there was talent.

The closest José Molina ever came to being a superstar was on the internet. On the internet, he was the face of pitch-framing statistics. According to numbers from Baseball Prospectus, the way Molina caught was worth 36 extra runs in 2008. The next year, it was worth another 19, and then that went up to 24 in 2010. Pitch framing was specifically cited as a reason why Molina wound up with the Rays. Molina always had a talent, but, at last, its real value could be quantified.

Not that Molina was the only catcher who stood out. The numbers have also celebrated guys like Russell Martin, David Ross, Jonathan Lucroy and Francisco Cervelli. On the other side, when there are players who are good at something, there have to be players who are less good. The opposite of Jose Molina was Ryan Doumit. Unlike Molina, Doumit could hit. But, as it turns out, Doumit couldn’t catch. His framing in 2008 cost his team an unfathomable 63 runs. The next year, in less time, he cost his team 29 runs. Then 24. Doumit was a defensive negative. The impact was worse than one would’ve imagined.

All right, let’s pause. Think about what was happening when the ball started to roll. It was being demonstrated, for the first time, that there were real differences in value among catchers, just based on how they caught. That actually flew in the face of prevailing sabermetric consensus at the time. Evidence mounted as the teams continued to shift toward being more number-savvy. Just in theory, what do you think would happen?

You’d think smarter teams would start to seek out good receivers. You’d think, additionally, that smarter teams would attempt to develop good receivers. That’s not as easy as just flipping a switch—that requires a lot of video work, to see what catching techniques help and what catching techniques hurt. But, just in general, you’d think pitch framing would be more heavily prioritized. What happens when something gets prioritized by an increasing number of teams?

At first, there were inefficiencies to be taken advantage of. The Rays, for example, got Molina for cheap. Some teams were skeptical of the framing data. Yet, over time, most teams have been turned into believers. Any executive or coach you talk to would agree that catchers have some degree of influence over the strike zone. Now we have well-established numbers for that ability. It’s not anything to be ignored. And when every team wants to get better at something, something that was only recently discovered, you look for the laggards to catch up to the leaders.

Every team wants a good-receiving catcher. Every pitching staff wants a good-receiving catcher. Every organization wants to develop good-receiving catchers. And it even feels like something a player can just learn. If a hitter is going to adjust to hit for more power, odds are, that same hitter is also going to strike out more often. There should be no such drawback with attempting to improve how a catcher catches. It’s just learning appropriate technique. Bad-receiving catchers might be helped. Then you’d have a catcher who just made himself better. No downside.

A Hardball Times Update
Goodbye for now.

Stories have gotten out to the public. There were articles written about the Astros trying to improve the way Jason Castro caught. There were similar articles written about the Angels and Chris Iannetta and Hank Conger, and there were articles written about the Padres and Nick Hundley. There have been more such stories, and there have been countless additional stories that just weren’t or haven’t yet been written. Teams continue to work on this. The A’s have had a system implemented in earnest for about the last 18 months. No team wants to settle for a Doumit-type catcher anymore.

We might as well get to some actual evidence. To this point, it’s all been theory. The theory is that, over time, gaps between teams and catchers should narrow. Bad-receiving catchers should start to disappear, raising the baseline. The best catchers, after all, can’t improve much, but the worst ones sure can.

The most advanced numbers out there come from Baseball Prospectus, so that’s what I’ve used here. Using its data, I calculated for each catcher framing runs above or below average per 7,000 opportunities, going back to 2008—7,000 being roughly the number of opportunities a starting catcher will have in any given season. In that way, it’s like UZR/150. I narrowed to only catcher-seasons with at least 2,000 framing opportunities, and in the plot below, you can see each season’s standard deviations within the catcher pool.

sullivan_teamframe_01

There remains, of course, an existing gap between the best and the worst catchers. This is all still very new, as a statistical measurement. But it’s worth noting that the two smallest standard deviations have come in the last two seasons. A smaller standard deviation means a lesser spread. It supports the idea that the gap is gradually getting smaller.

As an alternative plot, here’s the year-to-year average of the top 10 framing catchers, in terms of runs above average per 7,000 chances.

sullivan_top10_01

Similarly, you see a drop, with the two lowest averages coming in the last two years. In 2011, the top 10 catchers were, on average, 30 runs better than the mean. In each of the last two years, the average has been a hair above 20. It doesn’t make too much sense that the best-receiving catchers would be getting worse. The other explanation would have to be that the average catcher is just improving. Just for fun, here’s the top and bottom 10 for the 2016 season:

TOP 10 & BOTTOM 10 CATCHERS BY RUNS/7000, 2016
Catcher Runs/7000 Catcher Runs/7000
Miguel Montero 28.4 Alex Avila -14.8
Yasmani Grandal 27.7 Carlos Ruiz -14.8
Buster Posey 25.3 Ryan Hanigan -15.3
Jeff Mathis 23.4 Jarrod Saltalamacchia -15.4
Tony Wolters 18.1 Nick Hundley -15.9
Jason Castro 18.0 Chris Iannetta -18.3
Kevin Plawecki 17.9 Robinson Chirinos -18.6
Roberto Perez 17.7 Bryan Holaday -19.3
Tyler Flowers 17.5 Juan Centeno -22.4
Caleb Joseph 17.4 Dioner Navarro -24.3
Rene Rivera 17.4 Ramon Cabrera -39.5

To mix it up, here’s something on the team scale, now using information from FanGraphs. It’s pretty easy to calculate, for a team’s season, the difference between the number of strikes and the expected number of strikes. Obviously, there would be a relationship between this measure and pitch-framing ability. The more extra strikes a team gets, the better the receivers, and vice versa. I ran the numbers for every team’s season since 2008, and here are the year-to-year standard deviations.

sullivan_expstrike_01

This is fairly striking. The standard deviation in 2011 was 219 strikes. By 2016, that number dropped to 122, or 56 percent of what it had been. The two lowest standard deviations have come in the last two years. Going back to 2011, the difference between the best team and the worst team was 964 strikes. Last year, the difference was 437 strikes.

That’s still a difference. That’s still a pretty substantial and significant difference. But the differences have gotten smaller, which is what we would expect. Should the trend continue, the differences will get smaller still. Perhaps not smoothly, perhaps not year after year after year, but in the big picture the gap should get smaller. Evidence suggests more teams are on board with pitch-framing ability, and as that happens, the best framers have less of a relative edge.

Over the course of the nine seasons we have this data for, things have changed quite a bit. We can see just how much they’ve changed in this table of year-to-year differences for each team:

ADJUSTED EXPECTED STRIKE DIFFERENCE, 2008-2016
Team 2008 2009 2010 2011 2012 2013 2014 2015 2016 Total
Brewers   72   89  377  482  452  445  406  243  151  2,716
Braves  376  501  333  440  343   57 -189 -250 -102  1,510
Cardinals  330  276  265  176  175  116 -131   15   25  1,245
Diamondbacks  102  267   80  224  199   79  306  -24  -35  1,197
Yankees  306   68 -221  210  243  278  220   22   69  1,194
Reds  434  247  278  138  208   -4  -34 -108  -64  1,096
Giants -109   84   17  252  130  -38  159  101  145    741
Astros  -13 -126  168   87   32   -1  123  133  178    580
Nationals  -25  -16  200  298  138   25  -12  -79 -137    391
Phillies  162  200   75  -34  117   28  -42 -107  -68    330
Orioles   22  192   81   28  -61  -77   99  -20   46    310
Angels   42   -5   51  149   29  -11   80  -53    4    286
Blue Jays  182  -68   28  -19  -29   85 -181  -12  155    140
Padres  -68 -143   -4 -232   40  146  310  118  -36    132
Cubs  -93  -82   97   -7  -19 -129 -115  180  203     35
Red Sox  -16 -122 -219   26   75   81   90  -40   41    -84
Dodgers  244   68  -55 -202 -213  -43 -103   62  111   -130
Rays -232   -6 -134 -205  179  163   89   53  -47   -139
White Sox -107  -31  -64  -51 -122  -28  -45  267 -199   -380
Mets  -89 -125 -159 -147  111 -140   -2   36   79   -437
Royals   99   57  -46 -170 -142  -99 -101  -97  -67   -565
Pirates -463 -353    2 -202 -319  187  135  303  101   -607
Athletics   22 -132  -99 -168 -117   41  -66 -169 -131   -819
Rangers -269   45 -154 -138 -135 -131 -150   75    6   -851
Twins    5   56   10   85 -220 -346 -361  -94 -124   -990
Rockies -131  -13  -60 -146 -188 -148 -220 -219   64 -1,060
Mariners -234 -318 -315 -157 -312 -132  233   86  -97 -1,248
Tigers -180 -346 -123   42 -105   29 -160 -276 -137 -1,256
Cleveland  -54 -205 -284 -482 -298 -135  -62  -86  -56 -1,662
Marlins -316  -61 -124 -277 -189 -298 -276  -60  -77 -1,677

Quality pitch framing isn’t going to be rendered irrelevant, certainly not as long as there are humans responsible for calling balls and strikes. But remember the example of a hypothetical old-school, backward front office. That front office might get savaged in the market today, but that front office also doesn’t really exist. Likewise, a truly bad pitch framer would score horribly now. The average only gets better and better. But there just won’t be many truly bad pitch framers. When everyone is a good receiver, no one is a good receiver.

As long as there are umpires, and not robots, there will be a gap. We will never see a day when every single catcher in baseball is identically good at catching pitches. It is a skill, and only so much can be taught. Yes, anyone can improve his forearm strength. Yes, anyone can improve his wrist strength, and the way that he positions himself behind the plate. These are fundamentals. But there’s an instinct component. There’s anticipating where pitches are going to go. Taller catchers will be better up in the zone; smaller catchers will be better down in the zone. It’s not realistic to believe the average draft pick can be coached into becoming another José Molina. Some of that ability was innate.

But some of it wasn’t. Some of that ability can be coached. Maybe even much of that ability. And now that that ability can be recognized, it could end up selected for, in drafts and internationally. Teams are running out of excuses for having lousy receivers, and eventually, it stands to reason any advantages are going to be slight. Maybe the gap between the best and worst catchers gets trimmed to 20 runs. Maybe it gets trimmed to 10. It’s nothing that will happen overnight, but it looks like the process is already underway.

Pitch-framing numbers answered questions we didn’t know we had. A new statistical category emerged out of nowhere, and before long, the research proved its own worth. That was the birth of the revolution. Every team now wants good-receiving catchers. Every team, additionally, wants to develop more good-receiving catchers. The market is going to end up flooded with good-receiving catchers. By then we’ll no longer recognize them as good-receiving catchers. Pitch-framing is sufficiently important that baseball teams will prioritize it right into insignificance.

References & Resources


Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.
21 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Big Daddy V
7 years ago

I’d go a step further. The umpires have heard about pitch framing too, and so you have to assume they’ve been trying to teach themselves not to be fooled by it.

Hasn’t the called strike zone been getting closer to the rulebook in the past couple years?

Michael
7 years ago
Reply to  Big Daddy V

Exactly. Pitch framing represents two negative possibilities, and analysis and mainstream data threatens each.

First, that an umpire was duped by a tricky catcher to make a mistake against his will (or by an incompetent catcher to make a mistake in the other direction).

Second, that a complicit corrupt umpire is knowingly altering his call based on action after the ball has crossed the strike zone to reward a skilled catcher or punish an incompetent one, independently of the pitch being in the actual strike zone.

Each of these two scenarios in each of the two directions represents umpires failing to perform their jobs, and drawing empirical analysis and mainstream focus to these failures can only lead to pressure to avoid them. The only question is how gradually the umpires change and whether they evolve slowly or react.

Yehoshua Friedman
7 years ago
Reply to  Big Daddy V

Good. I also immediately thought about the umpires. The umpires think individually and share data. The umpiring schools are probably teaching this issue by now, so we can expect to see new umpires coming up in the near future who are more consciously and more adeptly correcting for catchers’ pitch framing. Time will tell. There will also be umpires who will actually over-correct against the more eager pitch-framing catchers. I expect to hear more about this in the coming years.

jianadaren
7 years ago
Reply to  Big Daddy V

Most framing gains have happened inside the zone. It’s not mostly a case of “umpire gets duped into calling a strike outside the zone” but rather “good receiving allows the umpire to be more convinced of actual strikes near the edges”.

It was the bad receiving that was turning strikes into balls.

Scott Lange
7 years ago

Interesting, but this:

“Pitch-framing is sufficiently important that baseball teams will prioritize it right into insignificance.”

… seems overstated to me. I mean, we’ve all known that home runs are important, and baseball teams haven’t prioritized it into insignificance. The relative impact of homers isn’t what it was when Babe Ruth was outhomering entire teams, but its still massive. Pitch framing starts out less significant than Ruthian homer discrepancies, and you’ve convinced me that it is shrinking some, but will it really shrink to insignificance? I’m not so sure.

Jeff Sullivan
7 years ago
Reply to  Scott Lange

Pitch-framing is a lot easier to learn and implement. It’s not *easy* easy, but it’s not like hitting home runs. Some guys who are good position players just aren’t home-run hitters, and if you try to make them home-run hitters, something else gets worse. In theory, most catchers could develop into pretty good receivers, provided they’re disciplined and they have strong wrists and forearms. If you work with a catcher on his framing, that won’t cause some change to his defensive approach that makes other parts of his skillset worse.

I don’t think it’ll ever become completely insignificant, but it’s just hardly going to matter. Maybe it’ll be comparable to baserunning, and maybe it’ll be even less significant than that.

Gabriel Gutierrez
7 years ago
Reply to  Scott Lange

Maybe there aren’t any 50 HR hitters anymore, but there are a lot more players with at least 20. We’ve come to expect that good players walk and hit HR at reasonable rates (20+). I think that’s what Jeff is talking about in that sentence. We should come to expect better pitch-framing catchers everywhere, starters and bench, especially with the modern emphasis on rest. A catcher will be still noted for his or her arm, but not so much as in the past few years for his or her framing skill.

Tommy Lasordid
7 years ago

Won’t pitch framing become obsolete when they automate the strike zone?

Xac
7 years ago
Reply to  Tommy Lasordid

Yep. I’m ready.

Fireball Fred
7 years ago

1) Framing has received so much attention in part because fielding stats for catchers were and are inadequate – just % CS still matters but doesn’t get as much attention as when it was the only stat.

2) One problem with framing is that (as Michael implies) it’s actually cheating, sort of, in that it fools the ump rather than the batter. Steps to avoid it, whether by human umps or automated replacements, amount to more effective enforcement of the rules.

AC of BNI
7 years ago
Reply to  Fireball Fred

The idea that pitch-framing is inherently dishonest relies on the assumption that the catcher has a better idea of the strike zone and strikes than the umpire does. However, pitch-framing isn’t just about “stealing” strikes; it’s also about making sure actual strikes are called as such. The catcher wants everything to be called a strike. Pitch-framing technique teaches him how to receive such that the pitch looks like a strike. But the catcher doesn’t always know whether or not the ball passed through the zone; he’s just hoping that he makes it look like it did.

The catcher may be aware that proper receiving technique will sometimes result in what was actually a ball being called a strike, but he doesn’t always know when it’s happening, and since poor technique will result in strikes being called balls, it’s the catcher’s job to receive properly. Of course, as everyone has pointed out, by the moment of reception the ball has already passed through the zone or failed to do so, and the umpire’s call, in theory, should be determined before the ball hits the glove.

Yes, once in a while the catcher is trying to get away with one, but the whole matter is more on the umpire than anybody else.

I would also suggest that the article perhaps slightly misstates the end result in the name of the sales pitch, but I suppose that’s how one has to write content. A catcher’s framing ability won’t disappear from consideration; it will become a baseline prerequisite. It won’t lose significance as a tool for measuring a catcher’s ability, but only for contrasting different ability levels. Just like being able to hit at least a little, or have some kind of arm, or throw back to the mound, or show up sober all worked their way into the checklist of minimum talent for the position, so too will some degree of framing.

Eastman
7 years ago
Reply to  AC of BNI

I agree with your last paragraph very much. When I was reading this, I wondered whether offensive output from catchers has decreased since framing numbers appeared. Now that teams can quantify framing, presumably framing skill will start be be selected for more heavily, with things like offensive skill being de-prioritized.

Also a stupid semantic comment for Jeff (I love the article and your writing in general). It rubs me, and presumably others, the wrong way when writers say that pitch framing has been “discovered” recently. The ability to quantify it has been developed recently, but I was coached on pitch framing in little league in 1998 or 1999. To me, using the word discovered discounts what coaches have known for a long time, and is a bit like a scientist citing new, high profile papers, and failing to cite older research conducted by previous generations of scientists.

Oldgoat_MN
7 years ago

I agree that umpires are almost certainly more aware of pitch framing than they used to be, but a catcher who sweeps his mitt outside the strike zone is still going to make it more difficult for an umpire to see the pitch as a strike than a catcher who holds his glove in tight to the plate.
I also agree that the difference between will continue to shrink.
Fun article.

Alex Poterack
7 years ago

So, there was a direction I kept expecting this article to go, and it never quite did. There’s evidence that the spread between the best and worst pitch framers is narrowing. This is taken as evidence that the average catcher is getting better at pitch framing. Do we have more direct evidence of that? Namely, are pitchers getting more strikes?

One of the big stories the past few years has been the marked increase in strikeouts. Has anyone attempted to determine how much of this increase, if any, is due to improved pitch framing?

Peter Jensen
7 years ago

“But pitch framing didn’t go away when the attention did. And, if I can give you some background: Pitch-framing statistics represented a major breakthrough. They were made possible by PITCHf/x around 2008 and 2009, and the greatest public efforts were made by Mike Fast, Max Marchi, Matthew Carruth and Dan Turkenkopf.”

Jeff – Pitch Fx did not initiate the research into pitch framing. Whether a catcher could improve a pitchers performance had been the subject of research since Craig Wright wrote The Diamond Appraised in the 1990s. Tom Hanrahan presented research in the SABR quarterly “By the Numbers” in August 1999 that catchers improved their pitchers ERAs as the catchers gained experience. He admitted that he didn’t know how they accomplished that improvement, but he suggested that pitch framing might be the cause. Phil Birnbaum followed with another article in “By the Numbers” in February 2000 that quantified what the spread in talent might look like for catcher pitch framing using the same method and values for changing a ball into a strike that John Walsh used 8 years later and that are still being used in pitch framing studies today. And Birnbaum’s estimate of +- 20 runs for the range of the best and worse framers is consistent with many of the models accepted today. Most of the studies that you reference from 2008 to 2010 used WOWY based only on PBP data and not Pitch Fx. Max Marchi’s was the first to utilize Pitch Fx location data as well as the first to use sophisticated statistical modeling methods. A good argument could be made that his methods and results are at least the equal and perhaps better than the mixed modeling that BP uses today.

dominik
7 years ago

I think Sabermetrics in front offices went from a smart math guy running some cool statistical regressions and finding an a big edge for cheap into supercomputers running billions of bytes of raw data to find smaller and smaller edges.

Something like in the early 00s when you could get good production for cheap dollars is not happening anymore now it is about very little edges.

Regarding framing I think it is two things, catchers are practicing it but also teams are less willing to put up with bat first catchers with mediocre receiving skills and more willing to put bad hitters there. Just a couple years ago Vmart and Napoli caught a lot of games. They are old now but I’m pretty sure that just a couple years ago Carlos santana would have stayed behind the dish longer and schwarber would probably stayed at catcher longer too in the “OPS era” Where people loved to put bats in premium positions even if they weren’t good in the field.

Yehoshua Friedman
7 years ago

I really expect a followup article or more than one over time looking at this issue from the umpirical point of view.

Matt
7 years ago

I’m family with one of the best graded MLB umpires the last handful of seasons, and I can confirm that they are keenly aware of the framing data, that they’ve been consistently improving league correct % on ball/strike calls. From what I’ve heard, they are grading out @ better than 95% consistently now.

Lanidrac
7 years ago

This is a good article, but keep in mind that pitch framing is only one of three major aspects of catcher defense, the others being his abilities to throw and to block balls in the dirt. A team may be willing to settle for a relatively mediocre pitch framer (not that they wouldn’t still try to get him to improve) if he excels in the other two areas. Of course, you also have to consider just how well he can hit. Pitch framing is important (may even be THE most important catching skill) but is still only a piece of the overall picture.

Brian
7 years ago

As “good” pitch framing becomes the norm, won’t the number of strikeouts league-wide only continue to increase? It seems eventually MLB will narrow the zone to try to bring some balance back between pitchers and batters. In that case, would there be even more of a premium on pitch framing?

Mr Scruff
6 years ago

I loved the sabermetrics era when the stat geeks laughed at coaches that insisted catcher defense mattered.

Glad the stat geeks finally caught up.