How are closers being leveraged?

Last week, Dave Studeman took a look at Win Probability Added numbers so far this season. Drawing my inspiration from Dave, I decided to traipse through Dave Appelman’s FanGraphs and see what piqued my interest.

Best for Bullpens

Dave noted last week that WPA is his metric of choice for measuring bullpen effectiveness. It makes sense: the job of the bullpen is to pitch well during game-breaking situations. WPA helps us see whether the bullpen broke the game.

Another reason why WPA is a fun way to look at bullpens has to do with strategy. With rare exceptions, managers aren’t able to selectively use their hitters or their starting pitchers in only the most important situations (Chris Jaffe’s excellent work momentarily ignored). A look at the distribution of leverage index (LI) for hitters confirms this:

While the plot doesn’t prove that LI is random, it is pretty good evidence that selective leveraging of hitters, if it happens, is very small effect indeed. The difference between the players with the highest LI (Brad Hawpe, 1.31) and the lowest LI (Adrian Beltre, 0.8) is small, and no player is greater than three standard deviations from average. The correlation between LI and OPS is virtually nonexistent. In other words, what we see is consistent with the fact that managers aren’t allowed to selectively use their best hitters in the most important situations. (Aside: How crazy would this sport be if managers to be able to selectively leverage their hitters? I bet Eric Wedge would love to insert Travis Hafner any time there are men on base!)

But, it is permissible for managers to to save their best relief pitchers for the most important situations. Second-guessing a manager’s choice and timing of relief is part of being a fan, and some managers are never forgiven for making the “wrong” choice. A quick look at the distribution of LI for relievers is illustrative:

(For relievers, I am using a measure called “gLI“, which is the LI of the situation when the reliever enters the game. No credit if for relievers who create their own drama!)

If relievers were inserted into the game at random times, then we would expect this plot to look just like the one for hitters. But that isn’t the case. This curve is much wider and the standard deviation is six times greater! Again, it’s evidence of what we already knew: Unlike for hitters, managers are allowed to use their best pitchers in the most critical situations.

The Most Boring Teams

Not every team has a lot of critical situation, however. Some teams just play boring games, with little in the way of low-scoring late-inning drama and much in the way of blowouts. One way to look at this is to check the leverage index for a team’s crew of relievers (Dave ran a similar list last week):

Team             LI
White Sox       1.54
Cubs            1.41
Reds            1.41
Athletics       1.37
Diamondbacks    1.36
Tigers          1.31
Rockies         1.25
Pirates         1.24
Padres          1.22
Phillies        1.21
Indians         1.19
Marlins         1.19
Brewers         1.17
Nationals       1.14
Dodgers         1.13
Orioles         1.13
Braves          1.11
Blue Jays       1.08
Devil Rays      1.08
Giants          1.07
Twins           1.05
Astros          1.03
Yankees         1.02
Angels          1.01
Royals          0.99
Red Sox         0.97
Mets            0.92
Cardinals       0.88
Rangers         0.73
Mariners        0.65

Remember, the higher the leverage, the more critical the situation is. The list shows how important of a situation a team’s relievers are being thrown into, which is a kind of proxy measure of how tense a team’s games are.

Chicago fans of both persuasions are being treated to exciting games this year, even if they don’t always come out on the winning end. On the other hand, despite being two games over .500, the Mariners haven’t played very exciting baseball. The Mets and Red Sox are exerting their dominance by dispatching opponents without drama; the Brewers and Indians are winning and putting on a show at the same time.

A Closer Look

If the closer is typically the best pitcher in the bullpen, and the best pitcher ought to be used in the most critical situation, then it stands to reason that the closer ought to be among the league leaders in leverage index. That is more-or-less true. Closers litter the list of gLI leaders:

Pitcher         Team          gLI
Trevor Hoffman* Padres        2.34
Matt Thornton   White Sox     2.25
Joakim Soria*   Royals        2.23
Salomon Torres* Pirates       2.19
Huston Street*  Athletics     2.13
Todd Jones*     Tigers        2.11
Brian Stokes    Devil Rays    2.07
Jose Valverde*  Diamondbacks  2.04
Alberto Arias   Rockies       2.04
Greg Aquino     Brewers       2.04

(>5 IP)
* = closer

But some closers don’t show up until you go way, way down the list. You have to go through over 50 relievers before you find Brian Fuentes, Armando Benitez, JJ Putz, Dan Wheeler — even Mariano Rivera! Does this mean that they are being poorly leveraged? Not necessarily. As we saw earlier, not all teams play games with lots of high-leverage situations. It’s not all that surprising that the Mariners closer hasn’t seen a lot of critical situations, given where the team bullpen hasn’t had an opportunity to make an impact.

As a quick-and-dirty way to correct for this, we can ratio a pitcher’s gLI (the leverage when he enters a game) to the LI (the average per-plate-appearance leverage) of his bullpen mates.

Pitcher         Team          gLI/Team LI
J.J. Putz*      Mariners      2.26
Joakim Soria*   Royals        2.25
Jon Papelbon*   Red Sox       1.98
Trevor Hoffman* Padres        1.92
Brian Stokes    Devil Rays    1.92
Joe Nathan*     Twins         1.9
Akinori Otsuka* Rangers       1.86
Al Reyes*       Devil Rays    1.84
J. Isringhausen*Cardinals     1.84
Jack Taschner   Giants        1.82

The top four, and eight of the top nine, are nominally the closers for their team, and a number of these pitchers did not show up on the previous list. Putz, who was nowhere to be found earlier, is now seen to be pitching in situations over two times as important as his bullpen mates. Just missing this list is Salomon Torres, Pittsburgh’s closer. Looking at the nominal closers for all 30 teams, only Henry Owens of the Marlins and Ryan Wagner of the Nats aren’t being used in more critical situation than their bullpen mates.

Just for Fun

This sort of look is fun, but there aren’t any hard and fast conclusions to be drawn. It looks like a number of closers are being used quite well, but it is important to note the limitations of looking at leveraging without going into more detail. For one thing, it’s not possible for one pitcher to be called upon in all the important situation since one guy can only pitch so much. The Athletics, for example, are using Huston Street in high-leverage situations but not so much more so than his teammates that his gLI/Team LI ranks among the lead leaders. But the A’s have played a number of close games, and Street can’t pitch in all the high-leverage spots. The relative strength of other pitchers in the ‘pen plays a role, as well. Joakim Soriais getting most of the action for the Royals, but then what other choice do the Royals have (Jimmy Gobble fans – spare me your wrath)? Perhaps the Royals are leveraging him properly given the strength of their bullpen, but a deeper staff or relievers might make a closer look as if he isn’t being leveraged properly.

Still, data from the WPA framework gives us a unique insight on relief usage. It’s every fan’s duty to second-guess their manager’s bullpen usage, and using leverage and win probability data helps us do so more intelligently.