Paying for the ‘pen

Fantasy baseball owners, for years, have sworn by the adage, “Don’t pay for saves.” Relievers come and go, the theory states, and are so fungible that there’s too much risk involved in spending a high draft pick or big dollars on a closer who could lose his value almost immediately through an injury or a couple of rocky outings that relieve him of his role at the back of the bullpen.

Many sabermetricians have argued that relievers are overpaid in the real game as well, as the inherent instability of a pitcher expected to go roughly 50 innings per season can make large bullpen investments a risky proposition.

Still available on the free agent market today are a number of skilled righty relievers, many of whom have had no publicly reported interest at all despite effective 2012 seasons. Because teams are more and more wary of pouring money into their bullpens, right-handed relievers like Kyle Farnsworth, Brandon Lyon and Jon Rauch remain unsigned despite excellent 2012 campaigns.

I wanted to consider this problem from a more global scale than simply looking at contracts for a single reliever. In this article, I hope to examine whether more expensive bullpens are actually better. Some teams spend huge sums looking for a lockdown ‘pen, while others scrape the bottom of the barrel in free agency and rely on pre-arb and arbitration-eligible players to form a much cheaper relief corps.

The following table shows the top and bottom five teams in bullpen spending, from Cot’s Contracts Opening Day numbers. There are probably a few prorated portions of major league minimum salaries not included here, but this should give a pretty good sense of what teams are spending on their bullpens. I considered any player who made over half of his appearances in relief to be a member of the bullpen.

The table also displays the top and bottom five teams in bullpen runs allowed per nine innings, and Fangraphs’ Wins Above Replacement. While neither statistic is perfect for evaluating a reliever (or a bullpen in general), both offer useful information about the performance of the teams’ bullpens.

RA/9 is a rate stat, considering what bullpens did on a per-inning basis. While that’s probably more useful information than WAR, which is notoriously fickle in the small samples being considered with relievers, it leaves out the important consideration of how many innings the bullpen pitched. In general, bullpens that pitch fewer innings will be able to concentrate those innings in their best relievers, resulting in much lower run-scoring rates. WAR, on the other hand, gives strong consideration to the number of innings the bullpen pitched as well as how the relievers pitched in those innings, which is how the Rockies can end up in the bottom five in RA/9 (thanks to their starting experiment that put a huge number of innings on the team’s bullpen), but in the top five in WAR, after pitching more than 100 innings more than any other bullpen in baseball.

Top 5  Bullpen Payroll    Top 5       Bullpen RA/9    Top 5   Bullpen WAR
Yankees    $37,873,925    Reds        2.99            Royals           7.3
Giants     $25,199,350    Braves      3.02            Rays             6.7
Astros     $21,441,000    Rays        3.13            Rockies          6.7
Tigers     $21,032,000    Athletics   3.16            Braves          6.6
Reds       $20,872,833    Orioles     3.32            Orioles         6.4
Bottom 5                  Bottom 5                    Bottom 5
Twins       $9,740,000    Astros      4.84            Astros           1.7
Pirates     $9,626,000    Rockies     4.85            Cardinal         1.6
Mariners    $8,623,600    Cubs        4.88            Angels           0.9
Cardinals   $7,558,000    Mets        5.11            Mets            -0.1
Braves      $7,205,500    Brewers     5.11            Cubs            -1.5

Immediately, we see some big issues with teams that spend big on their bullpens. Just between the two of them, Mariano Rivera and Rafael Soriano earned $26 million last year, more than any bullpen in the game. The unfortunate injury to Rivera sent more than half of that dollar figure down the drain just over a month into the season. With Rivera out, the Yankees finished 10th in RA/9 and ninth in WAR, which certainly isn’t what they expected for a nearly $40 million investment in their relief corps.

On the other hand, the Braves had one of the cheapest bullpens in baseball, as well as one of the best. At just under $2.5 million, Eric O’Flaherty was the most expensive reliever on the team. Ridiculous production from pre-arbitration players like Craig Kimbrel and Jonny Venters allowed Atlanta’s bullpen to be dominant at a fraction of the cost of the ‘pens of divisional rivals like Miami and Philadelphia.

These charts show the correlation between the two statistics and bullpen payroll. As you saw in the table, there more or less isn’t one.

image image

This isn’t to say, of course, that there’s no value in signing a top reliever. If a team wants to improve its bullpen, and doesn’t feel like dipping into its roster or prospect pool to do so, signing a good reliever is probably going to make the team’s bullpen better (duh). However, the teams with the most effective bullpens, at least last year, were largely able to dominate through their accumulation of young relievers who can lock down the late innings without breaking the bank. Teams are starting to get wise to the fact that a bullpen doesn’t have to be expensive to be good, and several prominent relievers appear to be suffering the consequences.

Newest Most Voted
Inline Feedbacks
View all comments
10 years ago

Very interesting article and analysis.  Love your work, keep up the good job!

I would like to point out Tom Tippett’s work at Diamond Mind before he took off to join Boston to help with their defensive analysis, where he found that teams now have to have a good bullpen if they hope to make the playoffs.  So, as you put it, it is a “duh” to say this, but clearly having a good bullpen is necessary for a team hoping to be a playoff team.

The key, as you tried to address here, is how much should a team be paying for that, and are they being efficient at it?

Frankly, I don’t think that WAR does a very good job of addressing the leverage situation with relievers.  So that’s not a good measure to use at the moment.

The issue with RA/9 – and this is not your fault either – is that the RA/9 during the season is not the RA/9 that the team has during the playoffs. 

For example, much was made at the start of the series about how the Reds bullpen was much better than the Giants.  However, while the Giants cycled through a number of not very good relievers, the Reds had a core set that pitched the whole season, basically.  So, when I compiled the ERA for the relievers on the Giants playoff roster and compared it with the Reds, they were basically the same (I recall the Giants having a slightly higher, but still excellent below 3 collective ERA).

Another confounding issue is that teams might just be in different phases of player development.  The Braves currently has the cheapest pen, but has that been their strategy?  Maybe they just happened to develop a number of relievers at the same time, and thus it is cheap now, but as these players go through arbitration and then free agency, if the Braves keep them, they will get very expensive.  And, of course, the Yankees just like to buy and buy, so they should never show up well on any level of budget efficiency. 

Then take the Giants.  They have developed two very good closers in Wilson and Romo, but nothing much else, so they had to go to trades and free agency for vets to fill the gaps, though they did recently pick up younger/cheaper players in Kontos and Mijares.  If they had choosed the young route, then they would have sacrificed a chance at the playoffs just to follow that strategy, which, of course, no team will do.

So the way I see it, teams will have a mishmash of bullpen construction, depending on their luck in development.  If you hope to compete, you will usually need to pay big money to somebody, as few teams can 1) create a bullpen solely of cheap pre-FA players and 2) keep developing relievers to replace the ones that get expensive.  Sometimes you need to pay up to have the good bullpen, if you want to be competitive.  So I don’t know that we’ll ever answer the question of bullpen efficiency tendencies easily.

Though I would note that Giants fans have been up in arms over the recent drafts as the Giants bulked up on relief arms, particularly last season.  And they have a particularly expensive pen that’s getting old.  It will be interesting to see how their pen evolves as the older players become ineffective, will they generate enough to replace, or will they have to go out and buy more?

10 years ago

Speaking of Tom’s work, he stopped doing the annual updates of his work when he joined the Red Sox, perhaps you can look over what he did and update them for today, to see how things have changed (or not).  Or even do a series covering the gap between then and today, so that there is a complete series.

Doug W
10 years ago

OGC… Appreciate the feedback. I totally agree that neither of these stats are particularly good for evaluating bullpen performance. I think it’s a tough problem that we need to figure out creative ways to attack. To me, leverage index is an important place to start, so we can do a better job of understanding which bullpens are at their best when it matters most and not penalizing them (as much, at least) for mopup relievers allowing runs in garbage time that don’t really matter nearly as much in the grand scheme of things.

10 years ago

Great job KC on that bullpen WAR?

The RA/9 is dominated by the stats of Rodney, Kimbrel, and Chapman. Honorable mention for Sean Doolittle, but mainly it’s those 3 guys.

Probably Axford is the guy killing the Brewers stats, too.

Yankees and Giants succeeded by paying. Tigers didn’t.

The best GMs are taking this beyond a financial question – the game is to get the performance at a low salary. !Viva Clube Sandy Rosario!

10 years ago

rubesandbabes, what is this trend with Sandy Rosario?  Is that the same guy the Giants picked up on waivers?

10 years ago

OT:  Doug, another study that Tippett did long ago was a historical examination of DIPS and he found a lot of different categories of pitchers who were able to succeed by defying DIPS, like knuckleballers and crafty lefties (his focus was mainly, however, on how did pitchers manage to succeed, if I remember right, been 10 years I think, and some of them defied DIPS).  But one thing he left out was what the percentage is, of all pitchers, who were able to be successful despite violating some rule of DIPS.  Do you know if such a stat exists anywhere?

Given that Cain is close to statistical significance in terms of IP (per latest numbers published by BP’s Matt Swartz a while back, in update of Pizza Cutter’s seminal study) and below league BABIP, he’ll be one of the few crafty righties around, and I wonder how rare it is for pitchers to succeed with a lower BABIP.

Doug Wachter
10 years ago

The Royals, for what it’s worth, have a number of interesting little bullpen arms. Literally, when it comes to Tim Collins. Can’t explain why I get so irrationally exuberant over the short guys but there’s just a certain type of joy that comes from watching a dude who’s 5’7” blow a 96 MPH fastball by somebody who’s got 8 inches on him. And Jose Altuve is in the AL now… excuse me while I check the schedule for the first Astros-Royals matchup.

Doug Wachter
10 years ago

OGC… An interesting question, and one that would take some deep digging. If I come up with anything it’ll definitely be article fodder.

10 years ago

Nice job, Doug!  It would be interesting to look at free agent contracts only—your approach will favor teams with the young, strong arms that aren’t yet eligible for free agency. That’s interesting info, but it doesn’t really address the question of whether a specific team with no other options should dip into the free agent market.