The Unofficial Rules: Of Holds and Blown Saves

David Robertson’s appearance in the AL Wild Card Game sent Shane and Paul down the rabbit hole. (via Keith Allison)

Preface: This article would not have been possible without the dedicated and slightly obsessive research work of my friend and occasional baseball wingman, Paul Golba. Readers should consider him the co-author of this piece. Any errors in it, however, are entirely mine.

The 2017 American League Wild Card game wasted no time in getting crazy. Yankees starter Luis Severino, destined to finish third in the AL Cy Young Award voting, recorded one out before being pulled, already trailing 3-0. His reliever, Chad Green, stifled the Minnesota Twins’ rally, retiring five straight batters. While he held the line, the Yankees came back to tie the game in the home first then edge ahead, 4-3, in the second.

Green’s perfection did not survive this prosperity. He loaded the bases in the third, with a single and two walks sandwiching a flyout. David Robertson took his place on the hill, coaxed a Byron Buxton groundout that pushed across the tying run, and finished the frame with a strikeout. Despite working out of a nasty jam with just one run scoring, Robertson had given up the lead.

As the game went to commercial, I dashed something down in the writing notebook I was using to record playoff highlights and observations. It read, Does that count as a blown save in the third inning? A trivial question, but much of baseball’s pleasure comes from trivial things.

The short answer, which I found the next morning at Baseball-Reference, was yes. The other short answer, which I found at FanGraphs, was no. It turns out there’s a long answer, because what my question is asking isn’t as clear as I had thought.

What It All Means

It seemed clear to me at the time. Once the Yankees took the lead, Green was in line to be awarded the win, assuming the Yankees stayed ahead. He had entered in relief, and thus wasn’t bound by the five-inning requirement for starting pitchers.

When Robertson relieved Green, he was eligible for a rule book save. He came in with a lead, having relieved a pitcher eligible for the win, and the tying run was on base. If Robertson held the lead and finished the game, the save would be his. Granted, in practical terms it was highly unlikely he would pitch the last 6.2 innings of an elimination game, but not impossible. There has been a longer save in major league history—and in a game that was a no-hitter through eight, no less.

I almost want to do a piece on that game now. Maybe another time.

Robertson could have had a save, but he let the (inherited) tying run score. This appears to fit the definition of a blown save. That, though, is the sticking point: There is not full agreement on what a blown save is.

The blown save, with its cousin the hold, have become familiar statistics, gaining in repute as relief pitching has gained in prominence. Non-closing relievers are naturally fond of holds, giving some acknowledgment of their successes, and the statistic has found its way into arbitration hearings, buttressing appeals for higher salaries.

It’s easy to assume blown saves and holds are official statistics. I came to assume it, or at least to act as though they were. They are not. The Official Rules of Major League Baseball directs scorers how to assign wins, losses, and saves, along with all the familiar in-game events like hits, errors, and sacrifices. It is silent on holds and blown saves.

The save itself began as an unofficial stat. Rules for it were laid down by sportswriter Jerome Holtzman before the 1960 season, although Holtzman said other versions existed as early as 1952. Holtzman had his version adopted by The Sporting News, which listed saves in its box scores and used the formula in deciding its Reliever of the Year awards. The influence of The Sporting News, still potent at that time, led to the stat becoming official in 1969, though its definition would be tweaked in 1974 and 1975.

The hold had a somewhat similar origin. It was introduced by John Dewan and Mike O’Donnell in 1986, appearing in the figures of STATS, the baseball data company where Dewan was part-owner and CEO. The rising profile of STATS raised that of holds as well, leading other authorities—but not the MLB rule book—to adopt their own versions of it.

The blown save arrived shortly afterward, in 1988. Rolaids adopted it as part of its formula for choosing the Rolaids Relief Man Awards, given to relievers in each league. The sponsored award lasted until 2012, giving way to MLB’s official Trevor Hoffman and Mariano Rivera Awards. Again, the high profile of the group instituting it helped spread the blown save but did not make it an official statistic. Interestingly, MLB.com’s Gameday box scores assign both blown saves and holds despite the lack of official MLB status.

Baseball Coding with Rust – Intro

by Eli Ben-Porat

Getting into the nitty gritty of baseball action via a new(er) programming language.

While the save has a set meaning (with a possible exception I’ll mention later), different sources have different definitions for both holds and blown saves. Robertson’s Wild Card performance was a blown save to Baseball-Reference and Gameday, but not to FanGraphs. Similarly, in the other 2017 Wild Card game, Robbie Ray of the Arizona D-backs got a hold from Baseball-Reference and Gameday, but not from FanGraphs. (Jorge de la Rosa and Archie Bradley were credited with holds in that game by all three authorities.)

What are the definitions? We looked at that question using several leading sources and found not only the larger-scale rules and differences thereof, but some interesting minutiae. The six sources we observed are:

1. Baseball-Reference
2. Baseball Info Solutions (as used by FanGraphs, and in the annual Bill James Handbook)
3. STATS, Inc.
4. Elias Sports Bureau (as used by ESPN.com)
5. USA Today
6. MLB.com Gameday

The core definition of a hold is that a reliever must enter a game in a save situation, record at least one out, and depart without having given up the lead at any time. This is the definition Dewan and O’Donnell originally crafted in 1986. All six sources use those three criteria today—but this was not always the case.

Up until 1994, USA Today was using STATS’s data, but that year the newspaper began using the company SportsTicker instead. It came up with an altered definition of the hold, one that did not require a pitcher to get any outs. It kept this definition for some years despite it rewarding pitchers who did literally nothing right in their mound stints. Today, however, USA Today is back on everybody else’s page: It requires an out to award a hold.

Another area of unanimity is that the hold-eligible pitcher must be entering after a win-eligible pitcher. If the starter departs with a lead but pitched fewer than the five innings generally needed for him to qualify for a win*, the reliever coming immediately after him cannot receive a hold or take a blown save. There’s no save situation, because there’s no potential winning pitcher yet for whom to save the lead.

*If a game goes only five innings before being stopped and called an official game, the starting pitcher can go four innings and still receive a win. This is, of course, quite a rare occurrence.

The six authorities also concur on a point that finesses the standing definition of a save. The rule book recognizes a save for a pitcher who pitches the final three innings or more of a game, regardless of the lead his team is holding when he enters. Perhaps the supreme example of this is when Wes Littleton entered a 2007 game in relief at the start of the seventh inning with his Texas Rangers leading the Baltimore Orioles, 14-3. He pitched three shutout innings while his team was tacking on 16 insurance runs and got the three-inning save for a game his team won, 30-3.

None of the six authorities recognizes a hold for a reliever entering in such a situation. Either the tying run must be on base, at bat, or on deck, or the lead must be at most three runs with at least one inning left to play. Even if the reliever throws three innings or more (without finishing the game), they won’t award a hold—with the exception of MLB.com Gameday.

It is alone on the three-inning hold, but not on the related blown-save scenario. MLB, USA Today, and Elias/ESPN will assign blown saves to relievers entering with three innings or more to go and a lead of at least four runs. It is rare these days that a reliever will be allowed to stay in a game long enough to blow the lead in such a situation, but it does happen. Miami’s Brad Ziegler triggered Gameday’s rule in a May 5, 2017 game against the New York Mets. Baseball-Reference, BIS, and STATS do not assign such blown saves. For them, if you cannot receive a hold, you cannot suffer a blown save.

Parenthetically, the BIS/Bill James Handbook definition of a save situation is slightly out of step with the rule book. For a three-inning save, the Handbook’s glossary definition says that the reliever must pitch effectively. While this qualification was in the official rule book once, it is long gone today. It’s quite possible the Handbook just has not cleaned up an old glossary entry.

A condition implicit in the definition of a hold is that the pitcher receiving the hold is not the finishing pitcher. Otherwise, he would be awarded the save. This seems straightforward, but one of our authorities, Baseball-Reference, does not follow this rule. B-R makes it possible for someone to get a hold for finishing the game if he enters in what is  considered a save situation but isn’t awarded the save.

It is the authors’ opinion that Baseball-Reference has automated its awarding of holds (among other statistics), and that the game-finished-plus-hold falls through a loophole in the algorithm it uses. In other words, we think it’s a programming mistake.

Something that isn’t a mistake but may qualify as an anomaly is when a pitcher is awarded both a win and a hold in the same game. This can happen when two or more relievers could receive a win by scorer’s discretion: A latter reliever gets a hold for maintaining a potential save situation but then is given the win as well. The rule book says you cannot get both a win and a save in the same game, but as it’s silent on holds, the win-plus-hold is a possibility.

Most sources don’t allow W+H combinations, but STATS does. BIS also did, at times. FanGraphs and the Bill James Handbook track holds from 2002 to the present. They allowed the win-plus-hold from 2002 to 2008, stopped in 2009, resumed for 2010 and 2011, then stopped again. Had one not known that both FanGraphs and BJH use the same source, the exact match between the two, along with their agreement on other fine points of hold and blown save rulings, would have led you to that conclusion.

(Which is actually what happened, because I never bothered until too late to scroll down to the bottom of the FanGraphs page to see whose data it uses. FanGraphs does make rare manual adjustments, causing an occasional disagreement between game box scores and pitchers’ career logs. See this 2010 game, where Jeremy Accardo, Scott Atchison, and Scott Schoeneweis had blown saves assigned to their career records, but not in the box score. Otherwise they match BJH perfectly, and I feel like a dope for making my co-writer prove it the hard way. Sorry, Paul.)

The main remaining technicality that affects how holds are awarded is whether  the game is official (i.e., has gone five innings) when a reliever enters. In theory, before that five-inning mark there cannot be a winning pitcher, so the question of saves and holds and blown saves is academic. This may seem to be a quibble, something that can be disregarded once a game has played out to official length, but the majority of sources disagree. BIS, STATS, Elias, and USA Today will not award a hold or a blown save to a pitcher who enters a game in the fifth inning or earlier. Baseball-Reference and MLB.com will, for both.

This finally ties back to my original question about the Wild Card game. B-R and MLB considered Robertson to be in line for a save in the third inning, the only two sources out of the six that did so, and thus the only two that hung him with the blown save when Minnesota got the tying run across. I’m surprised my speculation ended up a minority opinion, or at least by the specific reason why, but baseball remains full of surprises.

On the Fringes

There are even rarer and more abstruse situations that can affect how holds and blown saves are assigned. If you thought wrangling about three-inning holds and early relief stints was odd, you may want to brace yourself for mid-plate appearance pitching changes and hurlers who leave, then come back.

If a reliever takes the mound in the middle of his predecessor’s plate appearance, whether a walk is assigned to the departing or the entering pitcher depends on the count when the change is made. The rule book lays out the specifics. In short, for any three-ball count, or a 2-0 or 2-1 count, the previous pitcher is on the hook; otherwise, it’s the new pitcher’s responsibility.

One might think this same division would apply to determining whether the new pitcher has a save situation, but it does not. We examined the policies of FanGraphs and Baseball-Reference and found them divided.

Baseball-Reference follows the rule book. A reliever inheriting one of the aforementioned counts who then gives up a walk will have his save situation eligibility calculated after the walk is completed. This can create a save situation where one hadn’t existed, such as by bringing the tying run on deck. This has happened twice from 1975 to 2017, including in this 2008 game for Santiago Casilla. It also can erase a save situation that had existed by forcing in the tying run. The latter has happened three times from 1975 to today, though one instance from 2003 was complicated by involving a wild pitch and may have been decided in error.

FanGraphs ignores the rule book. It determines a save situation the moment a reliever enters the game, whatever the count he inherited.

We won’t speak to the “proper” definition, as both positions are defensible, and we can see it either way. (In fact, we each see it in different ways.) I am more skeptical about B-R’s stance in games in which a pitcher leaves the mound for a field position then returns to pitching.

Such a maneuver is unusual but certainly not unheard of. A new pitcher will be brought in to face one batter, perhaps more, while the previous pitcher is stored somewhere else, usually the outfield, to await his return to the mound to face more hitters. Whitey Herzog liked to pull this move, and a bench crunch once forced Davey Johnson into several innings of it in a 1986 Mets-Reds game that was one of the wackiest ever played.

What happens with save situations when a pitcher enters, exits, and re-enters? We were able to research this only for Baseball-Reference, as the maneuver hasn’t been done recently enough to show up in FanGraphs or STATS. The answer starts sensible but quickly turns bizarre.

B-R will assign only one hold per pitcher per game, so two stints cannot get you two holds. To get that one hold, you have to enter in a save situation and depart for the final time with that lead never having been lost, even if another pitcher intervenes. Here is an example from 2014, with Tony Sipp getting the bracket hold.

If the lead is lost during your temporary absence, not only do you lose the hold, you get a blown save, as does the pitcher who replaced you and actually lost the lead. See Les Lancaster and Paul Assenmacher in this 1990 contest. Then see Jeff Dedmon and Assenmacher in this 1986 game. Both times, Assenmacher was the meat in the blown-save sandwich, in the only two occurrences in more than 40 years. This could justifiably be called the Assenmacher Rule.

The blown save in absentia looks weird and feels unfair, and it may be another artifact of the algorithms Baseball-Reference uses to calculate blown saves and holds. Perhaps this and the other anomalies is a trade-off B-R accepts for efficiency in analyzing almost 2,500 baseball games a year.

Adding It Up

Do all of these varying ways of counting holds and blown saves make a difference in the numbers? They do, but not a big one. For the 2017 season, the gap between the most generous and most stringent authorities was about two percent for blown saves, and not quite one and a half percent for holds.

It would be a rarity for any pitcher to have his numbers change by more than one, either way, depending on which source one consulted. This is a relief in that the competing rules don’t throw everything topsy-turvy, but also an annoyance in that you do have to pay attention to whose numbers you are using. If you decide instead to ignore the numbers, in commitment to cutting-edge process-over-results sabermetrics of the type that scorns the win and even the save, this gives you that much more backing to do so.

The divergent definitions of holds and blown saves matter most only way out at the margins. That, of course, is where this trip began, with a Wild Card game that went way out to the margins quite early on. The detail of whether Robertson got a blown save didn’t matter to the Yankees or Twins, and perhaps it didn’t even matter to Robertson. It did, though, matter to at least one fan who got a little pleasure out of asking the question and seeing it answered both ways.

References and Resources

• 2017 Official Rules of Major League Baseball
• Baseball-Reference
• FanGraphs
• MLB.com Gameday
• Various annual editions of The Bill James Handbook
• ESPN.com
• USA Today
• Baseball-Reference Bullpen
• Retrosheet
• Mark Newman, MLB.com, “Holtzman helped ‘save’ baseball”
• Society of American Baseball Research, SABR bio sketch of John Dewan
• Alan Schwarz, The Numbers Game
• Rob Neyer, ESPN.com, “On Holds”
• Shane Tourtellotte, The Hardball Times, “Baseball’s craziest game?”