Relievers in the rough

What makes a good reliever? It’s a complicated question, and if you’re putting together a bullpen, it may not even be the right question to ask.

Most relievers didn’t start that way. At some point, perhaps in college, the minors, or even midway through a major league career, they were shifted out of a starting role. Sometimes it’s due to a 98-mile-per-hour fastball that screams, “Make me a closer!” More often, it’s due to ineffectiveness as a starter.

But not all failed starters make it as relievers. Some, like Eric Gagne, become far more valuable in the late innings than they ever would’ve been in the rotation. Others, like a large percentage of Triple-A relievers, aren’t good enough as starters or relievers to contribute at the big league level.

Until you stick a mediocre starter in the bullpen and give him a try, how do you know which category he falls into? Rather than asking what makes a good reliever, let’s ask: What makes a mediocre starter a potentially good reliever?

That adjustment both narrows and widens the question. Generally, good pitchers make good relievers. If Tim Lincecum or Felix Hernandez suddenly started pitching out of the bullpen, we could reasonable assume they would pitch well. So really, identifying good relievers is similar to identifying good pitchers.

The other question forces us to look deeper. What makes relief pitching different, allowing some pitchers to successfully specialize? Here are three ideas:

1. They pitch well in short bursts. Perhaps they have the stuff (one or two great pitches) to retire batters the first time through the order, but not enough to fool them again.
2. They have a large platoon differential. Relievers, even those who aren’t known as platoon specialists, can be deployed in a way that protects their weak side. By contrast, an opposing manager can stack his lineup against a starter with a large differential.
3. They pitch well out of the stretch. This is part of the job. With the exception of closers and some setup men, relievers are frequently called on to clean up others’ messes. Many relievers, in fact, always pitch from the stretch.

Some of these skills can be identified statistically:

1. We can look at a starter’s numbers his first time through the lineup compared to subsequent times through the lineup. This isn’t a great way to identify future relievers, since starters learn to pace themselves. In a relief role, they might throw fastballs harder, for instance. It’s still worth looking at, though. Even with less fastball velocity, a starter with a great fastball-change-up combination may be able to cruise the first time through the lineup, but suffer the second and third times as he exposes weaker pitches.
2. Platoon differentials are easy to express statistically.
3. Assuming that a pitcher nearly always throws from the stretch when runners are on base, we can look at a player’s splits with and without runners on base.

To find starters better suited to relief, then, we would expect to see three things: (a) a big difference between his performance the first time through the lineup compared with subsequent times, (b) a big platoon differential, and (c) strong numbers with runners on base relative to numbers with the bases empty.

As is usually the case with attempts to model hypotheticals like these with statistics, there are limitations. As noted, looking at performance the first time through the lineup doesn’t tell us how a starter would throw given only one inning of work. And the third stat presents the most problems: Usually if a pitcher is throwing with men on base, he’s already had some problems. But if a reliever comes in with men on base, the fact that he’s pitching from the stretch doesn’t reflect his own performance.

Despite these caveats, let’s press forward and see what we can come up with.

2009 Triple-A

As we approach spring training and teams start to decide whom to invite to camp, let’s look at a pool of players who are generally on the cusp: Triple-A starters. Last year, 104 pitchers threw 100 or more innings for their Triple-A team.

First, let’s look at how they did each time through the lineup. The average starter in this group posted an FIP of 4.37 the first time through the order, 4.62 after that. About 60 percent of them had better results against the first nine batters they faced.

Here are the 10 who suffered the most in the second time through the order and beyond, suggesting that in many of these cases, two innings might be as much as they should work:

Name             Team              IP  FIP 1st time  FIP 2nd+  FIP ratio  
Brad Salmon      Salt Lake      111.7          3.81      6.49       1.70  
Clay Buchholz    Pawtucket      102.0          2.43      4.02       1.65  
Dana Eveland     Sacramento     125.7          3.37      5.49       1.63  
Gustavo Chacin   Lehigh Valley  103.7          3.47      5.24       1.51  
Tobi Stoner      Buffalo        100.0          3.31      4.86       1.47  
Jason Jones      Rochester      137.7          4.00      5.85       1.46  
Rodrigo Lopez    Lehigh Valley  103.3          2.77      4.03       1.45  
Carlos Carrasco  Lehigh Valley  116.3          3.24      4.64       1.43  
Lenny DiNardo    Omaha          156.7          2.62      3.74       1.43  
Carlos Torres    Charlotte      132.7          2.48      3.51       1.42

At least at the Triple-A level, some of these guys were solid pitchers regardless of time through the order. But the performance of someone like Jason Jones suggests that a shift to the ‘pen may immediately make him look like a much better pitcher.

At the other end of the list is knuckleballer Charlie Haeger, who posted a 7.21 FIP the first time through the order and a 4.41 FIP thereafter. Lest you think it might be a feature of the knuckler, Charlie Zink was a much more expected 5.28/6.72.

Platoon differentials

Let’s do the same for the 104 starters, considering their FIP against lefties and righties. The average starter from this group posted a FIP about 22 percent worse against their weaker side. Here are the ten whose differentials were the largest:

Name              Team              IP  FIP vLeft  FIP vRight  FIP Ratio  
Clay Buchholz     Pawtucket      102.0       4.63        2.44       1.90  
Kei Igawa         Scranton/WB    155.3       2.87        5.24       1.83  
Douglas Fister    Tacoma         111.3       4.96        2.93       1.69  
Matt Kinney       Fresno         158.3       7.29        4.43       1.65  
Charlie Zink      Pawtucket      140.7       4.58        7.45       1.63  
Jerome Williams   Sacramento     119.0       6.99        4.34       1.61  
Chuck Lofgren     Columbus       100.3       3.50        5.61       1.60  
Rodrigo Lopez     Lehigh Valley  103.3       4.46        2.83       1.58  
Enrique Gonzalez  Pawtucket      143.3       6.25        3.98       1.57  
Jack Egbert       Charlotte      115.0       5.63        3.61       1.56

There’s Buchholz again! Given that 20 percent of the pitchers in this sample had a ratio below 1.1, it’s striking how big some of these gaps are. Maybe, just maybe, Kei Igawa could be successful in a limited major league role.

The stretch

These pitchers were a bit better with runners on than with the bases empty: a 4.48 FIP compared with a 4.57 mark. There’s no clear bias either way: About half of the starters were better with runners on; the other half were better with the bases empty, presumably pitching out of the windup.

Here are the 10 who had the most relative success from the stretch:

Player          Team                 IP  FIP None On  FIP Men On  FIP Ratio  
Josh Towers     Scranton/WB       104.7         5.90        3.53       1.67  
Scot Drucker    Toledo            114.3         5.75        3.60       1.60  
Kris Johnson    Pawtucket         103.3         5.62        3.66       1.54  
Chris Seddon    Tacoma            142.3         6.58        4.37       1.51  
Steven Hammond  Fresno            163.7         7.35        5.15       1.43  
JR Mathes       Iowa              136.7         5.12        3.73       1.37  
Kevin Mulvey    Rochester         153.7         4.68        3.44       1.36  
James Simmons   Sacramento        117.7         5.09        3.75       1.36  
Brandon Hynick  Colorado Springs  159.7         5.51        4.14       1.33  
Charlie Haeger  Albuquerque       145.3         5.89        4.51       1.31

For some reason, I have a hard time imagining Josh Towers as a reliever, but at least, he was successful out of the stretch last year.

One odd tidbit at the other end of the list: Chuck Lofgren, chosen by the Brewers in the Rule 5 draft, was by far the most extreme in the other direction, amassing a 3.59 FIP with the bases empty and a 7.59 FIP with runners on. If that tendency lasts, he’ll have an even tougher road ahead of him to make the Milwaukee bullpen this spring.

Bullpen bound?

To combine these three qualities, I divided the pool into quintiles and assigned each pitcher a number from 1 through 5 for each category. For instance, the 10 players listed above as the strongest out of the stretch were given a 5 for that category, while Lofgren, at the other end of the spectrum out of the stretch, was given a 1. Thus, the maximum overall score is 15 (top quintile in all three categories) and the minimum is 3 (bottom quintile in all three categories).

No one scored a perfect 15, but there is a 14 and a handful of 13s. Here are the 17 pitchers who scored 12 or above, along with their quintile scores for each category:

Name              Team              IP  Lineup Rank  Platoon Rank  Stretch Rank  Total Rank  
Chad Reineke      Sacramento     140.0            5             4             5          14  
James Simmons     Sacramento     117.7            3             5             5          13  
Kei Igawa         Scranton/WB    155.3            3             5             5          13  
Tobi Stoner       Buffalo        100.0            5             3             5          13  
Willie Collazo    New Orleans    132.3            3             5             5          13  
Wade Davis        Durham         165.0            3             5             5          13  
Ty Taubenheim     Indianapolis   106.0            5             4             4          13  
Rodrigo Lopez     Lehigh Valley  103.3            5             5             3          13  
Charlie Zink      Pawtucket      140.7            5             5             3          13  
Josh Towers       Scranton/WB    104.7            2             5             5          12  

Name              Team              IP  Lineup Rank  Platoon Rank  Stretch Rank  Total Rank
Scot Drucker      Toledo         114.3            3             4             5          12  
Andy Mitchell     Norfolk        119.3            4             4             4          12  
Ramon Ramirez     Louisville     130.7            4             4             4          12  
Carlos Hernandez  Durham         115.0            4             5             3          12  
Kevin Pucetas     Fresno         165.3            5             4             3          12  
Travis Blackley   Reno           115.0            5             4             3          12  
Gustavo Chacin    Lehigh Valley  103.7            5             5             2          12

Clay Buchholz was one of the few pitchers to show up on two of the top-10 lists, but because he was considerably better last year with the bases empty, his total was only 11.

Will any of these guys pan out as successful big league relievers? Your guess is as good as mine. But except for a couple of guys here with some prospect sheen, there’s a good chance we’ll get to find out—or, at the very least, we can expect they won’t get many more chances in the big leagues as starters.

Next steps

We could easily extend this (or a similar) study to the lower minors. Perhaps looking at some of these components for pitchers in High-A and Double-A would be a good way of singling out players worth a pick in the Rule 5 draft.

In fact, this approach may be more valuable the lower the level. By Triple-A, many pitchers who will be relievers are already relievers. At the college level, for instance, the vast majority of pitchers with professional futures are still starters. Perhaps some starters who could excel in relief are currently overlooked.

Most useful would be to do this study for some of the earlier years for which I have splits. As much fun as it is to intuitively pick the characteristics that make for good relievers, it would be much more valuable to know whether a list like this one from 2006 had any predictive value of relief success in the following seasons. We’ll examine that in a future article.

References & Resources
Thanks to Kent Bonham for the idea for this article.