A different way of measuring clutch

Normally, when a 20-something-year-old types “fa” into their browser and presses Enter, he is taken to a website owned by a certain multi-billion dollar social media software company.

When I, a 20-something-year-old, type “fa” into my browser, I am not taken to said social media website. I am taken to FanGraphs.com. This happens because I enjoy perusing baseball statistics more than I enjoy interacting with my friends (without actually interacting with my friends).

While perusing said baseball statistics, I commonly find myself on the leaderboards. On these leaderboards, I can view, for example, the players with the most home runs, or the players with the most Wins Above Replacement, or the players with the highest RE24.

You heard me. RE24. If you follow me on Twitter, or if you’ve read pretty much anything I’ve written in the past few months, you’ll know that I love RE24. I love RE24 not because it is a perfect statistic, but because it is simple, because it is a great gateway into sabermetrics for our RBI-inclined friends, and because it considers context.

I love context, and I don’t think we talk about it or think about it or research it enough in sabermetrics. Context is not terribly helpful for predicting the future, or for winning your fantasy baseball league, or for helping teams find market inefficiencies, or for whatever else would get one hired by a baseball organization. But it’s fun. It’s interesting. It tells a story. It leads to new ways of viewing and evaluating players and teams.

We can measure context in a number of ways. RE24 is one of them. Win Probability Added is another. WPA is probably the more popular of the two, and I certainly enjoy using it in certain situations; however, I believe that RE24 is a better way to consider context than WPA, which I defend here.

The basic premise is that WPA looks only at what has happened in the game so far, but can’t consider the context of the entire game. For this reason, I prefer to look only at run expectancy, rather than win probability. Nevertheless, whether you agree with my reasoning or not, what will follow may still interest you as a new way to consider context and clutch.

So we have RE24 as a way of measuring context-dependent offensive contribution. That’s wonderful for a wide variety of uses, including creating your own WAR, but why should we stop there? Surely we can think of other ways to use this wonderful context-dependent metric, right?

Right! That other way is clutch. I know, I know—clutch isn’t a very popular word for us statheads. But bear with me. Again, we’re not trying to predict the future; we’re just trying to find other ways to evaluate the past.

You may be aware that there already exists a “Clutch” metric, which, crudely, measures how well a player performs in high leverage situations relative to how well he performs in all situations. It’s essentially the difference between a player’s actual WPA and what one would expect his WPA to be if he performed at the same level regardless of the situation.

Since we’ve already decided to use RE24 instead of WPA, let’s transfer that idea to RE24. Because RE24 measures runs “produced” relative to the average player, all we need to do is find some way to measure the number of runs a player produces above average, independent of the situation/context. And wouldn’t you know, the core offensive component of WAR, wRAA, does exactly that.

So, all we have to do now is subtract wRAA from RE24, right? Unfortunately, no. In my research for this article, I initially thought it was this easy, but I soon noticed a disconcerting trend in the numbers: Players on teams like the Rockies and Red Sox consistently had a higher wRAA than RE24, and players on teams like the Mets and Padres had a higher RE24 than wRAA. That’s right—there was a park bias.

Upon further investigation, I realized that RE24, contrary to my previous assumption, is park-adjusted—that is, it uses run expectancy values that are tailored to the park, rather than ones that are uniform across baseball. This threw quite a wrench into my plan, as I don’t have the statistical or programming chops to calculate non-park-adjusted RE24.

My first thought was to use FanGraphs’ Batting runs, or Bat for short, which is the park-adjusted version of wRAA. Unfortunately, however, Bat swings the difference too far in the other direction. Players who played in places like Coors had a significantly higher RE24 than Bat, whereas it was the other way around with wRAA. Additionally, I believe that Bat removes pitcher plate appearances from its calculation of average wOBA, thus making Bat slightly higher than RE24 on average.

To be honest, I don’t know why RE24-Bat has a park bias if RE24 uses park-adjusted run expectancies. If you know, I would love to hear the explanation. But in order to present these numbers in a non-misleading way, my very hacky solution was to simply take the average difference in (RE24-wRAA)/PA for every team since 1974, and subtract that difference from every player-season (RE24-wRAA)/PA based on the team for which that player played. It’s not pretty, but it does the job, and the end result should be slightly closer to what we want.

Are you tired of reading words? Yes? Good, because I’m tired of writing them! Let’s get to the charts. For brevity’s sake, and because the actual formula is not simple, I will refer to my adjusted (RE24-wRAA)/600PA as SitHit (short for situational hitting). As explained above, the number, in essence, measures the difference between a player’s context-dependent run production and his context-independent run production—or, his situational hitting.

A Hardball Times Update
Goodbye for now.

Career leaders in total SitHit since 1974

Num Name SitHit
1 Tim Raines 119.08
2 Ichiro Suzuki 110.99
3 B.J. Surhoff 108.46
4 Bobby Abreu 100.62
5 Jose Cruz 100.20
6 Eddie Murray 94.73
7 Barry Bonds 92.72
8 Rickey Henderson 92.61
9 Carlos Beltran 82.67
10 George Brett 73.77
11 Mark Grace 71.80
12 Darin Erstad 71.57
13 Johnny Damon 69.64
14 Robin Yount 69.29
15 Terry Pendleton 68.46
16 Wally Joyner 68.27
17 Steve Garvey 67.60
18 Mickey Rivers 66.48
19 Will Clark 63.46
20 Marquis Grissom 62.83

If you didn’t already want Tim Raines to be in the Hall of Fame, this may convince you. Raines had 313 Batting Runs, which was already a great number to go along with fantastic speed, but his RE24 was 503, almost 200 runs higher! Add 20 wins to Raines’ already-impressive resume, and he is as sure-fire a Hall of Famer as they come.

Career leaders in average SitHit per 600 plate appearances since 1974

Num Name SitHit/600PA
1 B.J. Surhoff 9.44
2 Tim Raines 8.90
3 Darin Erstad 8.30
4 Ichiro Suzuki 7.63
5 Joe Morgan 7.21
6 Terry Pendleton 7.15
7 Carlos Beltran 6.83
8 Jose Cruz 6.47
9 Bobby Abreu 6.40
10 Wally Joyner 6.34
11 Barry Larkin 5.97
12 Will Clark 5.38
13 Willie McGee 5.23
14 Barry Bonds 5.22
15 Devon White 5.20
16 Steve Garvey 5.12
17 Mike Piazza 5.05
18 Marquis Grissom 4.98
19 Rickey Henderson 4.95
20 Mark Grace 4.90

B.J. Surhoff is an interesting name to see at the top of these lists, as he was actually a below average hitter in his career. However, his excellent situational hitting, along with great defense, may make him one of the more unappreciated players in recent memory.

Top qualified seasons by SitHit since 1974

Num Name PA Season Team SitHit
1 Tom Herr 696 1985 Cardinals 30.27
2 Buddy Bell 415 1981 Rangers 28.70
3 Brooks Robinson 539 1975 Orioles 28.66
4 Tim Raines 363 1981 Expos 25.77
5 Barry Bonds 634 1991 Pirates 24.22
6 Darin Erstad 543 2004 Angels 23.96
7 Eric Chavez 604 2001 Athletics 23.82
8 Clint Barmes 535 2006 Rockies 23.77
9 Rickey Henderson 647 1988 Yankees 23.55
10 Barry Bonds 675 1996 Giants 23.33
11 Joe Morgan 599 1976 Reds 23.29
12 Eric Young 513 1998 Dodgers 22.92
13 Moises Alou 619 1997 Marlins 22.57
14 Tony Gwynn 651 1997 Padres 22.49
15 Mickey Rivers 672 1975 Angels 22.39
16 Dustin Ackley 668 2012 Mariners 22.18
17 Jose Cruz 536 1986 Astros 22.08
18 Rick Wilkins 358 1994 Cubs 21.90
19 Lou Piniella 513 1978 Yankees 21.89
20 Garrett Jones 515 2012 Pirates 21.62

Tom Herr takes the crown for best season ever by situational hitting. While he was only 20 runs above average by wRAA, he was almost 60 runs above average using context-dependent RE24! That’s essentially the difference between Torii Hunter and Miguel Cabrera last season. Herr hit .356 with runners on base compared to .255 with the bases empty, not to mention a .396 with no outs.

And finally, just for fun, let’s look at the worst seasons by SitHit since 1974

Num Name PA Season Team SitHit
1 Robinson Cano 674 2009 Yankees -28.47
2 Scott Brosius 526 1997 Athletics -27.96
3 Mike Lieberthal 529 2004 Phillies -27.93
4 Lance Parrish 592 1980 Tigers -25.86
5 Larry Walker 524 1998 Rockies -23.95
6 Bill Madlock 509 1974 Cubs -23.16
7 Bill Russell 672 1978 Dodgers -23.06
8 Dwight Evans 563 1979 Red Sox -23.02
9 Benito Santiago 527 1988 Padres -22.42
10 Pedro Feliz 531 2004 Giants -22.26
11 Preston Wilson 513 2001 Marlins -22.19
12 Luis Rivas 521 2003 Twins -22.15
13 Dean Palmer 652 1996 Rangers -22.07
14 Dave Stapleton 581 1982 Red Sox -21.94
15 Ernie Young 528 1996 Athletics -21.88
16 Jody Reed 619 1989 Red Sox -21.84
17 Jose Hernandez 571 2003   -21.79
18 Rick Burleson 721 1977 Red Sox -21.71
19 Rafael Ramirez 342 1981 Braves -21.71

Yikes, that’s bad. In 2009, Cano had a wRAA of 25.4, but an RE24 of -7.4. In other words, there was over a three-win difference between his context-dependent and context-independent production. It turned out pretty well for the Yankees in the end, but some better situational hitting from Cano could have made their great season even greater.

There you have it. I realize that much of this article was simply my own thought process for this idea and research, but I hope you found it somewhat interesting. At the very least, SitHit, or RE24 or some other variant, is a new way to consider context, clutch, and contribution.

Matt is the founder of SaberSim, a daily sports projections and analytics company. Follow him on Twitter @MattR_Hunter and @SaberSim, or email him here and tell him all the things he should do to make the site better.
Newest Most Voted
Inline Feedbacks
View all comments
Peter Jensen
10 years ago

Matt – I have to laugh because in 1987 I gave a talk at the annual SABR conference where I presented the run value added (the name coined by Gary Skoog for the delta of run expectancy in an article in the 1987 Baseball Abstract) for 1986 as calculated from the then new first complete year of play by play data collected by the Bill James inspired Project Scoresheet.  One of the uses that I mentioned for run value added was to measure clutch.  The method was to measure the difference between RVA and Linear Weights.  Sound familiar?  If you need help calculating non Park Adjusted run value added contact me directly.

10 years ago

Matt, can you confirm if the batting runs you use includes SB, CS or not?  I’m pretty sure RE24 would include it.

10 years ago

Great job, Matt. I’m trying to think of similar research that has done the same thing. I recall discussing Herr’s amazing 1985 before.

10 years ago

I was thinking the exact same thing as Tango—my understanding is that RE24 includes SB and CS, while Batting Runs (at least as presented on FanGraphs) doesn’t. If that’s the case, then it certainly explains why your top ten is littered with prolific (and efficient) base-stealers. You’d have to either include SB and CS in Batting Runs or remove them from RE24 in order to get an apples-to-apples comparison.

10 years ago

Has anyone done any calculations to see if this is a skill (if there is a correlation from year to year, for example, not caused by some bias in the way the two stats, batruns and RE24 runs, are calculated) or it is just the way the cookie crumbles, so to speak?

Or, as Tango would say, “Not whether it is a skill, but how much of that skill exists in a given number of PA, compared to the noise?”

Marc Schneider
10 years ago

“Has anyone done any calculations to see if this is a skill (if there is a correlation from year to year, for example, not caused by some bias in the way the two stats, batruns and RE24 runs, are calculated) or it is just the way the cookie crumbles, so to speak?”

It seems to me as very much a non-statistician that if this was a skill, you would not see so many lesser players on the list.  If some of these guys were really so much better in the clutch than the rest of the time, shame on them because they should be doing better the rest of the time.  For example, Chipper Jones, to take one example, is not on any of these lists, yet surely he is a better hitter than, say, Darin Erstad.

10 years ago

I think WPA and RE24 are both interesting and useful stats, and I agree that using win expectancy can overvalue a given batter’s impact (giving more weight to a single that drives in a runner than the double that got him to second base, for example.)

But it seems to me that’s not entirely solved by RE24, especially with regards to measuring “clutch”. Say a hitter in a tie game in the bottom of the ninth hits a triple with no outs. The run expectancy for that hit will be extremely high. If that same hitter comes up with two outs, and arguably more pressure, and he hits that same triple—still second best (and most “clutch”) possible outcome in that situation—the run expectancy is diminished by the fact that the runners before him got out, and the runners after him can’t drive him in with an out, no?

10 years ago

Run expectancy by definition IGNORES inning and score.

Win expectancy by definition INCLUDES inning and score.

If you are going to talk about bottom of the 9th and close games, then you CANNOT talk about run expectancy.  You have to use win expectancy.

If you don’t like the idea that the win expectancy impact is going to be 2x or 5x of what you’d get from run expectancy, then you don’t care about inning and score.

Decide what you want first, and that’ll tell you whether you want runs or wins.  You’re not going to get BOTH.

Matt Hunter
10 years ago

Thanks for the comments everyone! Sorry I haven’t responded – was on vacation all of last week.

Yes, I used RE24 from FanGraphs, which I didn’t realize includes SB and CS, and no, the Batting runs I used did not include SB and CS. So the results are going to be skewed towards base stealers. Oops.

That also partially explains why an initial variant of SitHit had a year to year correlation of .26. I didn’t include it in the article because I thought it was due to park effects, but I’m guessing it’s because of both.

I may work on calculating an RE24 that isn’t park-adjusted and doesn’t include SB and CS, so we’ll see what sorts of results I get when I use that instead.

Thanks again for the feedback. Much appreciated.