Improving pitcher projections
What if CHONE could scout pitchers?
What if the niftiest projection system out there could watch the pitchers throw, and make determinations how they would progress based on that?
Fangraphs’ pitch data gives us the tools to figure this out.
We’re starting from a more difficult place than projecting hitters. Projecting pitchers is hard. For one thing, they easily break. You look at Rich Harden funny, and his elbow pops. (Try it!) For another, they’re more erratic than hitters. So, we look at strikeouts and walks and BABIP and ground-ball ratio and ERA and possibly pet names. But so far, no pitch data. You can bring CHONE to a game and all it will do is complain about the TRS-80 you ported it to. It won’t watch.
But what if it could integrate scouting? What if it could add some numerical input based on scouty knowledge? Could we improve CHONE? Could we get substantially more accurate pitcher projections?
The answers, for those of you tired of the foreshadowing portion of the article and waiting impatiently for the content portion, are “We can,” “We can,” and “Yeppers.”
Here’s one scouty thing for pitchers: It is better to throw harder than softer. Announcers and some pundits like to say that it’s all about mixing pitches, changing speeds, hitting spots—and all those things are nice, and if you can’t do any of them, you’re Kyle Farnsworth—but it’s still better to throw harder. If over half your pitchers are fastballs and you throw 82, you’re playing against Sean Smith, not Albert Pujols, and I’m not the first to observe this.
My hypothesis was that pitchers who throw the ball real fast will do better than their projections. Pitchers who don’t throw the ball real fast will do worse.
So, if we had two pitchers:
Pitcher A, 5’11” 185 lbs, 165 IP 3.94 ERA 118K 74BB 1.71 GB/FB .299 BABIP
Average fastball speed 94.7 mph.
Pitcher B, 5’11” 185 lbs, 165 IP 3.94 ERA 118K 74BB 1.71 GB/FB .299 BABIP
Average fastball speed 85.2 mph.
Scouts would like Pitcher A a lot more.
Should CHONE look at the speed of the pitches? Does the speed of the fastball have a prediction value outside of the component stats?
I looked at the top 20 and bottom 20 pitchers in fastball speed for 2006. To qualify, a pitcher had to 1) Pitch at least 100 innings and 2) Use at least 50 percent fastballs (Tim Wakefield is not relevant to this discussion).
I then looked at their 2007 CHONE and their 2007 results. I did the same analysis for the following two years.
What I hoped to see was that the hard-throwing cohort would outperform CHONE, and the softer throwers wouldn’t. If this were true, this would be a significant breakthrough—the use of pitch data, breaks, and other numerically-definable scouting-type information could be used to generate some really excellent projections; improving on CHONE appears to be improving on the best projections available. If the hypotheses were correct, this would be the beginning of a new way to project pitchers—a better way.
At this point, I must caution you, gentle reader. Unless you are some sort of Rotisserie or Scoresheet player, terrible pitching can hurt your eyes. Even reading about it may cause pain. The experienced fantasy leaguer knows that for every misery-inducing performance of one’s own team, there are many fabulously entertaining performances by one’s competitors. This year, those performances are called, “Rafaels,” after the Indians’ Mr. Perez.
Let us view those who try to get by with less velocity—but keep throwing fastballs. These lists start at the very slowest, and require that the pitcher throw at least 50 percent fastballs. The last column shows which was better, performance or CHONE:
2006 Slow-throwers 2007 CHONE 2007 Performance
IP ERA IP ERA CHONE or Performance?
Greg Maddux 197 3.93 198 4.14 CHONE
Livan Hernandez 211 4.79 204 4.93 CHONE
Mark Redman 180 5.06 41 7.62 CHONE
Kenny Rogers 190 4.40 63 4.43 CHONE
Aaron Sele 130 4.58 53 5.37 CHONE
Tom Glavine 192 4.18 200 4.45 CHONE
Kirk Saarloos 121 4.98 42 7.17 CHONE
Barry Zito 206 3.62 196 4.53 CHONE
Mark O’Connor 117 4.19 7.17 ERA in AA CHONE
Jeff Francis 192 4.42 215 4.22 Performance
Paul Byrd 179 4.47 192 4.59 Even
Casey Fossum 147 4.88 76 7.70 CHONE
Chris Capuano 195 3.92 150 5.10 CHONE
Woody Williams 161 4.51 188 5.27 CHONE
Josh Fogg 172 5.17 165 4.94 Performance
Jason Jennings 180 4.05 99 6.45 CHONE
Steve Trachsel 140 5.15 158 4.90 Performance
Mark Hendrickson 172 4.46 122 5.21 CHONE
Pedro Martinez 63 3.11 28 2.57 Even
Zach Duke 195 3.94 107 5.53 CHONE
Three very marginal wins for performance. 3-15-2 is a pretty impressive set for the theory that low-velocity pitchers are bad bets, even compared to a good projection system like CHONE.
Will it get better if we look at 2008? Darken your sunglasses; the meltdown’s brutal:
2007 Slow-throwers 2008 CHONE 2008 Performance
IP ERA IP ERA CHONE or Performance?
Mike Maroth 120 5.48 Nada CHONE
Livan Hernandez 198 5.19 180 6.05 CHONE
Tom Glavine 194 4.78 63 5.54 CHONE
Lenny DiNardo 107 4.88 23 7.43 CHONE
Mike Bacsik 137 4.93 None CHONE
Barry Zito 202 4.19 180 5.15 CHONE
Greg Maddux 205 3.91 194 4.22 CHONE
Justin Germano 160 4.11 43 5.98 CHONE
David Wells 144 5.13 Called in Fat CHONE
Matt Chico 166 4.93 48 6.19 CHONE
Paul Byrd 194 4.55 180 4.6 Even
Woody Williams 178 4.80 None CHONE
Orlando Hernandez 145 4.10 Nope CHONE
Jeff Francis 207 4.39 143 5.01 CHONE
Mark Hendrickson 143 4.28 133 5.45 CHONE
Noah Lowry 164 4.50 Broke CHONE
Chris Capuano 177 4.32 None CHONE
Andy Sonnanstine 184 4.50 193 4.38 Performance
Randy Wolf 98 4.04 190 4.3 Performance
Chuck James 155 4.41 29 9.1 CHONE
Even grimmer. CHONE collectively had some hope for these guys, but they were, as a class, even more thoroughly awful than expected; it’s like watching American Dad. If you rely on a fastball that isn’t fast, you’re going to get killed.
But maybe CHONE just has a sunny disposition when it comes to pitchers. (I think this is actually true; CHONE tends toward more optimism than some other methods, though it won’t be mistaken for the Bill James projections.)
Let’s go to the charts for the top-end velocity pitchers; the pitchers are listed from 1-20 amongst qualifiers in average speed – the last column again shows which was better, the projection or the performance:
2006 Best Fastball 2007 CHONE 2007 Performance
IP ERA IP ERA CHONE or Performance?
Felix Hernandez 177 3.34 190 3.92 CHONE
Justin Verlander 152 3.99 201 3.66 Performance
AJ Burnett 167 3.74 165 3.75 Even
Daniel Cabrera 172 4.19 204 5.55 CHONE
Josh Beckett 186 4.00 200 3.27 Performance
Scott Proctor 91 3.99 86 3.65 Performance
Brad Penny 179 3.77 208 3.03 Performance
Seth McClung 99 4.98 12 3.75 Even
CC Sabathia 195 3.58 241 3.21 Performance
Matt Cain 184 3.50 200 3.65 Even
Jeremy Bonderman 200 3.63 174 5.01 CHONE
Kelvim Escobar 154 3.69 195 3.40 Performance
Chien-Ming Wang 182 4.09 199 3.70 Performance
Ervin Santana 169 3.94 150 5.76 CHONE
Johan Santana 212 2.51 219 3.33 CHONE
Jorge Sosa 119 4.20 112 4.47 CHONE
Ian Snell 169 3.94 208 3.76 Performance
Roy Oswalt 211 3.28 212 3.18 Even
John Smoltz 191 3.59 205 3.11 Performance
Ben Sheets 155 3.01 141 3.82 CHONE
OK, pretty close: 9-7 in favor of performance. But this is a decent set for the hypothesis.
Let’s see how our hard throwers did in 2008:
2007 Best Fastball 2008 CHONE 2008 Performance
IP ERA IP ERA CHONE or Performance?
Felix Hernandez 191 3.53 200 3.45 Even
AJ Burnett 165 4.04 221 4.07 Even
Justin Verlander 188 3.88 201 4.84 CHONE
Dustin McGowan 157 4.41 111 4.37 Even
Josh Beckett 197 3.79 174 4.03 CHONE
Daniel Cabrera 195 4.57 180 5.25 CHONE
Tim Lincecum 118 3.28 227 2.62 Performance
Edwin Jackson 153 4.76 183 4.42 Performance
Zach Greinke 124 4.35 202 3.47 Performance
Kelvim Escobar 175 3.86 None, thanks CHONE
Fausto Carmona 180 3.85 120 5.44 CHONE
Jeremy Guthrie 155 4.76 190 3.63 Performance
Brad Penny 195 4.11 94 6.27 CHONE
Matt Cain 196 3.54 217 3.76 CHONE
CC Sabathia 221 3.50 253 2.70 Performance
Ben Sheets 137 4.01 198 3.09 Performance
Roy Oswalt 213 3.59 208 3.54 Even
Chien-Ming Wang 189 4.33 95 4.07 CHONE
Matt Albers 153 5.53 49 3.49 Even
Jake Peavy 217 2.99 173 2.85 Even
Alas, a 6-8 run. When I did this for PECOTA, the numbers had a much better showing for the performance than the projections. CHONE’s general optimism for pitchers may well be because it’s right to be more optimistic than other systems.
Now, I did these charts after the fact, and I made the prediction that 2009 would be similarly situated. Was my method able to predict the future? Let’s go to the slowest of 2009 (with the prior caveats; retired pitchers weren’t considered for this year.)
2008 Worst Fastball 2009 CHONE 2009 Performance
IP ERA IP ERA CHONE or Performance?
Livan Hernandez 172 5.23 183 5.44 Even
Barry Zito 176 4.70 192 4.03 Performance
Jeff Francis 169 4.58 143 5.01 CHONE
Jeff Suppan 161 5.37 161 5.29 Even
Chris Young 114 3.95 76 5.21 CHONE
John Lannan 148 4.68 206 3.88 Performance
Jarrod Washburn 151 4.41 176 3.78 Performance
Greg Smith 144 4.88 Minors/bad/hurt CHONE
Pedro Martinez 76 4.45 44 3.63 Even
Scott Olsen 176 4.96 62 6.03 CHONE
Mark Hendrickson 60 4.20 105 4.37 Performance
Brian Burres 127 5.24 Mostly minors/bad CHONE
Brandon Webb 209 3.70 Ow CHONE
Chris Sampson 72 3.88 55 5.04 CHONE
Darrell Rasner 96 4.50 113 5.40 CHONE
Nate Robertson 171 4.53 49 5.44 CHONE
Andy Pettitte 166 4.55 194 4.16 Performance
Garrett Olson 157 4.24 80 5.60 CHONE
Tom Gorzelany 152 4.32 47 5.55 CHONE
David Bush 180 4.30 114 6.38 CHONE
Best year ever for our soft throwers, going 5-17-3, and with pretty big wins by Zito and Lannan. While not reflected in the charts above, the data suggests to me that lefties should receive about one mile per hour credit for being lefties; it’s actually true that left-handers can get by with less velocity.
Let’s see how our hard throwers did in 2009:
2008 Best Fastball 2009 CHONE 2009 Performance
IP ERA IP ERA CHONE or Performance?
Joba Chamberlain 102 3.26 157 4.75 CHONE
Ubaldo Jiminez 166 4.34 218 3.47 Performance
Dustin McGowan 118 3.97 Broke CHONE
Felix Herndandez 185 3.60 238 2.49 Performance
Ervin Santana 191 3.77 139 5.03 CHONE
Josh Beckett 177 3.61 212 3.86 Even
AJ Burnett 167 3.83 207 4.04 Even
Tim Lincecum 154 3.21 225 2.48 Performance
Clayton Kershaw 115 4.15 171 2.79 Performance
Edwin Jackson 145 4.78 214 3.62 Performance
CC Sabathia 211 3.41 230 3.37 Even
Justin Verlander 185 3.94 240 3.45 Performance
Edinson Volquez 166 3.58 49 4.35 CHONE
Johnny Cueto 145 4.19 171 4.41 Even
Zack Greinke 137 4.14 229 2.16 Performance
Jeremy Guthrie 156 4.33 200 5.04 CHONE
Seth McClung 68 3.84 62 4.94 CHONE
Matt Garza 168 3.96 203 3.95 Even
Jorge de la Rosa 108 4.67 185 4.38 Performance
Fausto Carmona 129 4.26 125 6.32 CHONE
We see a slight edge here for the performance, but it’s not big – further study is needed to see if there’s a reliable effect. The 2009 crew of hard throwers had little more injury-related falloff than prior years.
For now, we can confidently assert that slow-throwers who rely primarily on a fastball will do worse than their prior component stats and worse than a sophisticated projection model would otherwise predict.
We can unconfidently assert that fast-throwers will do slightly better than their prior component stats and better than a sophisticated projection model would otherwise predict.
This is just the first step. We can look at how the pitch selection, and pitch speeds affect likely outcomes. At some point, we can incorporate pitch break. We can improve CHONE. Or, someone can improve CHONE who has actual database skills. Trained monkeys have better database skills than I do (at least, the trained monkeys who generate Marcel at Tango Palace do.)
I’ve taken a few additional steps in modifying the formula: [Velocity=Good]. Because I’m a reprehensible human being, I’m not sharing. But, here’s the list of leaders and trailers for next year, under the alpha version of the projection modification system. This is a ranking of *differences from an established projection system*, not overall goodness. If you go draft Jeremy Guthrie ahead of Tim Lincecum, you’re missing the point.
Good:
1. Homer Bailey (You can imagine my concern about putting a Dusty pitcher here.)
2. Ubaldo Jiminez
3. Edwin Jackson
4. Justin Masterson
5. Jeremy Guthrie
6. Justin Verlander
7. Brad Penny
8. AJ Burnett
9. Josh Johnson
10. Felix Hernandez
Bad:
1. John Lannan
2. Jered Weaver
3. Aaron Laffey
4. Derek Lowe
5. Dallas Braden
6. Brian Moehler
7. Jeremy Sowers
8. Zach Duke
9. Joel Pineiro
10. Ted Lilly
I expect the Good guys to have ERA’s about 5 percent less than CHONE projects, but I could be wrong. I expect the Bad guys to pitch many fewer innings than CHONE expects, and be far less effective, and I’m not wrong on that.
At the end of the season, we’ll revisit if Dave Studeman drops the restraining order and lets me stuff more crumpled charts on yellow lined legal paper on his doorstep.
What do you think? Have I missed something important? Or am I right?
Place your bets in the comments.
For 2009, the hard throwers had a simple average of 3.94 projected, and 3.94 actual. Weighted by innings, they beat the projections 3.75 to 3.91.
For the slow throwers they were worse 4.96 to 4.52 on simple average, 4.79 to 4.58 weighted average.
But I should go a step further, weight both projected and actual era by actual innings pitched. Give me a few minutes.
Wow! Comments from MGL. I am honored.
I know that the keeping-score method isn’t any good in close cases. The bad pitchers aren’t close cases. It’s more visually impressive to do it this way.
I used a method for which was “better” based partly on value; substantially more innings of similar quality is better. Pedro had far fewer innings while Hendrickson had substantially more; that’s why I did this. (In both cases, they were on the thin edge of the margins using my method, and I preferred to count them against my theory.)
So, in short:
1. You’re right, and I knew that. But I think the long list of CHONE’s is compelling nonetheless.
2. They’re not “mistakes,” at least not in the, “You can’t add,” version. The method I used may well be suboptimal, and it may have been wrong to slice the edge on one side or the other. But I did it deliberately.
3. MGL! Woo! I could not be happier that you like it!
—JRM
Excellent work, John! This is obviously worthy of further investigation.
I took both projected ERA and actual ERA, and weighted both by actual 2009 innings pitched. Doing that the hard throwers beat their projections 3.75 vs 3.95. And the soft-tossers post a 4.79 ERA, compared to 4.61 projected.
Great work by John, and I’ll take a little credit for suggesting he contact Studes and publish on THT.
The next question is how/why do they beat the projections? Do they strike more out? Allow fewer homeruns?
I did something similar to this two years ago and found no effect:
http://lanaheimangelfan.blogspot.com/2008/05/fastball-velocity-and-pitcher.html#links
The differences between what I did and John did are twofold:
1. I was looking at inexperienced pitchers only, trying to see if it adds more information to MLEs.
2. I divided all pitchers into 3 groups, fast/medium/slow. John is focusing on outliers.
Just an excellent study. John admits he’s not a database guy. And this proves that not having a million pitches in a database is not necessary if you have a good idea plus hard work.
Great job John.
I edited out about three hundred words at the end; things that might be of interest to the onlookers, or not:
A. The first version of this actually used PECOTA, back when PECOTA hadn’t been ground into a fine powder. I liked that study better, because PECOTA’s more pessimistic toward pitchers generally, and I ended up with robust data on both sides (I don’t care if it was more accurate; I want it to match my hypotheses, dammit!)
On a private board I’m on, someone suggested that maybe this shows a problem with CHONE rather than an effect by slow-throwers. Given my PECOTA work, I can say with confidence that the effect on the downside is simply not an artifact of some CHONE blind spot; it’s an artifact of, “Slow throwers are bad risks compared to their prior stats.” PECOTA’s greater pessimism doesn’t make those charts above much kinder. CAIRO doesn’t have the long dataset, but… same thing.
B. The primary reason for not including other findings here is that the methodology’s a little painful to watch and I’m unconfident in my other conclusions. There are actually yellow legal pads with scribble on them.
Unconfident conclusions, aside from the lefty thing: At the top end, guys with very high velocity but unimpressive peripherals are very good bets to improve; it’s the 2009 Edwin Jackson or 2010 Homer Bailey you want to bet on. Velocity has a lot of confounding factors which make the middle very hard to analyze, but I am guessing the effect runs throughout the velocity spectrum *for fastball pitchers* once you factor out the confounding problems. But I could be wrong about this; in any event, the only use this has now is for the outliers.
C. My discussion with Sean – which he relays accurately above – was very helpful and very unnerving. The idea that Sean Smith couldn’t find a correlation scared the hell out me; I understand Bayes’ Theorem quite well, thanks, and I use it in interpersonal disagreement. The prior of [Sean is right vs. me on big numbery things] is way high. Eek. Big thanks to Sean. (The last edit removed my visit to Sean in the dungeon of Tango Palace, the screeching of monkeys heard above.)
D. The review and interest by the all-star team here is really nifty.
—JRM
Interesting work. One concern: this could be telling us only that CHONE over-regresses very good and very bad pitchers (at least those healthy enough to qualify for the study), as opposed to velocity. If you simply took the 20 best and worst projections for starting pitchers and compared them to actual performance, would we see the same divergence? (Great pitchers outperforming CHONE, bad pitchers underperforming).
A suggestion: first adjust the projections to equal the actual outcomes in the aggregate. That way the comparison will be much clear. (And we don’t care, for these purposes, about the accuracy of CHONE’s league-wide ERA forecast).
For the 2009 list of pitchers, as a complete group my ERA projections were almost dead on once you weight both the projections and the performance by actual innings pitched.
I was .20 too high on the fastballers, almost perfectly matched by being 0.18 too low on the soft-tossers.
“The idea that Sean Smith couldn’t find a correlation scared the hell out me”
Shouldn’t, as my study was not a complete one and looked at a different subset of pitchers. Sometimes if you want to find an effect you’ve got to look under every rock. I just looked under 1.
It would be easy to check the best 20 starters in projected ERA and see how they match with reality, and see if I’m regressing enough.
Identifying the 20 worst is not so easy, since the projection list includes guys who were hammered for 50 innings in AA and have a projected ERA of 7.50 or something, most of whom won’t pitch in the majors.
And if you look for the guys with the worst projections who were actually allowed to pitch, well, that would be some serious selection bias. My guess is you could find a group with projected ERAs over 6.00 who in reality pitched at a 4.50 rate. That’s not an overly optimistic projection system, just pure sampling bias.
I combined 2007-2009 best fastballs lists.
Weighted by actual innings, projection is 3.85, actual ERA is 3.79
For the slow throwers, 3 years data, projection of 4.47, actual of 4.94.
Im not sure how effective this analysis will be without distinguishing between 2 and 4 seam fastballs. The pitches have significant differences in profiles despite their similar speeds
“One concern: this could be telling us only that CHONE over-regresses very good and very bad pitchers”
This is not the case, at least for the good pitchers. I took the top 20 projected ERAs for starters last year, and given actual 2009 innings they should have had a group ERA of 3.48. Their actual ERA? 3.61. If anything, not enough regression.
Sean: It looked to me like in 2007 and 2008 the soft-tossers underperformed, while the hard throwers broke even—making me think the aggregate ERA prediction might have been off in those years. If not, my suggestion should of course be disregarded.
You raise a good point about identifying the worst pitchers. Maybe the thing to do is to look only at established pitchers (at least 500 career IP, at least 150 IP prior season), and then try to create groups with projected ERAs similar to what John has here. Something like 1) projected ERA below 4.10; and 2) projected ERA of 4.40 to 4.90.
* *
Separate issue: I think it would also be useful to distinguish between soft and hard throwers in general, vs. pitchers who have gained or lost velocity in the previous year. Some of what we’re seeing with the soft throwers is a decline in velocity in the prior year, that means we should weight prior years less than CHONE does (Zito in 2006 and 2007, Chris young in 2008, Pedro in 2006). It would be interesting to look at this kind of player separately from those who are ALWAYS slow (e.g. Webb). And there may be similar stories on the hard throwers list, who improved their velocity in most recent season (recovering from prior injury).
Sean: OK, sounds like under-regressing the good pitchers isn’t a problem. But it also sounds like the hard-throwers don’t actually overperform (3.79 vs 3.85 according to your data), so we don’t really need to explain them anyway.
So question is why soft tossers underperform, and whether it’s really velocity or a recent DROP in velocity (or both) that predicts underperformance.
Good point. I’d hate to have been underprojecting Greg Maddux or Jamie Moyer all these years just because they don’t throw hard. The change might be what to focus on. I wonder how this looks for relievers too, that’s where you’ll find the best average fastballs. And again, I’d like to know how these pitchers beat their projections at the component level – strikeout rate is the obvious culprit but who knows.
So many questions I need to hire a staff of sabermetricians! Too bad I have no money.
(I also posted this at the book blog but thougth I should put it here.)
Good stuff, JRM!
Not to give it away but Steamer is going to try to use velocity in its 2010 pitcher projections.
For the pool of 873 pitchers we looked at the correlation between Marcel’s K/AB and actual K/AB was 0.716. But the correlation between Marcel “adjusted for fastball velocity” K/AB and actual K/AB was 0.741.
In other words, we go from explaining 51% of the variance to 55% of the variance.
Disclaimer: This was done by adjusting a Marcel projection for year Y with velocity from year Y which is, obviously, unfair since we only know velocity in year y-1. b/c velocity is so consistent, the major problem here might be that it’s picking up on the fact that guys who went from starting to relief outperformed their projected K/AB and had above average fastball velocity.
Anyway, I think you’ve hit on something real and substantial here. Good stuff.
I see two potential steps to understanding what it is that John has uncovered.
1. Age-control the sample. Given that CHONE or PECOTA start with previous years stats, John’s findings seem to indicate a perennial decline by soft-tossers and that begs the question of whether the soft-tossing sample is stacked with older declining pitchers.
2. Isolate the projected peripherals from ERA to determine whether the increased ERA is coming from year-to-year projections or is hiding within the static peripherals-to-ERA formulas.
Great stuff!
One (important) thing: Please, please, pretty please, with sugar on it, stop with the “keep score method” to analyze these things. With very few exceptions, that is a BAD (really bad) way to do the analysis.
The proper way is to look at the difference between ERA and performance for ALL PLAYERS IN EACH SAMPLE COMBINED, or to do a regression!
If you had 10 slow fastball pitchers, and for 6 of them their performance was better than their projection by an average of .25 runs and for 4 of them, their performance was worse that their projection by an average of 2 runs, guess what????
That being said, without weighting by inning, the slow tossers in 08, performed an average of .47 runs worse than their projections and the hard tossers performed exactly at their projections.
I did not include those who did not pitch in the majors in 2009. I am not sure why you put them in the “projection better” category. I don’t think that is fair to your study or analysis.
Plus, I think you have some mistakes. Pedro is classified as “even” in 2009 despite a .82 run difference and Mark Hendrickson is classified as favoring “performance” despite pitching .17 worse than his projection. I did not scrutinize all the other players.
Darrell Rasner pitched in Japan in 2009, where he had a bad season… ERA over 6.00 iirc.
Patrick:
Thanks for the catch. Patrick’s right; the chart’s wrong re: Rasner. Oops. At least he had the courtesy to be bad, supporting the hypothesis.
—JRM
My guess is simply that the means for the two groups are different and thus Chone’s regressions should be toward different means but are not. I, for one, do not use velocity as a variable in my projection algorithm, even though I should. I’m pretty sure that John would have come up with the same results if he had used my projections.
It would be the same thing if you did not regress hitter stats toward those of players with similar heights and weights, which many projection systems do not do. If you don’t, you will find that taller players over-perform their projections.
Same thing with age for batters. If you don’t regress toward the means of similar-aged players, you will find that players in their primes (26-29 or whatever) will outperform their projections as compared to younger and older players.
What about comparing projected ERA with luck and defense-neutralized pitching ERA-equivalent stats like xFIP or tERA? I mean, someone may have outperformed their projection because they were lucky, or because a ballpark played differently one year from the next (it happens). If a player outperformed their projection in ERA, but had an xFIP closer to their projection, that might be a case where we could call that one even…
First off, John. Interesting stuff.
I am a firm believer that projected pitching stats can be improved based on factoring in pitching stats as found on sites like Fangraphs.com.
But while I think that pitching velocity has a positive correlation with general performance, I’m not seeing why this would impact projection accuracy assuming the projections are based on fundamentals (K rate, xFIP) vs. performance.
Washburn’s 2009 performance isn’t a vindication that CHONE was a better predictor of performance. His ERA was an anomaly based on a ridiculously low BABIP (.257) even taking Seattle’s great D into account (F-Her came in at .289).
I would think the same would hold true for pitch movement as well – that it would end up being reflected in the fundamentals that would go into the projections.
I think there is one area – one that I’ve explored a bit on my site – that may not be seen in a pitcher’s fundamentals. It’s whether the pitcher’s luck-independent performance is SUSTAINABLE. Did a starting pitcher overthrow breaking pitches in a year to amp performance (think Steve Stone – 1980)? Is a starter or reliever more likely to be less than 100% if he pitched too much the year before (think every reliever under Joe Torre since 2000 aside from Mo)? I’m okay if this doesn’t make it into the projections, though, as I play a lot of fantasy baseball and prefer to have some small advantage over my competitors
Just my two cents…
One more thought. Since low velocity throwers tend to have higher ERAs, is it possible that what we’re seeing is that it is harder to project ERA for borderline pitchers vs. good pitchers. Perhaps it’s a lot more likely that a projected 5.00 ERA pitcher throws for a 4.00 ERA/6.00 ERA than a 3.00 ERA pitches a 2.00 ERA/4.00 ERA….