kwERA: The Starting Point for Pitcher Evaluations

Measuring a pitcher’s production and talent level is a problem analysts have been trying to solve for awhile. The main issue is how much credit for runs scored should be given to a pitcher, and how much should be given to other elements like defense or random variation/luck.

To help determine these values, ERA estimators have been created over the years to help give a better picture of the pitcher; they include FIP, xFIP and SIERA. All these start with strikeout and walk rates and then begin to add batted ball data. The basic strikeout and walk-based estimate has not been easily accessible until today at FanGraphs. Its acronym is kwERA, and for now it is available only as a stat on the custom pitcher leaderboards. But that is still pretty cool, and now that we have access to it, I want to look into why kwERA should be the starting point for any pitching evaluation.

While strikeouts and walks aren’t the only factors in estimating a pitcher’s talent level/ERA, they are the core attributes a pitcher has control over. A pitcher also has control over the direction a ball is hit (ground or air) and how hard the contact is (home run or routine pop-out). Other ERA estimators use these factors to for their calculations. Here is a look at the limitations for the estimators at FanGraphs.

One important thing to note is that when using the ERA estimators, it is important to note why they are being used. Some like FIP and kwERA show exactly what a pitcher did. They give the exact number of strikeouts, walks and home runs allowed by the pitcher. These are the reasons they are used in WAR formulas. SIERA and xFIP use batted ball data (and several other variables with SIERA) to estimate what would normally happen, not what did. I am looking only at the values as estimators for now.

FIP: FIP, which uses strikeouts, walks, and home runs, is the original ERA estimator. The problem with FIP is it takes a while for a pitcher’s home run rate to stabilize — normally, about 1,300 batters faced. Most starters face at most 900 hitters in a season. A person looking at single season FIP would be better off using the league average home run rate to predict the pitcher’s ERA the following season.

xFIP: xFIP was designed to help get to a stabilization point faster by using a pitcher’s flyball rate, which stabilizes in 70 batted balls — or about one-sixth of a season (600 batter balls is a normal season for a starter). The league average home run per flyball rate is assumed and this value is multiplied by a pitcher’s flyball rate. The problem is that pitchers with higher flyball rates have lower HR/FB rates and their HR/9 rates don’t increase linearly. Here are two graphs that show the HR/FB and HR/9 rates for pitchers with different flyball rates (minimum 40 innings pitched in a season). I grouped the pitchers in groups of 20 to help with the visualization.

To further prove the point, here a comparison of the season’s flyball percentage to ERA-xFIP and the next season’s ERA-xFIP (minimum 40 innings pitched in each season, with values again grouped by 20 for ease of viewing).

As can be seen from this graphs, it isn’t just the high flyball pitchers who are getting undervalued, it is the low flyball pitchers also. xFIP is not taking into account some of the advantages on the extreme ends of the batted ball spectrum. I will get back to xFIP later.

(Note: I will not use flyball rates again in this article. I find it is easier to learn what is considered to be a good and bad groundball rate and use those values. RIP flyball percentage.)

Average of xFIP and FIP: Each of the values has its limitations. Averaging the pair is kind of like comparing apples and oranges, as OPS does with on-base percentage and slugging average. The average works out to a fairly decent value, and better than some other estimators examined.

SIERA: SIERA should be considered just a projection/true talent measure. My main issue with SIERA by itself is knowing why it has a certain value such as 3.27. The reason is because these are all inputs to SIERA:

(where netGB=(GB-FB), and where +/-(netGB/PA)^2 is + when GB>FB and – when GB<FB.)

While SIERA addresses some of the issues I noted with xFIP above, we don’t have any context of the pitcher’s talent. It is just about impossible to find the exact contributions from strikeouts and walks with only xFIP and FIP available.

So for all three of the ERA estimators, the missing aspect is a reference point for the batted ball data (and the additional calculations in SIERA). kwERA provides that reference point. With kwERA as a starting point, I decided to create my own metrics, one of which may actually be useable. I looked at using wISO — which is ISO calculated with doubles, triples and homers weighted according to wOBA components — and GB% to adjust kwERA to get a better ERA estimators. I created an estimator with just each of the variables add and then one with both. (Note: Some of these values may need to be adjusted a small bit. I ran the analysis without the final kwERA numbers on FanGraphs.)

What I found was already known and stated in this article in that wISO, which is based on batted ball data, is not as predictive. I was hoping to tease a little more predictive value by looking at pitchers’ ability to limit hard contact. Some years those hard hit balls with become doubles and in others they will be home runs. In the end, it wasn’t a huge improvement.

wISO-adjusted ERA was a better predictor of next-season ERA and was closer to in-season ERA. If someone wanted to base a metric on what has actually happened (ERA, FIP or kwERA) using wISO, I would not have a problem with it, but it is not much of an improvement over FIP. Additionally, FIP doesn’t involve fielders, so it is a better measure of the pitcher for the factors involved.

The big improvement came when I compared the difference between ERA-kwERA and groundball percentage. Here is a graph of the two from 2002-2015, with each dot being 20 values averaged together for ease of view:

The key point from the graph is the peak at 34 percent, which is the least desirable rate for the pitcher. With ground ball rates less than 34 percent, the pitcher begins to see an advantage of a low groundball rate. Baking a groundball adjustment into kwERA creates one of the best ERA estimators (as seen later). Here is a basic formula, which I’m sure can be adjusted over time:

GBkwERA = kwERA*(-3.518 * GB^2+2.344*GB+.629)

We’ll see if GBkwERA gets housed at either my site, Baseball Heat Maps, or FanGraphs. Until then, xFIP can be a proxy for it — at least, until a pitcher gets down to 42 percent ground balls and the normal bad effects of a lowering groundball rate diminish until they turn positive. A person could use xFIP for 42 percent GB% and above, and kwERA for values under 42 percent.

Finally, I got the idea to combine both the ISO and GB% multipliers, but this led to horrible results. It performed worse than any of the other metrics as an ERA estimator outside of plain ole’ ERA.

Final comparison

The number of ERA estimators can be a little overwhelming, so I created one graph to compare the r-squared values from season 1 to the next season’s ERA for all of the ERA estimators I’ve examind in this piece, which again, is but a fraction of the ERA estimators out there these days.

As expected, SIERA reigns supreme, but GBkwERA comes in a close second. A surprising third place is the average of FIP and xFIP, which narrowly beats out the baseline value of kwERA. FIP and xFIP both have lower correlations on their own, but when combined, their weaknesses average out.

I am elated to have kwERA finally available at FanGraphs, as it gives a nice baseline for a pitcher’s talent level. With kwERA known, the effect of walks and strikeouts on a pitcher’s ERA is known and how much of the value is their batted ball data. I have taken the first stab at creating a simple yet effective estimator using kwERA and GB%, which I have titled GBkwERA. It doesn’t have the final 2015 values yet, and they weren’t corrected for park or league — so more work needs to be done there — but what I have come up with is a promising start. Using a simple best-fit line creates an ERA estimator with almost the same predictive accuracy as the more complex SIERA. In the meantime, enjoy the kwERA values and the additional perspective they bring to pitchers.

References & Resources

• Special thanks to Sean Dolinar for graphical assistance.
• Special thanks to Peter Melgren for statistical assistance.
• Mike Fast, The Hardball Times, “Leaders in kwERA”
• Dave Studeman, The Hardball Times, “I’m Batty for Baseball Stats”