Peephole Into the Postseason: Winners vs. Winners

Will the Oakland Athletics' s**t ever work in the postseason? (via djanimal)

Will the Oakland Athletics’ s**t ever work in the postseason? (via djanimal)

The Oakland A’s are, almost by acclamation, the sabermetrics community’s favorite baseball team. The reason why is a bit of egoism masquerading as objectivity, or maybe the other way around. The A’s, led by general manager Billy Beane, use their brains. They strain to find ways to buy wins cheaply, which is almost the only way they can afford to buy them. This led them to embrace, and in large measure validate, sabermetric findings that thinkers about baseball had been shouting into the void for years.

They aren’t the only baseball team to do this now. Most teams today have analytics departments, to make them look smart if not always to use what they produce. In innovation, Oakland has probably been leapfrogged by the Tampa Bay Rays, whose owners are building on the A’s foundation. But if they are still second-best in the eyes of sabermetricians, the reason why can be summed up in one word: Moneyball.

Best-selling books help. Brad Pitt movies help more. Even when the Rays got their own book, Jonah Keri’s The Extra 2%, their smarts worked against them. Michael Lewis revealed all in Moneyball, which helped other teams ape Beane’s methods and thus make them less effective for him. The brain trust of the Rays let us see the method and overall philosophy, but jealously guarded the details to keep their edge. Shrewd, but not calculated to win the hearts and minds of analysts who drink in those details.

So Beane and the A’s remain the first true love of us baseball nerds. Over the last 14 full seasons, Oakland has made seven playoff appearances, far more than we’d expect from a team as thoroughly outspent as the A’s have been. It’s a heart-warming, brain-satisfying success story—except for one little detail.

Beane said it best in a certain noteworthy book: “My [methodology] doesn’t work in the playoffs.” No, he didn’t say “methodology.” Yes, I’m being a schoolmarmish prude for not spelling out the much shorter word he did use. If you’re offended by my not being offensive, consider the irony in that. Savor it. We’re so overloaded with irony these days, it’s been devalued. We don’t appreciate it properly.

Back on my long-abandoned point, seven times Beane’s A’s have made the playoffs, and six times they have washed out in their first round. The lone exception was 2006, when they stunned the world by sweeping the Minnesota Twins, then restored balance to the world by getting swept out of the ALCS by the Detroit Tigers. As much success as Beane has garnered between April and September, he’s 1-7 in playoff series, and has yet to receive the validation of an American League pennant, never mind a World Series victory.

People have asked before why this should be. Back in 2006, in Baseball Prospectus’ Baseball Between the Numbers, Nate Silver and Dayn Perry studied correlations between playoff success and a host of team statistics, from batting average and isolated power to playoff experience and team record in September and onward. The three factors they found tracked best with playoff success were pitcher strikeout rate, fielding runs, and closer win expectation. (They used measures for fielding and closer performance, FRAA and WXRL, that are obsolete a whole eight years later, but their findings stand.)

“It would be misleading,” they warned, “to suggest that this is some kind of secret sauce.” They rethought this, and for a while you could look up teams’ “Secret Sauce” rankings at the BP website. Then they re-rethought this, possibly after added data weakened the correlations, and took the sauce off the menu. Back to Square One we went.

I am not going to plow through stacks of peripheral stats to find the hidden cause of Oakland’s woes. Instead, I am going to plow through stacks of one particular stat, not all that peripheral, to see if it predicts postseason success better than we manage it right now. My hunt is based on an obvious yet over-lookable principle: baseball playoffs, with one teensy-weensy, historically bizarre and hopefully never to be repeated exception, involve winning teams.

(That exception was the 1981 Kansas City Royals. In that strike-riven season, they won the second-half semi-crown of the American League West despite an overall 50-53 mark. The 2005 Padres slipped in as NL West champions at 82-80, the closest any other team has gotten to reaching the playoffs without a winning record.)

The skill set that lets you fatten up against also-rans may not be the same one that lets you hang in there with the first rank of the league. A playoff team that’s gotten there mainly by hammering the hapless could be in trouble because there are no punching bags left on the schedule. This might explain why some teams, such as Oakland, fizzle out.

There’s some precedent for this kind of analysis. Not too long ago, Vince Gennaro, president of the Society for American Baseball Research, broke down hitters by how they perform against strong pitchers and weak pitchers. The difference he found in league-wide performance against those two “buckets” of pitchers was about 180 points of OPS, but there were some batters with much wider splits, and others with much narrower ones.

In an appearance on MLB Network’s Clubhouse Confidential, he gave examples from both groups: Josh Hamilton and Derek Jeter. Hamilton feasts on poorer pitchers while struggling against the aces; Jeter doesn’t run up his numbers against the staff filler, but holds up well when facing the studs. Jeter certainly doesn’t reverse the splits, hitting better against good pitchers, but he narrows them a good deal, while Hamilton’s split is quite wide.

How this connects to postseason affairs is that good teams, the ones making the playoffs, tend to have good pitchers (that being part of what makes them good). Furthermore, with pacing concerns for a long season no longer paramount, those good pitchers will get an increased proportion of innings pitched. The dropping of a team’s fifth starter from the playoff rotation is a familiar example of this. Gennaro estimates that pitchers who throw the top 40 percent of regular-season innings pitch two-thirds of postseason innings.

A Hardball Times Update
Goodbye for now.

With the playoffs featuring an over-sampling of good pitchers, batters with a narrow split between top and bottom hurlers would naturally get an edge, while those with wide splits would be facing a steep climb. The postseason numbers for Hamilton and Jeter show how this plays out.

Triple-Slash Line Comparison
Player Regular Season Postseason
Josh Hamilton .296/.356/.531 .227/.295/.424
Derek Jeter .312/.380/.444 .308/.374/.465

(Career records through 5/30/2014)

Gennaro might have been cherry-picking his examples for effect—Jeter matching or outpacing his regular-season stats frankly isn’t normal—but the principle holds. The postseason is a good time for those who are stout against strong competition, and what holds for individuals ought to hold for entire teams. That’s the assumption I’m testing.

I gathered the records compiled against winning teams by every playoff and tiebreaker participant in major league history since 1903 and the first modern World Series. The records I found at Baseball-Reference combined teams with winning and even records: in a fit of pickiness, I combed out the games against .500 clubs.

Tiebreakers, from the 1946 NL playoff to last year’s Cleveland-Tampa Bay showdown for the AL’s second Wild Card berth, are technically considered part of the regular season. I include them here for a little added data, and because those games, by nature, do also pit winning teams against one another. This means I had to take care to remove the results of those tie-breaking games for the teams I was studying, at least for the tiebreaker. They went back in for regular postseason games.

I did leave the 1981 Royals’ playoff series in the data, even though it defies the assumption on which I’ve predicated the study. Their playoff opponent—ironically, the Oakland A’s—had them dominated in the record splits, whichever way you slice them, and swept them out of the postseason. The effect on the findings is slight.

I’ll start with a quick look at the case that got the ball rolling.

The Particular

Getting a statistically robust finding from the results of 37 baseball games is scraping close to being a fool’s errand. But that’s what I have attempted with the Oakland A’s. If their record against winners is significantly worse than those of its playoff opponents, we’ll have our first piece of evidence that this may be a better way to judge postseason chances.

We do not have that first piece.

Oakland’s recent postseason record has had a remarkable consistency. In six out of seven years, they went the distance in the ALDS before losing the fifth and deciding game. Add in the sweep-and-swept 2006, and they are 15-22 in that stretch. Teasing out a pattern of success and failure from events that consistent would be nearly impossible in ideal conditions. In this case, we may delete “nearly.”

In their eight playoff series of the 2000s, the A’s have had the better overall record six times, and the better record against winning teams five times. No noteworthy difference exists there. It is interesting, if trivial, that in those years Oakland has never had either the best or the worst winning-versus-winners mark. Five of seven years they’ve been second, the other two third. In overall record, they were second six of seven times, the other being fourth and last. Ironically, that was 2006, when they actually won a round.

Going by composite records rather than rankings, the A’s had a .534 record against winning teams in those playoff years, going against teams with a composite .512 mark against over-.500 opponents. By overall records, it was the A’s at .596 and their opponents at .566. The margin is somewhat reduced for the over-.500 games, but they still outperformed their adversaries, and gave us little reason to expect the near-whitewash they suffered.

(Don’t be too surprised by the seemingly low percentages against winners. It’s a tautology, but that group is tough to beat, which is what makes them winners. From 2000 to 2013, the whole AL’s record against winning teams was .445. A .534 mark for the A’s playoff teams is in line with expectations.)

So my original hypothesis receives a decided negative: the idea doesn’t explain the A’s underperformance at all. Perhaps, though, it may hold over a much bigger sample size, for all teams with superior records against good opponents. Let’s give it a look.

The General

A first look at postseason/tiebreaker series throughout history restores some life to my hypothesis. Whether it’s a superficial life, or the kind that matters most, is an intriguingly debatable point, which we’ll get to soon enough.

There have been 295 postseason and tiebreaker series, including single-game knockouts such as the 1948 AL playoff and the current Wild Card games. Of those, 22 have matched teams with identical won-loss records (as would always be the case with tiebreakers). Just three have pitted teams against each other holding equal marks versus winning clubs.

In series with differing overall records, the team with a better record has won 147 times and lost 126, for a mark of .538. In the case versus winning teams, the more successful club has taken 161 series and lost 131, for a percentage of .551. This isn’t a definitive result, as the margin and the sample size are not large enough in combination for that type of confidence. It does point, though, toward one’s record against winners being a better indicator of postseason prospects.

If we want more data points, we can break the series down into individual games. For overall performance, the team with the better record has won 735 postseason and tiebreaker games, and lost 628. The .539 percentage is almost identical to that for series. For success against winning teams, the better club has won 747 games and lost 663, for a .530 percentage. That is not only notably below the record for series, it now underperforms the mark dealing with overall records.

What just happened? Performance against winning teams is the better playoff predictor if we go by series wins, but worse if we go by individual games. (The latter result is not conclusive, either, at the usual 95 percent confidence level.) Which is the better indicator, winning the battles or winning the wars?

Arguably, this is a classic clash between process and result. Just as the process of producing and preventing runs leads, if imperfectly, to the overarching goal of winning games, so does the process of winning playoff games lead to the result of winning playoff series. (This is suspended in the case of single-game tiebreakers and Wild Card playoffs.) It’s not quite the same—you can have a winning record with a negative run ratio, while three blowouts in the World Series still loses to four nailbiters—but it serves our purposes.

I could have dug deeper and calculated the runs scored and allowed in all those playoff games, but a sudden fit of sanity prevented me. I’ll try not to let it happen again.

Like any good sabermetrician, I have to come down, however reluctantly, on the side of process. Series victories are the goal, but the sheer number of games makes them a more trustworthy measure of long-term success. Of course, neither result was conclusive, and if we split the difference, it comes out to performance against winning teams being really no different than performance overall in forecasting a winner.

There is, however, another level deeper to go (without counting up all the runs scored—I’m still in the grip of sanity there). My analysis so far has just checkmarked which team has the better record, and treated each instance as equal. But sometimes the margin is paper-thin, and sometimes it’s a canyon. Might it be that, taking the magnitude of the separation into effect, we could reach a different conclusion?

This requires using the log-5 formula, invented by none other than Bill James to estimate the chances of victory between two teams of known winning percentages. If A and B are the winning percentages of the teams, Team A’s chance of winning is given as:


(You saw a different version of this formula in Steve Staude’s recent article about his tool for calculating game and series win probabilities. They look different, but are mathematically equivalent.)

There is one pitfall in the log-5 estimates: they don’t take home-field advantage into account. For much of baseball playoff history, this doesn’t matter so much. Home field was alternated between the American and National Leagues, and later between the Eastern and Western Divisions. In today’s playoff structure, pre-World Series match-ups have home field awarded to the team with the better record, though earlier in the Wild Card era one could see a division winner with an inferior record get the home edge over a better wild card winner.

This means that, especially for the series of the last two decades that by sheer inflation match all those that went before, the playoff performances of teams with superior overall records should mildly outperform the log-5 estimate. We will keep this in mind when looking at those numbers.

From 1903 through 2013, the average overall winning percentage for teams with the superior record in their match-ups was .621; for the inferior team, it was .578. Putting this through the log-5 formula (noting that I did this with un-rounded numbers having plenty more decimal places), the better teams were expected to win 54.44 percent of those games. As noted earlier, they actually won 53.93 percent. The teams with better overall records undershot the log-5 estimate, even without adding the occasional home-field edge.

Doing this for records against over-.500 opponents, the better team averaged a .573 mark against winners, and the worse posted a .511. The log-5 formula gives the leaders an expected winning rate of 56.22 percent, well over the 52.98 percent they achieved in the actual playoff games. Also, there is no automatic home-field advantage ever given out for being better against winning clubs. There’s substantial overlap with an overall winning mark, so some of the advantages would carry over, but less so. There’s no closing the gap that way.

But perhaps I went about this the wrong way. I lumped all the better and poorer teams together to get collective percentages and a collective log-5, when maybe I should have done it individually for each playoff round, and combined those results. So I re-did it that way.

Postseason, 1903-2013
Measure of Better Team Proj. W%-Coll. Proj. W%-Indiv. Actual W%
Versus All 0.5444 0.5441 0.5393
Versus >.500 0.5622 0.5621 0.5298

It didn’t matter. The log-5 projections budged at the fourth decimal place, and that was all. The relative underperformance of powerhouse-beaters actually widened as measured against that of overall favorites. Overall, teams with superior overall records underperformed their log-5 estimates by about half a percentage point. Teams with superior records against winning clubs undershot the log-5 numbers by three and a quarter percentage points.

Measured by itself, the real-life performance versus winning teams falls well over two standard deviations away from the projections, passing the 98 percent confidence interval. And this is without factoring in the slight home-field effect that would probably widen the gap. The evidence is that, not only is strong performance against good teams not an advantage in the playoffs, but shockingly it appears to be a drag on a team’s prospects, with a margin highly unlikely to be due to chance.

I could have accepted a lack of advantage with equanimity. I’m accustomed by now to my theories getting shot down by the evidence. A result this counter-intuitive, however—doing well against good teams being an absolute disadvantage when facing opponents selected for being good teams—is throwing me down a rabbit hole.

Before I meet any hookah-smoking caterpillars or have playing-card royalty call for my head, I’ll look at the numbers another way to try to start wriggling free of this paradox. I broke out the results for the years 1903 to 1993, which both gives us the era when records never had an effect on home-field advantage and also roughly divides the total number of games in half. I went with a collective log-5, because it’s quicker and obviously makes very little difference.

Postseason, 1903-1993
Measure of Better Team Projected W% Actual W%
Versus All 0.5448 0.5474
Versus >.500 0.5595 0.5386

The fortunes of teams with better overall records nearly match the log-5 predictions. The gap for the better teams against winners has closed, enough so that for this smaller sample it falls well short of even a 90 percent confidence interval. All this means, however, is that the underperformance of projections over the last two decades, the ones obviously most germane to what we can expect to see in the future, has been that much bigger.

Welcome to Wonderland. Or maybe Bizarro World.

What the &%$# Do We Make of This?

I didn’t expect to be here. I was hoping to find the flaw in the Oakland A’s that made some sense of their lousy postseason performance in the Beane Era. I was expecting that ability against good teams wouldn’t make enough difference to explain much of anything. Instead, what I discovered flies in the face of logic—though it turns out it might point toward the A’s situation after all. They’re generally better against good teams than their playoff opponents, and they almost always lose. Their experience fits with the inside-out logic.

I am forced to fall back on what scattered learning about logic I have: if the logic is sound but the conclusion is nonsensical, check your premises. My foundational premise was that excellent play against winning teams from April through September would carry over into October. If that’s not so, then what about playoff baseball makes it different from the 162 games that preceded it?

One answer, at least more recently, is that pre-World Series playoff rounds give out home-field advantage (usually) to the team that finished with a better record. This doesn’t have nearly the effect regarding records against winners than it might. In the 273 postseason series where one team had a better overall record, 185 or 67.8 percent also had the superior record against teams over .500.

The problem is that even this attenuated home-field advantage for the contender-beaters should be raising their postseason record, when instead it falls well below expectations. Worse, the numbers took a tumble from pre-1994 to post-1994, the point at which home-field kicked in and we should have seen an uptick. So the paradox only deepens.

What other differences are there in playoff baseball? The cliche about baseball’s long season is that it’s a marathon, not a sprint. In the postseason, it is a sprint, or a string of them if you win early on. I’ve noted already that pitching staffs get handled differently come the postseason. This happens to position players, too: they, especially catchers, don’t get benched for a day to rest in the middle of a playoff series.

Is it possible that this could be a key? If you postulate that the lesser team in the match-up has an inferior record due to a lousier bench, back end of the rotation, etc. rather than for weaker front-line players, and if you speculate that it was the winning teams that were able to exploit the performance of the scrubs more effectively, then their absence might swing the odds. Those are, however, pretty big ifs.

Could it be a matter of attitude instead? The teams with robust records against winners could be complacent, while those struggling might have the sense to realize they’re on the wrong end of things, and look to make changes to give themselves a better shot. Baseball does have a way of leveling things, call it regression or what you will, and the opening of the postseason is a natural inflection point for teams to re-evaluate themselves.

Why this regression would manifest itself with records against winning teams much more than with full season records is a difficult matter. It would imply that teams are conscious of how they do against good opponents. It’s a fairly esoteric matter for fans; would they as players, and managers and coaches, see it differently?

I am already straining to come up with these theories. Any others I devised might well stretch credulity to the snapping point. Goodness knows that the theories other baseball commentators have offered for the original case haven’t exactly warmed sabermetricians’ hearts.

The accusation leveled against the A’s more than a decade ago was that playoff baseball was fundamentally different, requiring stratagems to manufacture runs rather than playing station-to-station ball, and that Beane’s team not only could not adapt but refused to. Adherents of baseball analytics rejected this as Neanderthal thinking, but the revival of the A’s fortunes have also brought the same familiar results in the ALDS.

One almost throws up one’s hands and says it has to be something psychological. Possibly it is: it’s humans playing these games, and humans are way more complex than mathematical equations for offensive production. But it’s unquantifiable, irreducible, effectively a black box. It may be that this comes from something we cannot yet understand, but such an explanation cannot be a satisfactory one.

So I will not force the issue. The result is what it is: teams with superior records against winning ball clubs have that strength crumble once the postseason comes along. As for the reasons, your guesses are as good as mine. Maybe even better.

A writer for The Hardball Times, Shane has been writing about baseball and science fiction since 1997. His stories have been translated into French, Russian and Japanese, and he was nominated for the 2002 Hugo Award.
Newest Most Voted
Inline Feedbacks
View all comments
8 years ago

Could the record against winning teams be a proxy for the difficulty of the pennant race? The fact that this shows up most strongly in the last two decades, where unbalanced schedules have been in effect, makes me wonder. I would have to think that being able to coast into the postseason and slot your top starting pitchers to begin each series, rest key players etc. would have provide a material advantage against a team that had to go down to the wire.

8 years ago

The team of the last 2 years doesn’t have anything in common with the Mulder/Zito/Hudson/Chavez/ Tejada teams except laundry.

Why have they lost in the playoffs the last two years? Two words:

Justin Verlander

8 years ago
Reply to  Rally

Young players against more seasoned veterans is at play too here I think. They also gave the Tigers a run for there money, and had both series at one point only to lose late.

Jon L.
8 years ago

I thought you were going in a different direction when you said, “I am forced to fall back on what scattered learning about logic I have: if the logic is sound but the conclusion is nonsensical, check your premises.” But then you keep trying to figure out why the A’s performed so poorly in the playoffs.

“In six out of seven years, they went the distance in the ALDS before losing the fifth and deciding game.” A great team can go 15-22 in a 37-game stretch against mediocre teams; they can certainly go 15-22 against other great teams. You’re putting a lot of focus on those six game 5 losses. If a good team went 0-6 over 6 games, or 1-7 over 8 games, what answers would you expect to find? It’s just not a very big sample.

It could be that the A’s focus on depth rather than stars weakens them relatively to the other teams in the postseason. It could be that other teams’ front-line pitching and relief were better, and could pitch more in the playoffs. But it certainly is the case that it’s a small number of games, and it wasn’t statistically predictable that the A’s would lose those games. They just did.

It’s still a great article. Really interesting analysis on performance against good teams and bad teams.

8 years ago
Reply to  Jon L.

Not really. They have the most valuable player int eh game over the last two years by WAR (Donaldson), for example. They have had some of the top pitchers in the league pitching for them in the playoffs. Giambi and Tejada both won MVP’s, etc.

Marty Schottenheimer
8 years ago

It comes down to a player’s upside potential and the extent that they are able to play close to maximum potential. Think of the Bell Curve from stats class. Not unlike Marty Schottenheimer’s KC Chiefs who played well in regular season, but fail in the playoffs, the A’s have regular season over-achievers that were managed adeptly. Over a long season, MLB players only put in a position where the likelihood of success is the greatest, will play to a level above their statistical mean; in the Bell Curve they are in a standard deviation the right of the mean.

Two conclusions can come from this. 1) The players cannot raise their game as much – they are already over-achievers playing to the right of the mean. More likely, statistically, is that they regress to the mean. 2) In the post-season, every single game is crucial. Managers can no longer make decisions that play out well in the long-term, but hurt chances in the immediate situation. How well did Weaver’s Orioles, in 1971 with 4×20 game winners, fare against Pirates when they only could use 3 starting pitchers in World Series? The better team lost.

Hard to have sympathy for a team that has such a great record of consistent excellence.

8 years ago

I disagree strongly. Just look how they’ve lost some of these series – it sometimes came down to one bloop hit, one dropped ball, one mis-called play by an umpire, etc. In other words, stuff that’s just part of the normal variability of the game of baseball.

Marc Schneider
8 years ago

I believe that teams going in as favorites-which would normally be the case with the team with the better record-have far more pressure to win. Baseball is a game that requires composure and relaxation. Teams that barely make it or have lesser records may have less pressure and, therefore, perform better.

Another issue, in my mind, is that W-L records today might be misleading due to differences in schedules and the sequencing of games. For example, a team that played the Dodgers in early 2013 (pre-Puig) might have run up wins against them when they were playing poorly which might be misleading; by the time October rolled around, they were a different team (as the Braves found out). Moreover, season records against particular teams, especially non-divisional teams, can be skewed by a number of factors; for example, if one team misses the ace. The Braves had a winning record against the Dodgers in 2013 but never faced Clayton Kershaw until the playoffs. If they had faced Kershaw, their record against the Dodgers would likely not have been as good. Or, again, in a few games, some wins might be flukey; a number 8 hitter might hit a home run to win a game. The point is that, under today’s schedules, it’s hard to actually know who is actually bthe better team. And that’s ignoring the obvious fact that, in a short series of baseball games, the best team has a relatively slight advantage over even the worst team.

All this goes to show that, as Beane suggested in his colorful way, you can build a team to make the playoffs but not really to win the playoffs.

8 years ago
Reply to  Marc Schneider

Actually, if you looked at the Dodgers record with Puig, his hot hitting, while it did improve their record, they went from losers to a .500 team with him. Same with this season, he’s been white-hot in May but they had a .500 record roughly.

They didn’t really take off until Hanley returned and started hitting out of his brain, and that coincided with two great months of pitching from the starters AND relievers, starters all under 2 ERA, relievers all under 1 ERA, hard to lose when your pitchers are doing that well collectively.

Still excellent point about how the schedule could skew results for teams, where certain teams miss the other team’s ace (or face the other team a heck of a lot, the Dodgers seems to figure out how to exactly put Kershaw up against the Giants a lot).

I would note that there is a way to bend the odds more to your favor, and that’s by having a rotation of guys who throw quality starts out regularly.

8 years ago

Frankly, I still like BP’s research on why Billy’s stuff don’t work in the playoffs even if they have abandoned it (which is not too surprising, the creator left). The findings focused on a strikeout oriented pitching staff, a strong closer, and strong defense. Billy has only that last one, again, because he thinks closers are fungible and he’s never been that interested or able to put together such a pitching staff.

Pitching rules. It is logical to me. We all know about how some pitchers pitch more quality starts than others, some a lot more. When you get into the playoffs, you are left with the creme de la creme for many teams, some more creme than others. If a team has more pitchers who pitch a high percentage of quality starts than another team, then logically and obviously, that team will be more likely to go deeper into the playoffs.

I use a modified form of quality starts called Pure Quality Starts by Baseball Forecaster, and they define a quality start using sabermetric standards, like K’s double walks, etc. I studied the playoffs in recent years and found that when you have a quality start by one pitcher and a non-quality start by another, his team wins around 80% of the time. Obviously two quality starts mean a coin-flip .500, someone wins, someone loses. So you can improve your odds of going deeper into the playoffs by having a staff of quality starters.

I know that’s a no-brainer, but nobody I’ve seen have taken that factoid to its logical conclusion: if you build a starting rotation that routinely spits out quality starts, like the Giants in recent years or the Phillies once they got Halladay, that greatly improves the odds of you winning a series. And thus going further into the playoffs. And strikeouts is a huge part of the PQS methodology, representing 40% of the rating, meaning, basically, if you got a pitcher who don’t strike out too many (as the A’s typically do), unless he’s a control master like Maddux (or Hudson this season) and walk very few, they won’t have too many quality starts.

Now, this is no guarantee, which is why I think BP was wrong to declare the tool was now not working. And that is partly the creator’s fault, because he’s the one who started using the tool wrong at BP. He used his methodology to show who is rated higher than other teams but his mistake was to do a simple rank of the teams that season. He should have been ranking them among ALL the teams in their historical database.

Just because a team was ranked higher does not mean it was much higher. If they are virtually the same, of course the series is probably a coin toss at best. But if one team was among the historic bests, then their odds improve, like the Giants in 2010, I looked up their numbers and the numbers of the top 10 they had in the book, and they were up there.

Also, the tool never said that having a higher rating means that team would beat a lower rating team. That is where the creator screwed up his study findings. It just meant that they were more likely to go deeper.

For example, if the teams in a season all had the lowest ranking historically, obviously one team has to advance, but they were all rated the worse all time, just slightly different. A lower team beating a higher team here means nothing if they are basically the same. It does not mean that the study results does not work.

So you need a strong rotation top to bottom for the playoffs, you need a strong closer, and you need good defense (to help the strong starters get even more outs). Good formula, Beane has never followed it though.

8 years ago

Has anyone done a time series analysis (or GMM) treating the regular season/postseason as a quasi experiment or event study to study the correlation (stickiness) of statistics? Those that are more meaningful model elements are probably worth studying more. However, and I think this is generally true of predicting what is in the end a really small sample and really uneven cell sizes, the variance in outcome is so high that the effect sizes of any model built off weighting factors based on their strengths as an instrument will be so small that the error term effects DWARF the mediating effect of the factors.

8 years ago
Reply to  Joe

also, another thing about the differences in sample- Are the variances between the samples heteroscedastic? They could well need to be transformed to perform an analysis.

Paul M
8 years ago

We A’s fans have lived the playoff frustration. It’s a sample size question.

In 2000, the two-time defending WS champs essentially played possum the final month, and because rookie pitcher Mark Mulder (who had found his groove in mid August) got hurt and the A’s had to push to the final day to make the playoffs (Ace Tim Hudson pitched that clinching game), Gil Heredia was forced into two starts, including the deciding 5th game. In 2001, we all know what happened in the 7th inning of Game Three– a once in a lifetime play by Derek Jeter. A’s would only have tied that game, but up 2-0 in games and 1-1 heading to the late innings, I’ll take my chances. They fell apart the next day (and lost Jermaine Dye to a brutal freak fouled off pitch/broken leg injury in the first inning), which led to the 2002 fail– since the Big Three decided not to trust the pitcher who had been shelled in that Game Four (Cory Lidle) and who also was a scab. Since a three man rotation thus ensued vs the Twins, Beane and Howe determined that the A’s best pitcher, CY winner Barry Zito, could only pitch once (he had never started on 3 days rest), and when Hudson got injured (and told no one about it) all of a sudden their principal advantage vs the Twins was out the window in the one series where they were clearly the superior team. The next year Byrnes and Tejada decided not to bother with home plate in Fenway, but the other two reasons they lost were: 1) starting pitching injuries– again– Mulder was out for the entire playoffs and Hudson got hurt in a Boston bar fight; and 2) an inexplicable decision by Ken Macha to bring back Keith Foulke 18 hours after a 50 pitch/multi-inning appearance the night before to close out a 4 run lead in Game Two, thus rendering him ineffective for the final three games. They won the first series in 2006, and have since lost thrice to Detroit, with 2012-13 basically being Verlander defeats…

The current A’s are the deepest offensive team with the deepest pitching staff of this bunch– though they may be (Gray’s youth and Kazmir’s health being the question marks) a little thin at the front of the rotation. We’ll see– but I simply do not think their failures can be ascribed to anything but: 1) luck; 2) the quality of the competition and 3) the impact on a small-budget team of injuries.

Paul M
8 years ago

and as to the formula, I beg to differ. Isringhausen was a strong closer in 2000-01; as was Foulke in 2003 and Balfour in 2012-13 and, for that matter, Huston Street in 2006. Billy Koch in 2002 was the sketchiest– and Howe can be blamed for pitching him all three games of a meaningless final weekend in a vain attempt to get him “right” after an injury and blown saves. A’s have always been among the leaders in defensive efficiency (they are first at present, I believe) and this has clearly become a new aspect of Moneyball given market inefficiencies. And I don’t have all the stats at my disposal but staffs consisting of Hudson, Mulder, Zito, Haren, Harden and, more recently Gray, Anderson, Kazmir (not yet tested in a playoff, I accept), Parker have hardly been deficient in the strikeout department.

8 years ago
Reply to  Paul M

The formula is not starting rotation, but pitching staff K-rate.

This year, basically average, 8th in AL:

2013, 7.3 vs. 7.7 average, 12th in AL:

2012, 7.0 vs. 7.4, 12th in AL

2006, 6.2 vs. 6.4, 10th in AL

2003, 6.4 vs. 6.1, 4th in AL

2002, 6.3 vs. 6.3, 7th in AL

Not once were they in the top 3 for the AL, and only once a lot higher than average. And that’s vs. the league, BP compared with historic playoff teams.

And for closers, they were measured by WRXL, and from BP’s book, the A’s closers were not that great historically.

From the book:

2000: closer was 155th (out of 180), FRAA 107th, K-rate 150th
2001: closer was 81st, FRAA 148th, K-rate 62nd
2002: closer was 111th, FRAA 19th, K-rate 103rd
2003: closer was 92nd, FRAA 17th, K-rate 80th

Meanwhile, when I tried to reduplicate the study to see what the Top 10 overall had, and compared the 2010 Giants against that, they were pretty near a Top 10 overall historic team. They were built to win it.

Brandon Firstname
8 years ago

Ridiculous article.

It’s randomness. Of course it’s randomness. A coin flipped eight times, and it landed on tails seven of them. That doesn’t mean that we need to re-evaluate how coins flip.

And yeah, baseball’s not a coin flip. But yeah, it’s pretty darn close. Especially in the playoffs. Everyone in the playoffs is good.

8 years ago

Actually, my view on the A’s performance in the playoffs vs. in the regular season rests on the A’s focus on youngsters.
The youngsters can go all out – play every inning as if it were the 9th. The A’s organizationally encourage this – which is a great thing for the regular season and absolutely contributes to their large run differential.
Few other teams think this way, however. More veteran teams/players – when the margin of winning is already large, slack off visibly. This can range from swapping out players early to the same player staying in, but mailing their play home.
Now why would this matter? My suspicion is that by playoff time – the A’s players have literally shown their all to everyone. There are no mysteries or (deliberate or accidental) deceptions left. Their opponents know exactly what each player can and cannot do, and can effectively plan against them.
The notion that the losses are due to bad luck isn’t automatically valid to me – because the A’s were rarely in such positions during regular play. The fact that the series were even that close might be simply an indicator that the A’s opponents were better prepared than in the regular season. As the above article notes – 3 blowouts is not as important as 4 nail biter losses in the World series or division championships.
Just an idle speculation.

8 years ago

I enjoyed this article, but I think your problem comes down to the answer to this question:

How many flips of a coin with 54% chance of heads does it take to reject the hypothesis that it is a fair coin?