The Magic and Mysticism of Baseball’s Projections

PECOTA. ZiPS. Steamer. Marcel. Oliver. Chone (RIP). CAIRO, MORPS, and BEANS. We turn to these projection systems every winter to paint the picture of the season to come. We use them to illuminate next year’s standings and All-Star teams, divine fantasy sleepers and busts, and reveal the hazy reasoning behind the hectic hot stove. The development of multiple freely available projection systems may be the the greatest contribution of sabermetrics to the baseball-watching public.

Building a better projection system is a tried-and-true track to a job in baseball. Nate Silver’s PECOTA spreadsheets wedged open the door to Baseball Prospectus. Sean Smith’s CHONE was ripped from the internet after the 2010 season when he found a job in a front office. Many of the systems listed above are proprietary, and fans, fantasy players, stats websites and even major league teams shell out cash yearly for access. The baseball community values anything it believes can deliver an accurate prediction for the coming season, a commitment it backs up in dollars and jobs.

Projections elevate sabermetricians to sabermagicians. Although much of sabermetric thought draws from the logical tradition, projection demands creation. No matter how much time is spent poring over historical statistics and deducing trends, projecting a hypothetical baseball season requires another significant step. Logic is solely deductive; it lacks the ability to create. Going from principle, theory and conjecture to real win totals and player statistics requires the projectionist to step outside the realm of logic and into the realm of mysticism.

The role of the projectionist bears a stark resemblance to the Mesopotamian baru, a priest-like figure whose rituals were a hugely important part of the decision-making process in their society. The baru practiced what we now call “divination” and would read the movement of drops of oil along a bowl, or the movement of smoke rising from incense, the patterns of hot candle wax dropping from the wick, or even the shapes and colors of the organs of sacrificed animals. The baru‘s ritual was always preceded by a prayer, in which the baru asked a god to reveal his intentions through the chosen medium.

As absurd as it sounds, the baru‘s rituals were treated with great seriousness and rigor. Historian J.J. Finkelstein wrote in a 1963 scholarly article titled “Mesopotamian Historiography,” “Sacrifices of animals were routine in the city temples, often being made on behalf of the king, who, on numerous occasions even offered the sacrificial animal on purpose. After slaughter, autopsies of the entrails were made as a matter of course, and a detailed record made of the findings against any contingency that might arise. Or, a clay model of some of the organs might be made in order to preserve the exact features of the original. The pathologists, so to speak, would consult their records in order to match up the current case with one in the past.”

Subtract the blood and guts and organs and you essentially have Nate Silver’s PECOTA development process. “Eventually, by stealing an hour or two a ta time during slow periods during the workday, and a few more while at home at night,” Silver wrote in The Signal and the Noise, “I developed a database consisting of more than 10,000 player-seasons…as well as an algorithm to compare any one player with another. It used a different method for comparing a set of players–what is technically known as a nearest neighbor analysis. It also considered a wider variety of factors–including things like a player’s height and weight, that are traditionally more in the domain of scouting.”

It may be odd to imagine Silver sifting through the digestive track of an expired sheep to project Dustin Pedroia as a perennial All-Star, or to imagine Dan Szymborski emerging from his computing cave with the ZiPS projections in the form of a clay model of a cow’s brain. But if we step aside from the sensational gore, an analysis of how the Mesopotamians approached this ritual reveals a strikingly similar intellectual tradition.

“The Mesopotamian form of learning known as ‘divination’ was rooted in, and is most characteristic of, the fundamental cognitive mode of the Mesopotamian intellect,” Finkelstein writes. “There probably has never been another civilization so single-mindedly bent on the accumulation of information, and on eschewing any generalization or enunciation of principles.” As Finkelstein explains, the Mesopotamian tradition fixated not on the uniqueness of historical events, as the Western world does, but rather on what he calls their “exemplicative value.” Nothing occurred by chance. Anything that can be observed could be known and understood empirically. For the Mesopotamians, Finkelstein writes, “Ultimate understanding of the universe would, in theory, require nothing but the painstaking accumulation of as much detail as possible about literally everything.” The Mesopotamians would have loved StatCast.

The result of the unceasing Mesopotamian attention to detail was the omen texts. “If a town is set on a hill,” one reads, “it will not be good for the dweller within that town.” Another, “If black ants are seen on the foundations which have been laid, that house will get built; the owner of that house will live to grow old.” And one that may have been warning us about Nate Silver himself, “If a fox runs into the public square, that town will be devastated.”

These omens read like religious proverbs, designed to serve as overarching social responses to certain situations. But this reading is overly simplistic, as Finkelstein explains. “A moment of time was apprehended and defined as the sum total of the occurrences and events known to be in temporal conjunction.” The omen isn’t making a suggestion that building a town on a hill may be dangerous. It is, rather, a historical retelling of how, in every case the writer of the omen has witnessed, the construction of towns on hills has ended poorly for residents. This is a limited view of history, to be sure, but it is history nonetheless.

“The greater the number of events noted for a single moment,” like the moment of a town being built upon a hill, or a fox running into the public square, or a 37-year-old kunckleballer posting a Cy Young season, Finkelstein writes, “the more refined and precise the prediction that could be based on them.” This is the exact philosophy Silver carried into his PECOTA project, and one every other projectionist has been mimicking for the past decade or so.

For us, with this philosophy, there is always the danger of injecting the self into the project, forcing the divination to tell us what we want to see. This was not an issue for the Mesopotamians. According to Finkelstein, “The universe of Mesopotamia was assuredly not a man-centered one; his mode of awareness drew his attention first to the external world, and secondarily, at best, to himself.” It is this hyper-awareness of surroundings and lack of self-awareness that allowed the Mesopotamian to see a clear future in nothing but a drop of oil or a trail of intestines.

Similarly, our own self-awareness allows us to see through the flaws in the divination process. Hyper-awareness of our own selves, of the many differing paths our single lives can take, forces us to run our simulations thousands of times before we are satisfied the effects of “luck” have been wrung out. Self-awareness forces us to see the thin or non-existent lines of causality tying the divination process to the events it is attempting to foresee. Even if it has been right before, we need a compelling causal reason why the divination is able to foretell the future before we can accept it. The Mesopotamian, with such a deep focus on the external world, instead requires a counter-example before disengaging from the belief.

As superior as our way may seem, the self can get in the way, especially when there is a desire to see the projection come to fruition. In the case of black box projections, the only rationale giving weight to the projection is the authority of a single statistician, organization, or “expert.” In others, statistics an individual or organization holds dear or even may be trying to sell may be emphasized while others are held back. Sometimes the explanation is too loaded with jargon for a layman to be reasonably expected to understand.

While there should be no doubt that every projectionist has a process, if that process is not publicly available and accessible, we are left with nothing but their word that the projections are honest and not instructed by the personality or preferences of the projectionists. It is the explanation, the reason why the divination works and could be successfully repeated, that separates the successful projectionist from the baru digging for livers and gall bladders. And if we wouldn’t listen to the guts of a sheep we don’t understand, why would we listen to the guts of a system kept secret?

Thus far, there only seems to be one acceptable way for our projectionists to establish an authority for their method: lower the RMSE, or “root mean square error.” In an analysis of numerous projection systems at Baseball Prospectus this December, Rob Arthur found what I consider to be a shocking amount of agreement. Arthur points out that players who had unexpectedly explosive seasons in 2014, like Cleveland’s Michael Brantley or Detroit’s Victor Martinez, were not just missed by one or a few of the projection systems under the microscope. “The astonishing fact,” Arthur writes, “is that not one but all of the systems missed on these players.”

“While the details differ,” Arthur continues, “projections are by and large similar to each other.” If we were like the Mesopotamians, unaware of ourselves with unshakable external focus, perhaps this would be a point in our favor, an indication that we are on the right track, and as we continue to add information to the pile, our predictions will only improve. But we know better. We know there is a limit to what we can understand about the future. We know that it will be unique, and thus we know you can’t predict baseball (you just can’t!).

The value of an individual projection system, then, cannot be simply that it gets it right. Every form of divination is flawed. Any attempt to collect all information will inevitably fail–it will miss something that wasn’t recorded, something that wasn’t considered, perhaps even something that doesn’t exist yet. And no matter how logical the projection system is, this information hole at some point will reveal itself, and the world of projection and world of reality will from then on be fundamentally irreconcilable.

Projectionists, faced with a totally impossible task, should embrace the mystical side of the craft. A projectionist working towards increasing our knowledge of the game should feel less desire to slice decimal points off RMSEs and more desire to find something new within this beautifully massive set of data the projectionists work with to tell us about how the game is played and why teams and players succeed and fail. That Steamer says Steven Souza and Wil Myers will be equals in 2014 is far less interesting than the divination process that leads to this surprising, jarring conclusion, and if it can tell us something about why our assumptions are so far removed from the world Steamer has created for 2015.

What’s more interesting, a projection system that tells gets it slightly less wrong than the rest of the pack, or the one that finds another Jose Bautista or R.A. Dickey? Such a system would have to be based on fundamentally different ideas than those currently seeding the projection processes used today. It would have to be built, essentially, from nothing. And any such successful system would be the result of a number of failures in the process.

It sounds daunting, to be sure. But our current methods do little but add an incremental amount of information to assist with the task each year. The Mesopotamians would be proud. We, on the other hand, should be fully aware of the flaws in the process. Without embracing this mysticism and admitting it is part of how we build and think about our projections, we are nothing but barus digging for insight in the digital equivalent of sheep entrails.

References & Resources

• Arthur, Rob. “Moonshot: The Power of Projections,” Baseball Prospectus.
• Jaynes, Julian. The Origin of Consciousness in the Breakdown of the Bicameral Mind. Houghton Mifflin Harcourt. Kindle Edition.
• Russell, Bertrand. Mysticism and Logic and Other Essays. Kindle Edition.
• Finkelstein, J.J. Mesopotamian Historiography. Proceedings of the American Philosophical Society, Vol. 107, No. 6, Cuneiform Studies and the History of Civilization (Dec. 20, 1963), pp. 461-472