Beyond Moneyball: Player Development Part 3

(Previously Part 1, Part 2)

Selecting good players is making good player selection decisions.

A “decision” is a commitment to a course of action that is intended to yield results that are satisfying for specified individuals. (“Decision-Making Expertise,” by J. Frank Yates & Michaell D. Tschirhart)

Decisions (which players to select in the MLB player draft) having a probability of 2 in 50 of creating satisfaction (actually becoming productive MLB players) for owners and general managers and probably most important of all, the fans, does not fit the definition of “good decisions” IF the expectation is something greater than 2 in 50.

A “high-quality decision” is one that does indeed achieve such satisfying results.

What is of particular interest is that experts who study the decision-making process, while agreeing to the definition of what a decision is, do not believe that good results are indicative of decision-making “expertise.”

“Information that is available only after a decision is made is irrelevant to the quality of the decision.”

“A good decision cannot guarantee a good outcome. All real decisions are made under uncertainty. A decision is therefore a bet, and evaluating it as good or not must depend on the stakes and the odds, not on the outcome.”

Researchers use different criteria for evaluating the decision-making process. “Coherence” refers to the procedures employed in making decisions.

“Procedures are ‘logically coherent’ if they do not contradict themselves, or equivalently, do not allow the decided to be self-contradictory in particular ways.”

One way to look at the concept of “coherence” is to consider a good decision-making process as one that maximizes the expected return on the decision.

How does one actually determine if someone is a good decision-maker? If one subscribes to: “A decision is therefore a bet, and evaluating it as good or not must depend on the stakes and the odds, not on the outcome”, then a good decision-maker is one who, over a lengthy period of time, wins more often than he loses.

At a fundamental level, what separates good (professional) gamblers and novice or problem gamblers is the factor of self control. The general rule of thumb for players is to avoid becoming emotionally involved in the game. Inducing emotional (rather than logical) reactions from gamblers is what makes the gambling industry so profitable. By remaining unemotional, players can protect themselves from recklessly chasing losses and avoid going on “tilt.”

Unfortunately, human nature being what it is, we all are vulnerable to the phenomena called “attention decrement.” Attention decrement states that people are inclined to observe a small number of cases, draw a conclusion on the basis of those cases, and then simply stop paying attention to pertinent facts that present themselves later. This is also a phenomena related to what Tversky and Kahnerman call “the belief in the law of small numbers,” which implies that pursuing further cases beyond the first few is unnecessary as well as burdensome.

Attention decrement and the law of small numbers suggest that when attempting to evaluate decision-making expertise, if the first few decisions associated with the decision-maker turn out right, we apply the label “expert.” But if they turn out badly, the decider might well be called “inept.” And our opinion gets cast in stone simply because we are too busy or too lazy to examine the entire width and breadth of decision-making. Often times it is the exception that proves the rule, which is to say it is unreasonable to say that we should ignore the results of a single decision when trying to infer a decider’s expertise.

Beliefs regarding decision-making expertise are also driven by one’s perception versus the reality of the situation. There is the tendency to infer decision-making expertise from subject matter expertise, i.e. equating a person’s knowledge of the subject to their ability to make good decisions. In-depth knowledge is a logical requisite for good decision-making, but knowledge in itself does not equate to good decision-making.

“Consensus among peers” is another often used criteria for judging decision-making expertise. “Experts” are people who already are acknowledged by their peers as experts. This begs the question of how such impressions of expertise arise, especially if one subscribes to the law of small numbers combined with the “luck of the bet.” In other words, a few good decisions in the beginning of one’s career can go a long way toward earning one recognition from one’s peers as an expert decision-maker. Also, factors such as personal style (form versus function) combined with good presentation and communication skills can play a significant role in judging who is or who is not an expert decision-maker. Self-confidence is a particularly strong factor in the perception of a good decision-maker.

Selecting players: Decision-making and rolling the dice.

Decision-making is the cognitive process leading to the selection of a course of action among variations. Every decision-making process produces a final choice. It can be an action or an opinion. It begins when we need to do something but know not what. Therefore, decision-making is a reasoning process that can be rational or irrational, can be based on explicit assumptions or tacit assumptions. (Wikipedia 2007)

All decisions are subject to probability; when you make a decision, you’re making a bet on an outcome. Decision-making as it is applied to selecting and developing players, monetarily, is no different than a game of chance. If one is playing a game of chance where money is involved, the overall (end of the night/day) success or failure of your decisions is dramatically affected by the size of your bankroll. The bigger your bankroll, the more latitude you have to make poor decisions, which follows the dictum of the “Kelly criterion.”

Kelly criterion

In probability theory, the Kelly criterion, or Kelly formula, is a formula used to maximize the long-term growth rate of repeated plays of a given gamble that has positive expected value. It was described by J. L. Kelly Jr. in a 1956 issue of the Bell System Technical Journal[1]. The formula specifies the percentage of the current bankroll to be bet at each iteration of the game. In addition to maximizing the growth rate in the long run, the formula has the added benefit of having zero risk of ruin; the formula will never allow a loss of 100% of the bankroll on any bet. An assumption of the formula is that currency and bets are infinitely divisible, which is not a concern for practical purposes if the bankroll is large enough.

The most general statement of the Kelly criterion is that long-term growth rate is maximized by finding the fraction f* of the bankroll that maximizes the expectation of the logarithm of the results. For simple bets with two outcomes, one involving losing the entire amount bet and the other involving winning the bet amount multiplied by the payoff odds, the following formula can be derived from the general statement:

f* = (bp-q)/b
where
f* is the fraction of the current bankroll to wager;
b is the odds received on the wager;
p is the probability of winning;
q is the probability of losing, which is 1 – p.
As an example, if a gamble has a 40% chance of winning (p = 0.40, q = 0.60), but the gambler receives 2-to-1 odds on a winning bet (b = 2), then the gambler should bet 10% of the bankroll at each opportunity (f* = 0.10), in order to maximize the long-run growth rate of the bankroll.
If the gambler has zero or negative edge, i.e. if b = q/p, then the gambler should bet nothing.
For even-money bets (i.e. when b = 1), the formula can be simplified to:
f* = p-q
Since q = 1-p, this simplifies further to
f* = 2p-1

The Kelly criterion was originally developed by AT&T Bell Laboratories physicist John Larry Kelly Jr. based on the work of his colleague Claude Shannon, which applied to noise issues arising over long distance telephone lines. Kelly showed how Shannon’s information theory could be applied to the problem of a gambler who has inside information about a horse race, trying to determine the optimum bet size. The gambler’s inside information need not be perfect (noise-free) in order for him to exploit his edge. Kelly’s formula was later applied by another colleague of Shannon’s, Edward O. Thorp, both in blackjack and in the stock market.[2] (Wikipedia 2007).

In the language of the Kelly criterion, organizations that have limited bankrolls need to increase their “p” (probability of winning, increasing their odds of selecting the best player), decrease “q” (their probability of taking a player who does not pan out) while at the same time increasing “b” (odds received on the wager, i.e. expected return on player selected, ideally a 20-game winner within two to three years after being drafted).

The Kelly criterion applied to major-league baseball is quite straightforward and intuitive. Organizations that have more money to spend have the greater chance of fielding players that win games because they have greater forgiveness in both the number of choices they make and forgiveness for making poor player choices. Simply stated, executives that win games with smaller budgets are “probably” better executives than those who win games with large budgets.

The corollary to this is that unsuccessful organizations seek executives from organizations that have small payrolls under the assumption that these executives a more effective than executives of teams that have similar success but higher payrolls. This quite often is a fallacy of assumption similar to assuming that player physical attributes equate to untapped potential.

This hiring and firing practice also has the potential to enslave all major-league organizations to the principle embodied in the Kelly criterion, that in the end it is the teams with the largest bankrolls that will have the greatest success. Simply because over time and repeated hiring and firing of the same bodies, the continual exchange of bodies by organizations within the MLB domain creates a commonality of personnel expertise. And in so doing, organizational cultures and practices will blend together within the entire MLB domain to produce a homogeneous distribution of expertise such that all organizations within the MLB domain will have essentially the same probability of winning “p” and probability of losing “q.”

Teams with large bankrolls (revenues and willingness to spend on player signings) can engage in a form of Martingale spending on players.

Martingale (betting system) originally referred to a class of betting strategies popular in 18th century France. The simplest of these strategies was designed for a game in which the gambler wins his stake if a coin comes up heads and loses it if the coin comes up tails. The strategy had the gambler double his bet after every loss, so that the first win would recover all previous losses plus win a profit equal to the original stake. Since a gambler with infinite wealth will with probability 1 eventually flip heads, the Martingale betting strategy was seen as a sure thing by those who practiced it. Unfortunately, none of these practitioners in fact possessed infinite wealth, and the exponential growth of the bets would eventually bankrupt those foolish enough to use the Martingale. Moreover, it has become more impossible to implement in modern casinos, due to the betting limit at the tables. Because the betting limits reduce the casino’s short term variance, the Martingale system itself does not pose a threat to the casino, and many will encourage its use, knowing that they have the house advantage no matter when or how much is wagered. (Wikipedia 2007)

Major league baseball has attempted to deal with the inequalities between team bankrolls in a different way through implementation of a luxury tax and draft bonus slotting. This is analogous to casinos setting a table or house limit on betting. Unfortunately for low-revenue organizations, unlike casinos, the business of major league baseball has very soft table limits and teams frequently ignore slotting limits and/or are willing to absorb luxury tax penalties. This again reinforces the need for organizations that wish to practice tight fiscal responsibility to take a serious look at their player selection and development practices.

For any activities involving the element of chance (decision-making) and lacking any other measurement criteria (or capabilities), continued success over time is a reasonable indicator that the person and/or organization is doing something right. And those individuals and organizations that can achieve long-term success with small bankrolls are probably better decision-makers. The key phrase here is “long-term success with small bankrolls.”

Drafting and selecting players: an exercise in making good decisions.

Good decision-making is multi-faceted and can be viewed as a series of issue resolutions.

Issue 1-Need: “Why are we (not) deciding anything at all?”
Issue 2-Mode: “Who (or what) will make this decision, and how will they approach that task?”
Issue 3-Investment: “What kinds and amounts of resources will be invested in the process of making this decision?”
Issue 4-Options: “What are the different actions we could potentially take to deal with this problem we have?”
Issue 5-Possibilities: “What are the various things that could potentially happen if we took that action—things they care about?”
Issue 6-Judgment: “Which of the things they care about actually would happen if we took that action?”
Issue 7-Value: “How much would they really care—positively or negatively—if that in fact happened?”
Issue 8-Trade-offs: “All of our prospective actions have both strengths and weaknesses. So how should we make the trade-offs that are required to settle on the action we will actually pursue?”
Issue 9-Acceptability: “How can we get them to agree to this decision in this decision procedure?”
Issue 10-Implementation: “That’s what we decided to do. Now, how can we get it done, or can we get it done, after all?”

While the above list of issues and their resolution help qualify what constitutes a good decision-making process, it does not tell us specifically the characteristics of a good or expert decision-maker.

A good decision-maker has the ability to resolve issues through a combination of symbolic (knowledge) and heuristic (experience) manipulations. Experts know more about the domain and therefore can access and use their knowledge more efficiently than the novices. Though the semantic knowledge of the domain—that is, pieces of factual knowledge of the domain—may not vary between experts and novices, the episodic knowledge (past experiences) of/within the domain permits the experts to link and evoke the relevant and appropriate inter-connected pieces of knowledge in the problem-solving processes.

Experts differ from non experts in ways of their handling and solving problems in relation to the problem representation, constraints and reasoning arguments. Novice problem solvers attempt to apply general, non-domain-specific methodologies. Typically, they attempt to draw upon general experience and apply processes of logic and deduction and are constrained by the “facts and evidence.” Domain experts develop their representations in problem solving by adding a lot of domain-specific constraints to their representation of the problem. In addition, experts present extensive arguments in the form of domain specific reasoning and structures in the problem solving process.

There continues to be evidence that the strategy in problem solving varies from experts to novices. Novices typically deal effectively with what are termed “well structured” problems or decisions.

An example of a well-structured problem/decision would be “our second baseman was injured last night.” This decision is a relatively simple and well-structured one: the need to obtain another second baseman. And if there is a competent potential replacement within the organization, the problem of obtaining a replacement becomes a relatively straightforward one to solve. If no replacement is available, the problem becomes more complex.

Ill-structured problems, such as your first-round draft choice performing below expectations, do lend themselves to strategies employed in other domains such as mathematics. But expert problem solvers are able to draw on an extensive reservoir of past experiences solving analogous problems in the same domain and can switch between various methods and strategies. Such strategies may not be as clear cut as those in mathematics, but domain experts may retrieve in a clear stepwise manner the domain-related procedures form their knowledge and experience (their schema system ) in problem solving.

On the other hand, novices follow either a trivial and or rather confusing path in executing the solution. In the case of the first-round draft choice not performing well, the rookie ball coach, based on what he sees, perceives the problem as one of mechanics and therefore attempts to solve what he perceives as the problem by a change in the players mechanics.

Whereas the minor-league coordinator engages the player in conversation and in so doing learns that there is an injury that the player for reasons having to do with team policy violations did not made known. Or after watching the player perform and his discussions with the player, the minor-league coordinator diagnoses of the problem as simply a matter of a lack of experience with high-level competition and advises the manager and pitching coach to employ this player in situations that are less demanding yet allow that player to gain experience.

The thought process of an expert is one of seeing the bigger picture (domain expertise) based on his knowledge and experience, which allows him/her to make decisions more rapidly (autonomous symbolic manipulation based on past experiences) and with greater probability of success.

The $64 question: exactly what is the domain of player selection and development expertise?

Selecting and developing MLB players requires understanding of what constitutes high-level baseball performance. It also requires an understanding of how the human body creates high-level baseball performance. This understanding and its application are what “Beyond Moneyball; Player Development, the Science of Creating the Unfair Advantage” uses as the definition for the domain of player selection and development expertise. To the baseball veteran, these may sound somewhat trivial. But as with most things, beauty is in the eye of the beholder and no two people are capable of seeing the same thing the same way.

Next time: Player development an exercise in problem solving.