Automatic Scouts: The Changing Landscape of Scouting

A common fear in many industries is the growing sense that “robots are stealing our jobs.” It’s a mixture of paranoia, anticipation, and a highly optimistic view of what might be possible; ultimately, the fear that automation and technology will entirely replace human intervention is rooted in the idea that technology is just that advanced.

Baseball isn’t immune from the idea that robots will supplant humans in certain jobs. The speculation (or wishful thinking) in the game focuses on the most visible way in which technology could eliminate the human element from baseball — replacing the human umpire behind the plate with the mythical automated strike zone. The idea is hardly new, and the #RobotUmps hashtag enjoys a renaissance with every controversial call on the field, though an adequate robotic replacement for the umpire has yet to come to fruition.

But if you’re truly worried about how robots and technology are encroaching upon the human element of the game, look a bit further behind the plate, past the protective netting. The small cohort of people sitting there with clipboards and radar guns is slowly diminishing. Quietly, front offices are cutting their scouting staffs and reorganizing those who remain, with the Astros being particularly aggressive. While baseball isn’t exactly replacing human scouts with robotic ones, some organizations are making strides in that direction, which has implications for the human element, not only in the form of the scouts themselves but also in the product (and people) on the field.

In a 2013 series about Scout School, Ben Lindbergh noted:

My guess is that assessing makeup will become an increasingly important part of scouts’ responsibilities as PITCHf/x, biomechanical analysis, and other applications of motion-tracking tech permeate the lower levels of the minors and bleed into amateur ball. Watching a player work and collecting character references isn’t something a computer can do, although refinements in psychological testing — not a new concept in baseball, but an increasingly popular one — may eventually offer an automated alternative. (Psychometrics could be the next frontier for sabermetrics.)”

Lindbergh’s comment about technology changing the role of scouts in baseball was prescient. There’s less emphasis on travel and game attendance for some teams, as scouts are called upon to confirm video findings and analytics. While many would (and do) recognize the value in an ongoing collaboration between scouts, video personnel, coaches, and analysts, others feel that the role of the scout is of lesser importance to the game. Certain teams appear to be embracing the idea that high speed cameras, Trackman, and Statcast can provide the same information as a human scout.

Using equipment that is essentially standardized across major league ballparks to supplant pro and advance scouting isn’t that farfetched, and can be an attractive method for teams looking to cut costs. Increasingly, similar technologies are becoming commonplace at both the college level and at high-profile amateur showcases. As a result, teams are able to gather more information about amateur players without having a scout present.

Although a high school or college field is unlikely to have a high speed camera setup on par with what you’d find at a minor league ballpark, there certainly has been increased usage of cameras on the field, which doesn’t even take into account the online dissemination and propagation of videos. Manufacturers of bat sensors and pitch recognition apps, like those from MLB-approved vendors such as Blast Motion and Diamond Kinetics, are marketing their products to amateur players, and have infiltrated youth and college baseball. Bat sensors can provide information regarding a player’s swing mechanics and hitting approach. If you need to hone your pitch recognition skills, Diamond Kinetics has an app for that, through its Axon Sports arm. And for an aspiring pitcher, a baseball embedded with pitch tracking technology can collect information about your pitching mechanics, not to mention your velocity.

Bat sensors and pitch-tracking baseballs can provide a wealth of information for the amateur player, enabling them to play the role of quantitative analyst using their own data. But as with all modern data collection, one must be cognizant of where the data is being stored, and with whom it is being shared. It’s not entirely clear who owns the   data or how it’s being shared, and it’s very possible that an eager player or unwitting parent is going to overlook the fine print of the terms of service.

The increasing use of technology marketed to amateur players could eventually eliminate an aspect of amateur scouting at the extreme lower levels. No longer will an organization have to rely on word of mouth, a bird dog scout, or a tip from a coach to see a budding young star—it will be able to shorten or skip entirely an initial evaluation, because the bat sensors, pitch trackers, and cameras are doing the initial scouting trip.

Some of these technologies are being integrated into the current scouting processs. Beyond the Axon Sports pitch recognition software, there’s also NeuroScouting, which purportedly “identif[ies] and develops the neural systems that are critical for elite baseball performance.” Teams are using pitch recognition software to train minor league players, but also as a quick gauge of a prospective draftee’s skills. Indeed, NeuroScouting’s website claims “NeuroScouting’s analytics are able to assist professional level sports teams in making informed decisions about potential drafting or signing prospects and young players,” suggesting that their technology shouldn’t be limited to use only after a player has joined an organization.

Again, while some organizations will appreciate an integrated approach — one that combines the use of technology and a scout’s observations in order to collect as much data as possible—- others have shown that they are willing to forgo the scout’s input, with the idea that technology can fully capture and quantify an amateur player’s baseball skill set.

So we have a few, evolving ways of collecting information and evaluating a player’s tools without a human laying eyes on a player. But what about intangibles, like makeup and character? Perhaps most importantly, how do we collect information if we haven’t agreed upon what we’re collecting? Ask scouts about intangibles, and for a definition of makeup, and although it is generally agreed that makeup is important, many acknowledge that it’s a hard concept to precisely describe. Some common themes arise, and in the author’s inquiries, scouts cited work ethic, resilience, competitive spirit, and learning ability as qualities they evaluate when determining a player’s makeup.

Learning ability includes a player’s reception to coaching and criticism, which can go hand-in-hand with work ethic. Resilience includes not only how a player overcomes adversity on the field, but can shape his recovery from an injury or derailment off the field. In some ways, it’s easier to think of makeup in terms of projecting how a player will react  in the face of a particular situation. If a coach provides constructive criticism, how will the player respond? Not only do we need to know if a player will be receptive to a new idea, but if he is willing to work toward making that new approach or pitch a reality. If a player undergoes surgery, will he adhere to his recovery regimen? Although the personal qualities involved are hard to define and difficult to correlate to an outcome, the potential situations are very real, as is their impact, and they can play a tremendous role in a player’s success or failures.

Makeup can sometimes be overlooked by analysts, because it’s hard to quantify. It’s inconsistent, because it’s highly subjective— there’s no standard metric. And there are ongoing debates as to its importance in a player’s climb to the majors. Because it’s so subjective and its ultimate impact on players’ careers is still unclear, it has been an attractive target for removal from a player’s scouting profile, as well as a goal for automation and technological interventions.

For some scouts, makeup and character come down to whether a player is a good person. We’ll never invent a scanner capable of reading a person’s heart and soul, but there have been no shortage of attempts to use technology to detect feelings and general demeanor.

Sentiment analysis is an automated way of analyzing the words and phrases of a body of text, particularly a social media account. Although a simple keyword search would benefit some players, sentiment analysis goes beyond a search for single words, and can assess a social media feed’s general positive or negative attitude based upon certain words and phrases. A recent study applied language analysis using Python to Facebook profiles and found it fairly accurate in confirming clinically diagnosed depression. Further, the analysis was able to predict a depressive state evidenced by a Facebook profile up to three months before medical documentation of depression. A simple modification in the code could change the focus of the language analysis from depressive phrases to sentiments that may be more indicative of a player’s makeup and character.

Automated methods of analyzing social media feeds may be useful, but they can only tell you so much when posts are taken out of context, and machine learning provides a biased lens. An aspiring draftee may offer his social media feeds up to an organization for scrutiny, but a savvy player will delete unsavory tweets — or, better yet, never tweet at all. However, while one would have to consent to having a private Facebook or Twitter feed examined, it’s possible a voice sample could be subject to the same language analysis. Voice recognition through Alexa can detect your overall physical and emotional state, and possibly if you’re depressed. Again, one could modify the word combinations such that one could evaluate a player’s confidence or competitive spirit. Given some of the odd metrics scouts sometimes use to judge a player’s character, it wouldn’t be  surprising if someone attempted to use the pitch and tenor of a player’s voice to gauge his makeup.

Faced with assessing so many amateur players, scouts do have some rough guesstimates they may use in evaluating a player’s body and build; it can be as simple as judging his body frame to the way he shakes hands as a gauge of his grip strength. It’s not really clear if this is any more valuable in assessing future performance than some other shorthand that certain scouts adhere to. For example, some scouts use religion as a proxy for character and morality, when there’s no basis in fact tying one’s religious affiliation with a propensity for “goodness.” This is about as helpful as a scout who contends that a prospect will never become a major leaguer because he has red hair.

That said, the role of scouts in projecting a player’s growth, both mentally and physically, and assessing makeup and character, is much more complex than these snap judgments. One scout likened it to acting as a private investigator, while another commented that there is an expectation that scouts play the role of psychologists.

The psychologist metaphor is apt; psychology and baseball have enjoyed a close relationship over the years. Harvey Dorfman was a pioneer in the use of mental conditioning to enhance a baseball player’s performance on the field, and organizations have continued to employ mental performance staff. This has benefits from a mental health perspective, but it’s important to note that the addition of psychologists to organizations is often for the benefit of training and performance, which isn’t limited to physicality. If you can train your body to maximize its potential, why not train your brain to maximize its potential as well?

Thus, it’s not surprising that in order to assess makeup, some organizations are administering psychological tests to evaluate a potential draftee’s personality. Personality tests permeate our lives everywhere, from Facebook to online dating websites to corporate America, where companies try to assess an employee’s character. Although there are many personality tests out there, the Myers-Briggs Type Indicator (MBTI) remains popular.

These personality tests, including MBTI, are flawed for a number of reasons — the development of the test itself means that biases are baked right in, and it’s notoriously easy to game a test so that you arrive at the result you desire. That doesn’t stop people from looking for more scientifically  validated personality tests, and it doesn’t stop team psychologists from administering written and telephonic evaluations with potential draftees. This provides an internal metric; however, these tests will also include a number of internal biases based upon the design and content of the test. Nevertheless, remotely administering a test on a laptop — especially with a webcam recording a player’s facial expressions —- remains an attractive and frugal option.

One could also argue that another tried-and-true method of gathering information about a player’s personality and makeup can also be conducted remotely — collecting feedback from coaches. Often one of a scout’s most important tools in evaluating makeup and character is word of mouth, from coaches and teammates. While you would hope that respondents would provide an honest assessment of a player, there’s always a concern that someone might gloss over some unsavory aspects of a player’s character, essentially a “pass the trash” move to remove an undesirable person from the clubhouse. Sorting through those accounts, and squaring them with their own assessments, is an important part of what scouts do as they evaluate a player’s character.

Clearly, although there are a number of ways one can undertake an assessment of makeup without a human presence at the ballpark, there’s no surefire way to measure it. Although the makeup of a player who has spent time with an organization will be more of a known quantity, for amateur players there are a lot of unknown qualities. As one scout noted, this is where assessing intangibles and makeup can have the most impact.

Amateur scouts watching a game not only evaluate a player’s baseball skills, but also endeavor to assess his character. They are watching how a pitcher reacts after a hitter hits a home run off  his supposed best pitch. They are watching how the player interacts with his teammates in the dugout. They may even be watching how the player interacts with people off the field.

We talk about projections, and trying to project how a high school kid will grow into an adult body and develop into a baseball player, but a scout’s projections also need to consider how that kid will emotionally mature into an adult. As one scout noted, “Most scouts will tell you that evaluating makeup and projecting it forward is the hardest part of their jobs— determining which immature 18-year-old will grow up and which one will stay childish into his 20s is as difficult as it is critical.”

Here’s where the family visit comes in. For younger athletes, a family visit can be invaluable to a scout. It can provide fodder for attempts to project body type, as a scout mentally notes the player’s parents’ physicality (although using parental phenotypes to project an offspring’s future development is highly speculative). Learning about the family’s athletic record could be useful — an athletic family doesn’t just reveal information about a player’s physicality and athleticism, but can suggest a tolerance for an extreme training regimen, and perhaps the emotional and financial wherewithal to succeed in the minor leagues.

Even if the minutiae don’t make it into the scouting report, it’s human nature to pick up on small details and interactions. For example, during the family visit, a scout is reading body language and interactions among the player and the family members. If the scout asks a question, who answers? Does the player defer to his parents, or is he more outspoken? Judging not only the answers to the questions, but the manner in which they are answered, can provide a bit of insight into a player’s makeup.

Conducting this amount of in-person investigation for every possible amateur player is a gargantuan task. Projecting the baseball-playing potential of just a handful of players is daunting, but given the number of amateur players who would need to be evaluated for character, it can be overwhelming. A family visit may take up to a hour, not counting  travel time, the time spent writing up the visit, or the time spent coordinating the visit. Add to this the time spent meeting with coaches and teammates, and the time actually spent watching a game, and the financial cost of hotels and meals, and it’s easy to see why teams would seek to cut back on this level of assessment.

As one scout noted, conducting a thorough investigation of a player’s makeup and character is akin to serving as a detective, but this level of detail may be invaluable in avoiding surprises in off-field activity. A scout noted that past incidents may foretell future off-field incidents, and an additional read on a player’s character may distinguish certain behaviors as youthful indiscretions rather than a fatal flaw in makeup.

In the future, teams may attempt to use polygenics to aid in projection. Indeed, DNA testing services like 23andMe allege to use genetics to determine your body weight, muscle composition, and ring finger to index finger ratio. But this is still a nascent science which has yet to prove itself, and may not be any more accurate in projecting a player’s growth and skill set than a scout’s eyes. It also introduces a host of thorny privacy concerns for players. Again, a complementary approach taking in all available data will lead to a more comprehensive profile of the player as he is now, and the player he may become.

So what does this all mean? It’s inaccurate to say that the role of scouts in identifying talent is being made redundant by new technology. Instead, the role of the scout will evolve. Rather than technology supplanting the role of scouts in many organizations, thereby diminishing the human element, perhaps the scout’s own skill set will evolve to complement the technology, just as the technology will evolve to allow scouts to spend more of their time on those aspects of a player’s game and character that cannot be assessed using technology.

Let’s turn back to Lindbergh’s account of scout school:

According to Larson, 90 percent of prospects fail to reach their [Overall Future Projections]. Most often, he says, it’s because of intangibles. It’s much more common for a player’s makeup to prevent him from making the majors than for it to propel him there.

If 90 percent of prospects fail to reach their future projections, a team would be wise to place additional emphasis on exploring the reasons for the discrepancy between projections and reality; the expense can be quite real, after all. Consider the amount of money involved in the bonus pools and slot values of the first 10 rounds of the draft. A player who is drafted in the first few rounds, where there is a significant signing bonus involved, will have several sets of eyes from a team evaluate him, and there will be many in-person meetings. A bit further down, we see that a fifth- round pick is slotted at between $300,000 to $400,000. Even a 10th-round pick is slotted between $136,800 to $143,300.

Bear in mind that these are just signing bonuses; we also have to consider a team’s expenditures in terms of the costs of developing a baseball player. (This certainly does not excuse organizations from underpaying their minor leaguers, the abysmal wages that have been well-documented.) There is increased attention — and increased spending — in player development, which is seen as an obvious way for teams to differentiate their organizations. Eno Sarris notes, “When it comes to improving the outcomes for their own players, by investing in hardware, software, R&D personnel, and coaching staff, teams are willing to bite down hard.” If teams are willing to go to such lengths to ensure the best possible outcome for an untested player, why not also invest in finding a quality person to begin with?

Many extremely talented players who are selected in the early rounds never reach superstardom, but have perfectly adequate playing careers. And there are players who seemingly have great potential who never make it to the big leagues. When a promising player doesn’t accumulate major league service time, we sometimes attribute this to deficiencies in makeup, character, off-field activities, even the company a player keeps. Selecting the wrong player, even in the middle rounds of the draft, means you missed out on someone else. It could be the difference among  someone who gets cut for breaking curfew during instructional league, a fringy org player, or even a future Hall of Fame candidate.

We may not be able to easily quantify makeup and character, but it’s foolish to dismiss it as unimportant. There may not be the same precision in scouting that there is in a stat, but that doesn’t mean that scouts’ processes aren’t rigorous, or evaluations important. As ESPN Senior Writer Keith Law notes, “We so often ascribe the unexpected failure/success of a player to makeup — he didn’t work hard enough, he couldn’t make adjustments, he just refused to fail, he willed himself to the majors — yet we’re apparently considering eliminating the best avenue to get that data on players before the point of success or failure.”

Given the potentially significant financial investment made in a player before the point of success or failure — everything from a signing bonus to the resources spent on developing a player — a relatively small expenditure in evaluating makeup and character may direct an organization to better identify the players worth that investment. And it may even help avoid a situation wherein off-field activities would keep a player — and a team — from performing at their best.

References and Resources

• Ben Lindbergh, “Welcome to Baseball Scout School”, Part 1 and Part 4, Grantland, October 2013
• Janie McCauley, “As baseball evolves, advance scouting largely turns to video,” Associated Press, June 29, 2018
• Keith Law, “Chapter 16: No Trouble with the Curve: How Scouting Works, and How the Statistical Revolution Is Changing It,” Smart Baseball, 2017
• Eric Longenhagen, “The State of Scouting,” The Hardball Times Annual 2018
• Jared Wyllys, “Eight Minutes and a Laptop: Donnie Dewees One of Many Prospects Reaping the Benefits of Neuroscience,” 2080 Baseball, February 16, 2017