So Teams Have Lots of New Data. Then What?

The Pirates are trying to integrate analytics into their philosophy. (via Keith Allison)

The Pirates are trying to integrate analytics into their philosophy. (via Keith Allison)

Over the past few years, every major league team has moved into analytics—trying to use sophisticated data and analysis to improve the organization’s performance. But that’s just a start. The question is how effectively this tool is used.

In part one yesterday, we outlined the four foundations of a data-driven organization, and looked at two of them: having the right metrics and insights, and having the right data and systems. But the numbers aren’t the whole story. Teams can’t use them to full effect unless they have the right people and the right organizational culture in place.

Right People

A team might have a handle on the best metrics and impeccable data and systems, but that still doesn’t mean it will maximize its return from analytics. Somebody still has to do something with all the information. Data can’t act on their own. This is where people come in, and they can make a huge difference when it comes to the advantage teams hope to achieve.

The people side of analytics can play itself out in a number of ways. The place to start would be with the analytics staffers themselves.

The variation in the talent of analytics staffers goes far beyond programming or statistical skills. The ability to build relationships and communicate insights to different members of the organization is huge. This is just as true in baseball as it is in any other industry I’ve worked in.

It’s hard enough to change someone’s behavior. That goes for players who may have developed certain tendencies over the course of 20 years playing the game, or for decision makers in the front office who have their own heuristics and experience to draw on when deciding what players to draft, sign, trade or release. Accepting information that confirms what you already know or believe to be true is easy; what’s harder is to buy into information that differs from what you believe. While there is no foolproof approach to changing someone’s mind, it helps to have the message delivered by someone you admire, respect, or at least have a strong personal relationship with.

(There is a ton of literature on the process of influence and change. A good place to start is Robert Cialdini, Influence: The Psychology of Persuasion.)

There is naturally some separation between those in the analytics department and those they are trying to influence, but that separation doesn’t need to be permanent. Some teams are even creating a hybrid position that bridges the gap between the analytics staff and coaches. This is done by placing someone with prior playing or coaching experience in the role of translating the findings from the analytics staff so that it is properly understood and acted on by the on-field managers and coaches.

Another strategy is to get analysts and coaches more exposure to each other. Ben Lindbergh’s profile of the Pirates shows that the relationship-building Mike Fitzgerald has embarked on has allowed them to better integrate analytics into what they do. He has spent an enormous amount of time on the road with the team.

Fitzgerald…makes most road trips: If the Pirates are playing, he’s almost always at the park. I surveyed several analysts from other front offices, and none of them knew—or would admit to knowing—of another employee with Fitzgerald’s statistical expertise who travels close to full-time with a team.

“It’s taken a long time to be at the place where we’re at now, where the relationships are there, and they trust what we give them and we trust what they’re saying,” (Dan) Fox adds. “Everything has jelled a little better over time. So maybe the competitive advantage is that it takes time to build it and once it’s there, you need to be able to sustain it.”

There are many advantages to this, which Lindbergh notes—better feedback loop, faster communication on what questions the team needs answers to, knowledge transfer, etc.—but none of this would be possible if Fitzgerald didn’t have a talent for relating to and building relationships with people. Additionally, Fitzgerald learned some great lessons about how to best communicate data to the coaches as a result, which we discussed yesterday.

Communication is key, and while I don’t want to overestimate its importance I don’t want to underestimate it, either. John Foreman, the chief data scientist at MailChimp and a noted author and speaker on the topic of data science, has noted that one thing the analytics profession needs more of is translators:

[W]ithout the ability to communicate, it becomes difficult to understand others’ challenges, articulate what’s possible, and explain the work you’re doing. . . .

Take any opportunity you can to speak with others about analytics, formally and informally. Find ways to discuss with others in your workplace what they do, what you do, and ways you might collaborate. Speak with others at local meet-ups about what you do. Find ways to articulate analytics concepts within your particular business context.

Push your management to involve you in planning and business development discussions. Too often the analytics professional is approached with a project only after that project has been scoped, but your knowledge of the techniques and data available makes you indispensable in early planning.

Push to be viewed as a person worth talking to and not as an extension of some number-crunching machine that problems are thrown at from a distance. The more embedded and communicative an analyst is within an organization, the more effective he or she is.

I’m not suggesting that talented analytics staffers who have robust backgrounds in statistics, research methods, programming, baseball and solid communication and relationship building skills grow on trees; neither do five-tool players. The point is that if you are looking to gain greater value from analytics you need to set the talent bar high when building your analytics staff, and teams need to make sure they are looking at a basket of skills, not just one or two.

You also have to have willing and able partners for the analytics staff. That includes front office personnel, managers and coaches.

There is only so much you can do to convince someone that there is a better way to do something, whether it’s an approach at the plate, how to throw to certain hitters, or how to coach a player. At a certain point, your efforts need to have a willing partner on the other side of the conversation. That’s why it’s so important to find managers and coaches who can add value in an analytically driven organization.

The coaching staff not only has to be willing to listen to new ideas from the analytics team, but also have enough knowledge and experience to be able to interpret the information (there is only so much you can simplify certain concepts and insights). Those leaders must be able to take those insights and make them work out of the “lab” and on the field. And, like analytics staffers, they must have the right communication and relationship-building talent to make players understand that they need to do something different to improve their performance.

This is a lot to ask, but where you see these types of people you tend to see teams that are getting more out of analytics. Gabe Kapler wrote last year about the importance of communication, and how managers and coaches are that key link in translating between the front office and the players:

I always envisioned myself as a three-hole hitter until my weaknesses were exploited at the major-league level. When I went to the Rays, Joe Maddon hit me everywhere from first to ninth. I remember the days I hit in the middle of the lineup. I was flattered and, frankly, surprised. But after a few months of comfort in the organization, I felt more certainty about where I was slotted. I knew that wherever I was in the lineup, I’d be in a position to succeed. I understood that my perceived weaknesses were negated by solid match-ups on a day that I hit fourth. I knew that if I was called upon to pinch run or play D, it was because I gave the club the best chance to win that game. It didn’t happen overnight. I trusted our data and I trusted Joe Maddon. He’d earned said trust, because he and others spent the time explaining the organization’s philosophy to me when I was “toasty” (as Joe called it) and when I was worthless.

The trust was invaluable when they came to propose tweaks to my game. They showed me evidence that I hammered the pitch down in the zone and scuffled on the pitch up. I assumed the opposite throughout my entire career. The delivery of the information was not in one dose, but sooner rather than later, I began to recognize the validity of this information. I tried to lay off the pitch up more often. It was effective, not always, but enough to make a difference.

You can argue that, during Maddon’s tenure, the Rays ranked near the top when it came to talented analytics and coaching staff that had both communication and relationship skills. When the Rays decided to try different strategies—whether it was where they hit players in the lineup, how they shifted their defense, when they used certain players out of the bullpen—the front office wasn’t simply making proclamations and forcing the coaching staff and players to comply. Instead, the coaching staff and front office were well aligned and the latter allowed the former to use the fantastic relationships they had built with their players to introduce the ideas and convince them of their merit. This alignment isn’t a given.

When Jonah Keri interviewed Billy Beane last year, one of Beane’s quotes wonderfully sums up the different ways people make a difference in analytics for a club:

This is really still just data,” he said. “And it’s all about what you do with the raw data. In every athletic endeavor, there’s so much data thrown at us right now that, ultimately, it’s the process and what you do with it that matters.”

While Beane talked up the importance of making something useful out of data—“that’s maybe 30 percent of this game”—he put even greater emphasis on implementation, specifically: finding a manager who’s open to new ideas; hiring scouts and number crunchers who can work together instead of against each other; and recruiting quants who know how to explain data in plain English so that the manager and other field personnel can easily put that information into play.

So a team is in good shape now that it’s got the right metrics, data, systems, and people, right? Well, not quite.

Right Organizational Culture

The final piece of the puzzle, and probably the most difficult to build or manipulate, is organizational culture. There is no shortage of definitions for the concept of culture. Let’s assume for our purposes that culture is simply the way that an organization gets things done. This is very much a functional or instrumental view of culture and one I’ve borrowed from my work at Gallup. (Gallup’s model is broader than just culture, but culture makes up a significant part of an organization’s identity.)

Every organization has a purpose, something it is trying to achieve and its reason for existing. In professional sports, purpose is pretty similar for each team—win as many games as possible on the way to a championship. Now, there may be some variation where revenue and profitability are on equal footing, but at the end of the day teams exist to win games. Regardless, each organization will have its unique culture that tells its members how things are to be done, how they should pursue the organization’s purpose.

You can find culture in a number of places within an organization, and leadership plays a pretty heavy role in determining the kind of culture that an organization or team will have. Leaders set the vision for the team and decide as what kind of people to bring into the organization, what gets valued and celebrated, how work teams are structured (i.e., what does the decision making structure look like) and how people are evaluated.

If culture is about restraining or funneling the behavioral choices of people, you can see how the list of areas above serves that purpose. Everyone who works for a team—whether it’s a member of the front office, minor league coaches, the major league manager, or even players—consciously and unconsciously makes choices every day about how to do their jobs. Culture pushes them to do things a certain way, or at least makes certain ways of doing their job more likely than others.

The importance of culture cannot be undersold when it comes to how a team, or any organization, uses and gets value from analytics.

Say a team has a top-flight analytics staff that has the best metrics, data and systems for communicating its insights. Let’s even say the managers and coaches understand data and are open to using it. What’s to stop that team from getting a big return? Well, plenty. At the end of the day, those managers and coaches need to decide what information to use and what advice to provide players, who to start and who to sit. If they make decisions that align with the data, but run counter to the wishes of the general manager or his assistants, they may find themselves on the hot seat, especially if those decisions don’t appear to work out. The front office may have its own preferences about how the team should operate, and if that doesn’t include following the analytics and metrics from the analytics staff that creates a disincentive for decision makers to use the information to its fullest potential.

This dynamic can also play in reverse, when an existing culture makes it difficult for a new leader to implement ideas and get people to operate in a different way. Case in point, the Houston Astros.

Astros general manager Jeff Luhnow has admitted, that changing the culture of a team is an incredibly difficult task, one that he originally thought was simply a function of the leadership. At last year’s Saber Seminar, Luhnow discussed how in his Cardinals days he met resistance to some new ideas. He and saberist and consultant Mitchel Lichtman were trying to convince Tony La Russa to use his relief pitchers in a way that aligned to the leverage of the game situation, not simply the inning (e.g., your best reliever doesn’t have to wait until the ninth inning to come into the game). That was just one example where Luhnow experienced difficulty in getting individuals to change how they did their jobs.

After arriving in Houston, Luhnow thought change would be easier. He soon realized that wouldn’t be the case:

Once I got to Houston, I thought, ‘Well, OK, I’m the general manager now, I can do what I want,’” Luhnow said, semi-seriously.

However, he soon realized that a promotion wouldn’t make the task any easier. “Our staff wasn’t really configured to [implement the shift],” Luhnow said. “There was not a lot of acceptance. We didn’t have the tools built to really sell it well, and so after a few conversations with the major league staff, and expressing a desire to go down this path, we really didn’t do a whole lot.”

“The complaints started to come from the pitchers, from some of the infielders, from the media, from basically anybody out there, and sure enough, as the season wore on, we found more and more reasons not to do the shift.”

“The biggest challenge that I think the 30 clubs face in digesting all this information and utilizing it is, ‘How do you implement change in an organization where the industry has essentially been a little bit resistant to change?’”

Certainly, leadership plays a large role in what the culture of an organization will be, but culture isn’t just top-down, it’s also bottom-up. In some cases, culture can be changed through “marketing,” as Luhnow discussed, campaigning for a new way of doing things through existing personnel. It can also be altered by literally changing the people in the organization, swapping those who think and behave one way for those who align better to the desired culture. Sometimes, people and the degree to which they can work together well simply won’t be able to align. That’s especially the case regarding the value of using data-driven insights versus existing beliefs and experience.

You can also impact culture by changing the incentive structure. For example, how performance is judged can and should have a decision-making component: What information did you use to make a decision—data (and, if so, what data?), experience and/or heuristics? How did you evaluate different or opposing views—whether the decision “worked” or not, did it have a high predicted chance of success? These are questions that should be asked when evaluating front office personnel, managers and coaches, and the decisions they make throughout a season.

Establishing a decision-making framework and then holding people accountable for how they make decisions, not just whether those decisions worked, is a powerful way to ensure that data and analytics are not just used, but used properly by everyone who works for a team.

Wrapping Up

As long as these articles have been, they simply scratch the surface. All the areas discussed—metrics, systems, people, culture—are incredibly complex topics in their own right. Aligning an organization, or a major league team, around any new idea is difficult. However, if it isn’t done—if you don’t take the time to audit and correct each of these areas—you can’t expect to get the most out of any new strategy or resource, and that includes analytics.

Obviously, some teams have recognized the importance of these areas and done some of the necessary work to better align themselves. Some of the best aligned organizations (from what I can tell from press reports, hearing individuals speak at conferences, and talking to those in the industry) not surprisingly have had some of the most successful runs in recent years. That being said, even those teams will admit that they have gaps, that they have areas (whether people, culture, or something else) where they need to improve. And none of them think it to be easy.

There is certainly an advantage to beefing up a team’s analytics capability, but it takes more than just hiring an analytics staff. Teams must understand all the mechanisms that are needed to see a return from analytics, and be willing to make the necessary investments in each area if they want to gain an advantage. Given how hard it is to make changes in each of these areas, I would get started right away. Otherwise, teams may find themselves at a much larger disadvantage in just a few years compared to teams that have already started down this path.

References & Resources


Bill leads Predictive Modeling and Data Science consulting at Gallup. In his free time, he writes for The Hardball Times, speaks about baseball research and analytics, has consulted for a Major League Baseball team, and has appeared on MLB Network's Clubhouse Confidential as well as several MLB-produced documentaries. He is also the creator of the baseballr package for the R programming language. Along with Jeff Zimmerman, he won the 2013 SABR Analytics Research Award for Contemporary Analysis. Follow him on Twitter @BillPetti.
newest oldest most voted
Chris Mitchell
Member

Really enjoyed these articles. Are there any specific examples of current teams that have great analytics departments, but for whatever reason, fail to parlay it into good decision making?

Calvin Liu
Guest
Calvin Liu

Interesting work, but I wonder just how much “Big Data” just means a herd mentality via other means.
In the past, the herd mentality would be scrutinizing the public (and private, via player moves) of successful teams/managers.
Now, the same thing can be done only via numbers.