What Sports Can Teach Us About Analytics: The MIT Sloan Conference

Last weekend’s MIT Sports Analytics conference was, pretty easily, the best conference I’ve attended. And I attend a lot of conferences. With a 9:00 AM session on Baseball Analytics featuring one former Red Sox General Manager and the club’s current Director of Baseball Information Services, obviously it had a potentially unfair advantage over the competition, what with my unhealthy obsession with that sport. But it’s a common misperception that all there is to a conference are the speakers or, in this case, panelists. The reality is that there’s a lot more to putting on a good conference. Venue matters. Session topics matter. Session placement matters. Sponsors matter (the Bloomberg lunch demo was unreal). For some people, things like the food probably matter.

Happily, the folks from Sloan could not, in my view, have done a better job with the show. It was professional, it was tightly executed, and it was so compelling that I either stood (a few sessions were standing room only) or sat in a chair for basically the entire day. No breaks, no calls, nothing. It was like a Pedro Martinez start, circa 2000: you didn’t dare get up for fear of missing something important. How many shows are you going to attend that feature a panel of ESPN columnist Bill Simmons, Indiapolis Colts President Bill Polian, Houston Rockets GM Daryl Morey, Kraft Group (owners of the New England Patriots) President and Williams alum Jonathan Kraft and Dallas Maverick’s owner and technology entrepreneur Mark Cuban, moderated by Michael Lewis, debating Moneyball, the latter’s bestselling book?

Not too many, I should think. So kudos to the folks from Sloan: I cannot recommend their show highly enough.

What was particularly interesting for me as a technologist, as opposed to a baseball fan, was the fact that on some level, the subject matter was incidental. What we were there to talk about was how to collect and use data to make more informed decisions; that the context happened to sports was interesting, but hardly unique. Analytics usage in sports has accelerated as salaries and payrolls have escalated. When it’s time to sign free agents to guaranteed contracts of tens of millions of dollars, it behooves the club to make the best decision it can. How? By using the data it can collect, obtain or derive.

As they say when a popular player is traded or not resigned: baseball is a business. A different business than, say, heavy industry manufacturing or pharmaceutical research, yes, but when it comes to using data to make better decisions, business is business.

Here are ten lessons, then, I think traditional businesses might learn about analytics from their counterparts in sport:

Culture as an Obstacle

Simon Wilson, the Head of Performance Analysis from Manchester City: “We’re especially jealous when we come over and look at the culture of using data in sport.” Ironically, American sport is perhaps the best illustration of the challenges that culture can present. <a href="Bill James started publishing the Bill James abstract in 1977. Thirty-plus years later there are still clubs that regard the conventional wisdom that James dared challenge as sacrosanct and infallible.

The lesson? No matter how good your data, a portion of the poputation will not accept it. If you want to drive analytics into your industry, be prepared to fight an uphill battle. The bad news is that this can make life difficult for analytics converts and evangelists. The good news is that it can be an opportunity.

Look For an Edge

While there are curious exceptions – the NFL, for example, apparently forbids technology of any kind (even a calculator) in the coaches box – for the most part sport allows teams to compete off the field as well as on. If the culture is anti-analytics, then, this can potentially be a good thing: it gives you an edge.

John Abbamondi, the Assistant General Manager of the St. Louis Cardinals, acknowledged this when talking about FieldF/X, a new system being put in place to provide significantly better metrics for measuring defensive performance. His concern? “One of the things I worry about is that it’ll make measuring defense too easy.” If everyone has access to the same excellent metrics, in other words, there’s very little opportunity to gain a competitive edge.

The lesson? When looking for an edge, don’t look to areas that are commoditized. Focus instead on areas where it’s difficult to measure. Even if you do it poorly, the odds are that you’ll still have better intelligence than your competitor who’s not looking there at all.

“Emotion Dooms Analytics”

Paraag Marathe, the San Francisco 49ers’ Executive Vice President of Football & Business Operations, said that, and he’s right. It’s very difficult to make good business decisions if you’re making them emotionally.

The lesson? Leave that to your competitors. Make the best decisions you can based on actual data. The Boston Red Sox have made some wrenching emotional decisions the past decade, after eighty some odd years of courting fan sentiment. The results? Two World Series titles.

Consider Context

Aaron Schatz, the Editor in Chief of Football Outsiders, discussed the draft valuations of SEC running backs versus Big 10 running backs. And while he can’t prove it yet, he has a working hypothesis which asserts that running backs from the SEC tend to be undervalued in the draft, while those from the Big 10 tend to be overvalued. Why? Because of context. Film is a huge component of the scouting process in the NFL, but it’s difficult to account for the relative differences in league, from average offensive and defensive line size and weight, offensive schemes, and more. Big Ten running backs seem to look better than they actually are; SEC backs, worse.

The lesson? Context matters a lot. Try to consider not just the data, but where it came from, and how it might potentially be biased. Then use the data to weight and adjust for those biases.

If You’re Not Using Analytics, Your Competitor Will

John Dewan, the Owner of Baseball Info Solutions, summed up the differences between baseball teams diplomatically: “Not every team appreciates the value of defense equally.” Those that do, the evidence suggests, have a significant advantage over those that don’t.

The lesson? If you’re not using analytics in all areas of your organization, you can be sure that your competitor will be. Which will be his advantage and your handicap.

It’s As Much About What Data You Don’t Present as What You Do

Just using publicly available data – forget all the extra proprietary information the clubs collect – I could tell you what the batting average is against Josh Beckett’s two seam fastball located in the bottom half of the zone in the third inning in day games at home against hitters in the bottom half of the order.

Does that actually help anyone? Probably not. The challenge, with sports as every other business we speak with, isn’t too little data, generally. It’s too much. The question becomes how you determine what to present and what not to.

The lesson? Don’t overwhelm with statistics. Work backwards from what you want to know, or might want to know: that will inform your choice of data.

Speak English

Indianapolis Colts president Bill Polian: “Speak English, please.” Bill Simmons, ESPN Columnist: “For stats to make it to the next level, they’ll have to be able to relate to everyone, not just people with Math degrees.” Statistics and analytical professionals, like people in a variety of specialized disciplines, can sometimes forget that not everyone speaks their language. If I told you that Jon Lester was a 5.6 WAR player last year, does that mean anything to you? It would if you’re a baseball nerd; for everyone else that’s just Greek. As Tom Tippet, the Red Sox Director of Baseball Infomation Services put it,”There are a lot of people in baseball operations that don’t have degrees in math or get how these work. The challenge is to make it usable.”

The lesson? Duh: speak English. Abstract the terminology where you can and attempt to explain in practical terms what your analytics actually mean. If you can’t communicate that, it really doesn’t matter how good your data is.

Making Unpopular Decisions

The question I asked the baseball analytics panel was, at its essence, pretty simple: how do you handle making unpopular decisions that are nevertheless the right decision to make, according to the data? Bill Belichick, for example, was widely excoriated by Patriots fans this past season for going for it on 4th down against the Colts and failing, ultimately losing the game. Why is this interesting? Because as Schatz discussed, the data said that was unquestionably the right decision to make. More, Bill Polian, the General Manager of the Colts – the team that benefitted from the failure – agreed.

The lesson? Some decisions will be less popular than others, invariably. The key is to keep the big picture firmly in view, because if you’re making the right decisions consistently, you should win. And if you win, everyone forgets the unpopular decisions.

Measure Everything

Because throwing a ball overhand is, in a biomechanical sense, an unnatural motion, pitchers – particularly young ones – are a serious injury risk. In an effort to keep them healthy, teams are increasingly employing a variety of statistical measures – both general and individualized – to build training and throwing programs designed to maximize their health. Key to this is data: having orthopedic data on stresses to the motion generally, to delivery types more specifically, and finally to an individual athlete. Observational surveys are being conducted which accumulate more and more data on who got hurt, when, and how, from which we can attempt to extract patterns of injury and thus identify potential risks.

The lesson? Measure absolutely everything you can. You will not be able to anticipate what you might need data on, so collecting as much as you can in advance is likely to be your hedge against such future needs.

The Challenge of Integration

Data is almost always more valuable than it is in a vacuum. The Red Sox, for example, have constructed essentially a single database for players – their own and other clubs – that incorporates just about anything you could want to know about a player. Scounting reports, performance data, video, contract status: everything. Because while it’s nice to know who the best hitting outfielders are, it’s even better to know – per their contract status – which ones are available and which ones aren’t.

The lesson? Look for opportunities to integrate all of your data. Not just for the convenience of a single repository, but because the sum of data is usually more than its component parts. Potentially far more.