Tactics: How to Optimize the Use of Analytics in Hockey
All models are wrong, but some are useful
Statistical modelling, in its essence, is an imperfect science. By extension, hockey analytics will never be perfect.
Over time, statistical models have become more accurate in their attempt to evaluate player contributions. This is, in part, thanks to the increase in the granularity and quality of data combined with the advancement of statistical methodology.
Through data-driven insights, these experts can help teams gain competitive advantage as combining data with video analysis is a way to optimize the decision-making process in modern hockey. Stats provide information regarding the events that occur on the ice, while video analysis provides information on why these events happen.
As teams rely more and more on analysts to help gather more precise insights on the game, here are a few elements that differentiate great analysts from good ones, in my experience.
Turning Data into Actionable Insights
In its simplest form, data is information. Good analysts find ways to present all the information in a concise manner to help coaches/GMs with their decisions.
However, great analysts are able to use this information to provide actionable insights. Whether it be in the form of data visualizations, charts, graphs or even within informal discussions, actionable insights broadly encompass information that coaches care about.
Coaches/GMs might directly ask analysts for actionable insights in the form of specific stats. But great analysts are also able to anticipate the needs of coaches/GMs and come up with stats that coaches might not even know the existence of.
The ability to anticipate the needs of coaches/GMs isn’t instantaneous. With experience and through the development of a working relationship with a coach/GM, great analysts often gain an excellent understanding of how their coach/GM likes to operate and leverage analytics.
And from there, they can find ways to introduce new analytical ideas to enhance the presentation of actionable insights.
Communication is the Key
Good analysts can do the math well enough to drive actionable insights. But what differentiates great analysts from good ones is the way they communicate these insights.
Traditional hockey people communicate in hockey terms. As such, when presenting complex statistical models to them, simplicity and understandability are key.
Different coaches/GMs might care about different aspects of a statistical model. Some might solely care about the insights that can be gathered from the model, while others might also want to understand how the model is built.
While tailoring explanations is important, in both cases, it is crucial to keep it simple. The last thing you want is for your coach/GM to be confused when you present a new concept.
For instance, when presenting expected goals to a coach/GM, talking about shot quality based on distance to the net, angle and contextual factors (strength, score,…) is a concise way of introducing this concept.
Moreover, great analysts are able to incorporate the explanation of models within their analysis. Whether it be thanks to use cases or skillful storytelling, illustrating abstract concepts with concrete examples allows coaches/GMs to link these models more easily to their hockey knowledge.
All models are wrong, but some are useful
This quote from British statistician, George Box, summarizes statistical modelling and analysis very well.
No hockey analyst currently has or will ever have the “perfect solution”. So ultimately, the goal of every analyst should be to build useful models and be wrong about hockey less often.
What differentiates great analysts from good ones is the understanding of modelling limitations. Since statistical models are never going to be perfect, great analysts actively seek to understand how their models are wrong.
This allows great analysts to continuously improve their models and develop more and more accurate tools to provide actionable insights.
As they gain a better understanding of model imperfections, they also incorporate nuances in their analysis. This way, they ensure that the actionable insights which are provided not only follow the math, but also consider its flaws.
In line with that and with the Women’s World Championship starting tomorrow, we will be performing SARAH model updates to evaluate the habits of WHKY players more precisely.
I will outline the model updates and the limitations of the first iteration of the SARAH models in a newsletter post, in the coming weeks.
If you are a coach who wants to learn more about how to leverage hockey analytics efficiently at any level, Jack Han & I recently released a course that may interest you.
In this 2-hour course, split into 9 different chapters, Jack & I share our experiences working with different organizations worldwide and discuss best practices when starting to use analytics in coaching.