Warning
Before we dig into this one, fair warning – this piece is technical and all about methodology, and is mostly concerned with how to get somewhere – rather than what that somewhere actually is. If you want to skip it and wait for the results, they will follow in the next piece. But, if you’re a big nerd like me (or you want to find where I stuffed up and have underrated your favourite league), read on.
This is part two of a three-part series. You can catch up on part one by clicking here.
Approach
Data collection
To compare from one league to another, the simplest thing to do is to have a player who has played in one league, then moved to another the following season (or within that season) and compare their performance across leagues. The league they score fewer points in is considered harder to score in. Given an overall lack of creativity, this analysis relies on the exact same logic – though it expands the sample past just one player. To compare league to league, we take the performance of players in one league (measured by their point-per-game ratio (P/G)) and compare it across the leagues. The league where the sample of players have a lower P/G is viewed as a tougher league to score in.
So that this can have some element of robustness to it, and not be reliant on a handful of players who had something weird happen, I use a few criteria. Any proper statisticians reading this may be about to have kittens, but the practicalities here mean I needed to take some liberties – or I’d end up like Jim Carrey in the Number 23, but with Elite Prospects data[i]. For a transaction (a movement by a player from one league to another) to be included, the following must be the case:
- At least 10 games played in each league.
- In the same or immediately adjacent season (one before or after) – though 2020-21 was allowed to be vacant due to the pandemic.
- Within the seasons from 2013-14 to 2022-23 (inclusive)
Now I have collected the transactions, I can make comparisons. However, not all leagues have many (or any) players making direct transactions between them – that is, moving directly from one league to a particular other league. For example, we don’t see players move from the Australian Ice Hockey League (AIHL) to the American Hockey League (AHL) directly. So I need to determine a pathway from all leagues to a ‘comparison league’ – using interim steps. The interim steps form a pathway from all leagues to the comparison league (see Table 2).
The comparison league allows us to make an apples and apples comparison. For the comparison league, the ECHL (formerly called the East Coast Hockey League in the US and Canada) has been used – and this is mostly arbitrary. You could use other leagues, provided you can find enough pathways to that league. The ECHL presented a useful case as it was a large enough mid-tier pro league that other leagues weren’t “too far” from it. So, that leads to pathways.
Pathways
However, “too far” needs to be defined to have any use at all. In this case, I used pathways from one league to another to get the comparison back to an ‘ECHL equivalent’. With each leg of the pathway, the comparison gets weaker (as each leg has an amount of uncertainty associated with it – so the more legs, the more uncertainty adds up), so I kept these to a minimum. Wherever direct comparisons were possible, I used these (14 leagues – e.g. from the Elite Ice Hockey League [EIHL] in the UK to the ECHL), with no more than two interim legs used.
23 leagues used one interim leg (e.g. from the Alps Hockey League [Austria/Italy/Slovenia] to the International Central European Hockey League [ICEHL – Austria/Hungary/Italy/Slovenia] to the ECHL) and seven leagues used the maximum of two leagues between (7 leagues – e.g. from Vysshaya Hokkeinaya Liga [VHL – Russia] to Kontinental Hockey League [KHL – Russia/Kazakhstan/Belarus/China] to AHL to ECHL). Table 2 sets out the steps for each league. To ensure there is something resembling a statistically significant sample, each leg has at least 30 transactions – where possible. There are three pathways which are exceptions to this as there weren’t enough transactions to reach that level:
- Allan Cup Hockey (Canada) and USports (Canada – university) (23 transactions)
- Chinook Hockey League (Canada) and USports (20 transactions)
- Asia League (Japan and South Korea) to ICEHL (13 transactions)
Table 2: Pathways from each league to the ECHL
| League | Country | Leg 1 | Leg 2 | ECHL |
| 1. Liga (Cze) | CZE | Extraliga (SVK) | ECHL | |
| 1. Liga (Svk) | SVK | Extraliga (SVK) | ECHL | |
| 2. Liga (Cze) | CZE | 1. Liga (Cze) | Extraliga (SVK) | ECHL |
| Australian Ice Hockey League (AIHL) | AUS | ECHL | ||
| Allan Cup | CAN | Usports | ECHL | |
| Allsvenskan (Swe) | SWE | ECHL | ||
| Alps Hockey League | AUT/ITA/ SLO | ICEHL | ECHL | |
| American Hockey League (AHL) | US/CAN | ECHL | ||
| Asia League Ice Hockey | JAP/ROK | ICEHL | ECHL | |
| BeNeLiga | BEL/NED | Oberliga | DEL 2 | ECHL |
| Chinook HL | CAN | Usports | ECHL | |
| Deutsche Eishockey Liga 2 (DEL 2) | GER | ECHL | ||
| Deutsche Eishockey Liga (DEL) | GER | ICEHL | ECHL | |
| Division 2 (Swe) | SWE | HockeyEttan | Allsvenskan (Swe) | ECHL |
| ECHL | US/CAN | ECHL | ||
| Elite Ice Hockey League (EIHL) | UK | ECHL | ||
| EliteHockey Ligaen (Nor) | NOR | ECHL | ||
| Erste Liga | HUN/ROU | ECHL | ||
| Extraliga (CZE) | CZE | ICEHL | ECHL | |
| Extraliga (SVK) | SVK | ECHL | ||
| Federal Prospects Hockey League (FPHL) | US | SPHL | ECHL | |
| FFHG Division 1 (France2) | FRA | Ligue Magnus | ECHL | |
| HockeyEttan (Swe) | SWE | Allsvenskan (Swe) | ECHL | |
| ICEHL | AUT/ITA/ HUN/SLO | ECHL | ||
| Kontinental Hockey League (KHL) | RUS/BLR/ KAZ/PRC | AHL | ECHL | |
| Ligue Magnus | FRA | ECHL | ||
| Liiga | FIN | ECHL | ||
| Ligue Nord-Americaine de Hockey (LNAH) | CAN | Ligue Magnus | ECHL | |
| Mestis | FIN | Liiga | ECHL | |
| Metal Ligaen (Den) | DEN | ECHL | ||
| National Hockey League (NHL) | US/CAN | AHL | ECHL | |
| National League (Sui) | SUI | AHL | ECHL | |
| NCAA | US | ECHL | ||
| NCAA III | US | SPHL | ECHL | |
| National Ice Hockey League (NIHL) | UK (ENG) | EIHL | ECHL | |
| New Zealand Ice Hockey League (NZIHL) | NZ | AIHL | ECHL | |
| Oberliga (Ger) | GER | DEL 2 | ECHL | |
| Polska Hokej Liga | POL | Extraliga (SVK) | ECHL | |
| Regionaliga (Ger) | GER | Oberliga | DEL 2 | ECHL |
| Romanian Hockey League | ROU | Erste Liga | ECHL | |
| Southern Professional Hockey League (SPHL) | US | ECHL | ||
| Suomi-Sarja | FIN | Mestis | Liiga | ECHL |
| Swedish Hockey League (SHL) | SWE | Allsvenskan (Swe) | ECHL | |
| Swiss League | SUI | National League (Sui) | AHL | ECHL |
| United State Hockey League (USHL) | US | NCAA | ECHL | |
| Usports | CAN | ECHL | ||
| Vysshaya Hokkeinaya Liga (VHL) | RUS | KHL | AHL | ECHL |
Calculations
Then I take the average P/G for a player moving from League A to League B is compared at each step along this path, giving a ratio (League B P/G)/(League A P/G). For example, for players in the sample moving from the ECHL to AHL, they score (on average) 0.78 P/G in the ECHL and 0.31 P/G in the AHL, giving a ratio of 0.40. For those going in the other direction (AHL to ECHL), the scoring rates are 0.26 and 0.79, respectively – resulting in a ratio of 3.03. Inverting the first ratio (so it reflects AHL to ECHL – the result is 2.49) and taking a weighted average of the two ratios results in a P/G ratio of 2.83. That is, someone scoring at an even 1.00 points-per-game in the “A”, is equivalent to someone scoring at 2.83 points-per-game in the ECHL.
To compare this scoring fairly from here requires one more step. Because some leagues score more goals per game than others, a point is easier to pick up in some leagues. So, then I compare the goals per game in each league (across all seasons from 2013-14 to 2022-23, except 2020-21) to the scoring rate in the ECHL, and adjust the P/G ratio accordingly. Taking from the AHL example above, the AHL averages 5.92 goals per game over the period, which is only 94.2% of the rate of the ECHL at 6.28, making scoring a little easier in the ECHL. So we adjust the 2.83 P/G ratio upward to reflect this – and it gives a final AHL to ECHL P/G ratio of 3.01.
The numbers are based on a database of:
- Over 1,500 players
- Over 3,300 transactions
- 536 different pathways from one league to another
Many leagues fit within this (very loose) group as well that I have excluded for various reasons. Firstly, aside from the USHL, we didn’t include any junior leagues – this is because with players moving from junior to senior play lots of things are happening at once – they are growing, maturing and moving into hockey with and against adults. By leaving these out, it just reduces at least a couple of these changes from the mix. Secondly, I chose to exclude leagues such as the Belarusian and Kazakhstani Leagues largely in the interests of not losing my mind (and they tend to have lots of transactions with the VHL – so you can get a guide as to them sitting somewhere below the VHL on the ranking).
Limitations
Like any piece of analysis, there are limitations here.
The first, and most obvious, is around the data collected. I collected by identifying players who meet the criteria and including them in the database. A better approach would be grabbing the whole database of Elite Prospects and ensuring that all possible data points are included (and the cost involved with getting the whole database is outside my budget for this endeavour). In a perfect world, this would include every player who meets the criteria listed above – not just the ones I could find. To counter this, significant samples have been used, but there remains a chance that there are biases involved in the data collection.
Secondly, the time period involved here is quite large. A ten-year period sees lots of change in leagues and their scoring, which has been explored in more detail by others[ii]. In short, the relative scoring across these leagues in the sample is unlikely to have remained constant. Originally this started as some work to inform AIHL import recruitment a few years ago, so building on older years was much easier than starting from scratch – but it does open up the issue of a longer period. That is before looking at the big time-related elephant in the room, COVID-19. The impact of the pandemic on hockey leagues was incredibly varied. Some leagues shut down, some operated basically as normal, and others operated under vastly different circumstances – all of which changes the nature of the league.
Thirdly, most leagues (especially those lower down the ranking list) have players with a wide range of playing ability. A simple example of this is the AIHL – where the highest scorers in recent seasons have been putting up over three points per game, and have strong pedigrees from leagues such as the ECHL, DEL2 in Germany and EIHL (to name just three). However, if we picked up players from AIHL teams’ third and fourth lines were picked up and dropped into those leagues – they would struggle immensely. This is to say that at best, this looks at an ‘average player’ moving from league to league – but which player is truly the average?
Finally (at least that I’ll address here) is the interpretation aspect. This isn’t a comparison of which the ‘best’ league is – it is just about scoring, and the equivalency of points in one league to another. There is obviously some correlation between that and which league is stronger, but that analysis would take a lot more into account – goaltending for starters. This is more useful if looking at scorers and the role they play – when trying to compare a stay-at-home D-man from one league to another using this, it won’t help a whole lot at first glance.
[i] At least I assume, I haven’t seen that movie – I don’t think anyone has.
[ii] A particularly good example is by Chace McCallum here: https://chacemccallum.substack.com/p/introducing-nhlz
Leave a Reply