Comparisons of Ranking List Example
An example of code is provided to illustrate the rating of teams with various methods and then to compare the ranking lists.
Teams ratings are produced by the following methods:
WinLoss
Colley
Massey
Elo (win version)
Elo (points version)
Keener
OffenseDefense
AccuRATE
GeM
Ranking lists are compared with Kendall tau
In this example we have considered the first 20 matches of EPL 2018-2019
Python code
1"""Ranking Lists comparisons"""
2
3import ratingslib.ratings as rl
4import pandas as pd
5from ratingslib.datasets.filenames import (FILENAME_EPL_2018_2019_20_GAMES,
6 dataset_path)
7from ratingslib.ratings.methods import rating_systems_to_dict
8from ratingslib.ratings.metrics import kendall_tau_table
9from ratingslib.utils.enums import ratings
10from ratingslib.utils.methods import print_info, print_pandas
11
12# ==========================================================================
13# Rating teams for the FILENAME_EPL_2018_2019_20_GAMES that contains the first
14# two match weeks of EPL 2018-2019
15# ==========================================================================
16
17stats_markov_dict = {
18 'TW': {'VOTE': 10, 'ITEM_I': 'FTHG', 'ITEM_J': 'FTAG',
19 'METHOD': 'VotingWithLosses'},
20 'TG': {'VOTE': 10, 'ITEM_I': 'FTHG', 'ITEM_J': 'FTAG',
21 'METHOD': 'WinnersAndLosersVotePoint'},
22 'TST': {'VOTE': 10, 'ITEM_I': 'HST', 'ITEM_J': 'AST',
23 'METHOD': 'WinnersAndLosersVotePoint'},
24 'TS': {'VOTE': 10, 'ITEM_I': 'HS', 'ITEM_J': 'AS',
25 'METHOD': 'WinnersAndLosersVotePoint'},
26}
27attributes_votes = {'TW': 10.0, 'TG': 10.0, 'TST': 10.0, 'TS': 10.0}
28
29ratings_list = [
30 rl.Winloss(normalization=False),
31 rl.Colley(),
32 rl.Massey(),
33 rl.Elo(version=ratings.ELOWIN, K=40, ks=400, HA=0,
34 starting_point=0),
35 rl.Elo(version=ratings.ELOPOINT, K=40, ks=400, HA=0,
36 starting_point=0),
37 rl.Keener(normalization=False),
38 rl.OffenseDefense(tol=0.0001),
39 rl.Markov(b=0.85, stats_markov_dict=stats_markov_dict),
40 rl.AccuRate()
41]
42ratings_dict = rating_systems_to_dict(ratings_list, key_based_on='version')
43
44filename = dataset_path(FILENAME_EPL_2018_2019_20_GAMES)
45# Dictionary that maps column names of csv files
46COLUMNS_DICT = {
47 'item_i': 'HomeTeam',
48 'item_j': 'AwayTeam',
49 'points_i': 'FTHG',
50 'points_j': 'FTAG',
51 'ts_i': 'HS',
52 'ts_j': 'AS',
53 'tst_i': 'HST',
54 'tst_j': 'AST',
55}
56rating_values_dict = {key: r.rate_from_file(
57 filename, columns_dict=COLUMNS_DICT) for key, r in ratings_dict.items()}
58
59# Print Ratings and Rankings
60print_info("Rating and ranking results")
61s = "-" * 100
62pd.set_option('float_format', "{:.4f}".format)
63for k, r in rating_values_dict.items():
64 print(k)
65 print_pandas(r)
66 print(s)
67
68
69# ==========================================================================
70# Kendall Tau comparison of ranking lists.
71# ==========================================================================
72print_info("Kendall Tau comparison of ranking lists")
73kendall_tau_results = kendall_tau_table(
74 ratings_dict=rating_values_dict, print_out=True)
75print("\n", s)
Results
The results of rating values are the same with Rating/Ranking Soccer Teams Example
The table below compares the ranking lists generated by the rating methods. The lower diagonal elements represent Kendall’s tau values of each pair, while the upper diagonal elements the p-values of each pair from the two-sided hypothesis test, whose null hypothesis is an absence of association.
=====Kendall Tau comparison of ranking lists=====
Winloss Colley Massey EloWin EloPoint Keener OffenseDefense Markov AccuRATE
Winloss 1.0 5.9e-06 0.0631 4.92e-06 1.6e-05 6.06e-06 0.0385 0.194 8.15e-06
Colley 0.831 1.0 0.111 7.1e-08 6.28e-06 1.78e-06 0.0347 0.0739 7.02e-06
Massey 0.339 0.26 1.0 0.14 0.186 0.186 0.186 0.0468 0.205
EloWin 0.859 0.907 0.247 1.0 5.17e-07 1.82e-07 0.0279 0.0821 2.46e-07
EloPoint 0.786 0.737 0.221 0.839 1.0 1.37e-13 0.0336 0.0638 1.28e-08
Keener 0.824 0.78 0.221 0.872 0.937 1.0 0.0336 0.0638 5.93e-09
OffenseDefense 0.377 0.345 0.221 0.367 0.347 0.347 1.0 0.288 0.0551
Markov 0.236 0.292 -0.326 0.291 0.305 0.305 0.179 1.0 0.0639
AccuRATE 0.818 0.738 0.207 0.868 0.928 0.95 0.313 0.302 1.0