Comparisons of Ranking List Example

An example of code is provided to illustrate the rating of teams with various methods and then to compare the ranking lists.

  1. Teams ratings are produced by the following methods:

    • WinLoss

    • Colley

    • Massey

    • Elo (win version)

    • Elo (points version)

    • Keener

    • OffenseDefense

    • AccuRATE

    • GeM

  2. Ranking lists are compared with Kendall tau

  3. In this example we have considered the first 20 matches of EPL 2018-2019

Data

Soccer Data: The first 20 games of EPL (2018-2019 season)

Python code

 1"""Ranking Lists comparisons"""
 2
 3import ratingslib.ratings as rl
 4import pandas as pd
 5from ratingslib.datasets.filenames import (FILENAME_EPL_2018_2019_20_GAMES,
 6                                           dataset_path)
 7from ratingslib.ratings.methods import rating_systems_to_dict
 8from ratingslib.ratings.metrics import kendall_tau_table
 9from ratingslib.utils.enums import ratings
10from ratingslib.utils.methods import print_info, print_pandas
11
12# ==========================================================================
13# Rating teams for the FILENAME_EPL_2018_2019_20_GAMES that contains the first
14# two match weeks of EPL 2018-2019
15# ==========================================================================
16
17stats_markov_dict = {
18    'TW': {'VOTE': 10, 'ITEM_I': 'FTHG', 'ITEM_J': 'FTAG',
19           'METHOD': 'VotingWithLosses'},
20    'TG': {'VOTE': 10, 'ITEM_I': 'FTHG', 'ITEM_J': 'FTAG',
21           'METHOD': 'WinnersAndLosersVotePoint'},
22    'TST': {'VOTE': 10, 'ITEM_I': 'HST', 'ITEM_J': 'AST',
23            'METHOD': 'WinnersAndLosersVotePoint'},
24    'TS': {'VOTE': 10, 'ITEM_I': 'HS', 'ITEM_J': 'AS',
25           'METHOD': 'WinnersAndLosersVotePoint'},
26}
27attributes_votes = {'TW': 10.0, 'TG': 10.0, 'TST': 10.0, 'TS': 10.0}
28
29ratings_list = [
30    rl.Winloss(normalization=False),
31    rl.Colley(),
32    rl.Massey(),
33    rl.Elo(version=ratings.ELOWIN, K=40, ks=400, HA=0,
34           starting_point=0),
35    rl.Elo(version=ratings.ELOPOINT, K=40, ks=400, HA=0,
36           starting_point=0),
37    rl.Keener(normalization=False),
38    rl.OffenseDefense(tol=0.0001),
39    rl.Markov(b=0.85, stats_markov_dict=stats_markov_dict),
40    rl.AccuRate()
41]
42ratings_dict = rating_systems_to_dict(ratings_list, key_based_on='version')
43
44filename = dataset_path(FILENAME_EPL_2018_2019_20_GAMES)
45# Dictionary that maps column names of csv files
46COLUMNS_DICT = {
47    'item_i': 'HomeTeam',
48    'item_j': 'AwayTeam',
49    'points_i': 'FTHG',
50    'points_j': 'FTAG',
51    'ts_i': 'HS',
52    'ts_j': 'AS',
53    'tst_i': 'HST',
54    'tst_j': 'AST',
55}
56rating_values_dict = {key: r.rate_from_file(
57    filename, columns_dict=COLUMNS_DICT) for key, r in ratings_dict.items()}
58
59# Print Ratings and Rankings
60print_info("Rating and ranking results")
61s = "-" * 100
62pd.set_option('float_format', "{:.4f}".format)
63for k, r in rating_values_dict.items():
64    print(k)
65    print_pandas(r)
66    print(s)
67
68
69# ==========================================================================
70# Kendall Tau comparison of ranking lists.
71# ==========================================================================
72print_info("Kendall Tau comparison of ranking lists")
73kendall_tau_results = kendall_tau_table(
74    ratings_dict=rating_values_dict, print_out=True)
75print("\n", s)

Results

The results of rating values are the same with Rating/Ranking Soccer Teams Example

The table below compares the ranking lists generated by the rating methods. The lower diagonal elements represent Kendall’s tau values of each pair, while the upper diagonal elements the p-values of each pair from the two-sided hypothesis test, whose null hypothesis is an absence of association.

=====Kendall Tau comparison of ranking lists=====
               Winloss   Colley  Massey    EloWin  EloPoint    Keener OffenseDefense  Markov  AccuRATE
Winloss            1.0  5.9e-06  0.0631  4.92e-06   1.6e-05  6.06e-06         0.0385   0.194  8.15e-06
Colley           0.831      1.0   0.111   7.1e-08  6.28e-06  1.78e-06         0.0347  0.0739  7.02e-06
Massey           0.339     0.26     1.0      0.14     0.186     0.186          0.186  0.0468     0.205
EloWin           0.859    0.907   0.247       1.0  5.17e-07  1.82e-07         0.0279  0.0821  2.46e-07
EloPoint         0.786    0.737   0.221     0.839       1.0  1.37e-13         0.0336  0.0638  1.28e-08
Keener           0.824     0.78   0.221     0.872     0.937       1.0         0.0336  0.0638  5.93e-09
OffenseDefense   0.377    0.345   0.221     0.367     0.347     0.347            1.0   0.288    0.0551
Markov           0.236    0.292  -0.326     0.291     0.305     0.305          0.179     1.0    0.0639
AccuRATE         0.818    0.738   0.207     0.868     0.928      0.95          0.313   0.302       1.0