18 EPL)

In this example, we assess the predictive performance of each rating system by combining it with ML classifier. The target class is the final outcome of soccer matches in the English Premier League (2009-2018 seasons). The predictions are performed through Naive Bayes classifier of scikit-learn library and we apply the walk-forward procedure.

Load packages

[1]:

import ratingslib.ratings as rl
from ratingslib.app_sports.methods import (Predictions, prepare_sports_seasons,
                                        rating_norm_features)
from ratingslib.application import SoccerOutcome
from ratingslib.datasets.filenames import get_seasons_dict_footballdata_online
from ratingslib.datasets.parameters import championships, stats
from ratingslib.utils.enums import ratings
from sklearn.naive_bayes import GaussianNB

Set target outcome

[2]:

outcome = SoccerOutcome()

Get filenames from football-data.co.uk for seasons 2009-2018 (English Premier League).

[3]:

filenames_dict = get_seasons_dict_footballdata_online(
    season_start=2009, season_end=2018,
    championship=championships.PREMIERLEAGUE)

We create a list of rating methods and then we convert it to dictionary. * For Massey a minimum limit of 20 games has been set to start the rating of teams. This number has been selected to provide enough games, and it ensures that the games graph is connected. * For Markov the damping factor b was set to 0.85 * For ELO The choice of parameters is those suggested by FIFA, K=40, ks=400 without taking into account the home field advantage (HA=0) * For WinLoss and Keener normalization is employed to produce fairer ratings since the teams may have a different number of games played (due to postponed or rescheduled matches). * For OffenseDefense the tolerance number we have selected to be 0.0001

[4]:

ratings_list = [
    rl.Winloss(normalization=True),
    rl.Colley(),
    rl.Massey(data_limit=20),
    rl.Elo(version=ratings.ELOWIN, K=40, ks=400, HA=0,
           starting_point=0),
    rl.Elo(version=ratings.ELOPOINT, K=40, ks=400, HA=0,
           starting_point=0),
    rl.Keener(normalization=True),
    rl.OffenseDefense(tol=0.0001),
    rl.Markov(b=0.85, stats_markov_dict=stats.STATS_MARKOV_DICT),
    rl.AccuRate()
]

The ratings in the dataset start from the second match week.

[5]:

data = prepare_sports_seasons(filenames_dict,
                              outcome,
                              rating_systems=ratings_list,
                              start_week=2)

Load season: 2009 - 2010
2.9%5.7%8.6%11.4%14.3%17.1%20.0%22.9%25.7%28.6%31.4%34.3%37.1%40.0%42.9%45.7%48.6%51.4%54.3%57.1%60.0%62.9%65.7%68.6%71.4%74.3%77.1%80.0%82.9%85.7%88.6%91.4%94.3%97.1%100.0%
Load season: 2010 - 2011
2.8%5.6%8.3%11.1%13.9%16.7%19.4%22.2%25.0%27.8%30.6%33.3%36.1%38.9%41.7%44.4%47.2%50.0%52.8%55.6%58.3%61.1%63.9%66.7%69.4%72.2%75.0%77.8%80.6%83.3%86.1%88.9%91.7%94.4%97.2%100.0%
Load season: 2011 - 2012
2.9%5.7%8.6%11.4%14.3%17.1%20.0%22.9%25.7%28.6%31.4%34.3%37.1%40.0%42.9%45.7%48.6%51.4%54.3%57.1%60.0%62.9%65.7%68.6%71.4%74.3%77.1%80.0%82.9%85.7%88.6%91.4%94.3%97.1%100.0%
Load season: 2012 - 2013
2.9%5.7%8.6%11.4%14.3%17.1%20.0%22.9%25.7%28.6%31.4%34.3%37.1%40.0%42.9%45.7%48.6%51.4%54.3%57.1%60.0%62.9%65.7%68.6%71.4%74.3%77.1%80.0%82.9%85.7%88.6%91.4%94.3%97.1%100.0%
Load season: 2013 - 2014
3.0%6.1%9.1%12.1%15.2%18.2%21.2%24.2%27.3%30.3%33.3%36.4%39.4%42.4%45.5%48.5%51.5%54.5%57.6%60.6%63.6%66.7%69.7%72.7%75.8%78.8%81.8%84.8%87.9%90.9%93.9%97.0%100.0%
Load season: 2014 - 2015
3.0%6.1%9.1%12.1%15.2%18.2%21.2%24.2%27.3%30.3%33.3%36.4%39.4%42.4%45.5%48.5%51.5%54.5%57.6%60.6%63.6%66.7%69.7%72.7%75.8%78.8%81.8%84.8%87.9%90.9%93.9%97.0%100.0%
Load season: 2015 - 2016
2.9%5.7%8.6%11.4%14.3%17.1%20.0%22.9%25.7%28.6%31.4%34.3%37.1%40.0%42.9%45.7%48.6%51.4%54.3%57.1%60.0%62.9%65.7%68.6%71.4%74.3%77.1%80.0%82.9%85.7%88.6%91.4%94.3%97.1%100.0%
Load season: 2016 - 2017
2.9%5.9%8.8%11.8%14.7%17.6%20.6%23.5%26.5%29.4%32.4%35.3%38.2%41.2%44.1%47.1%50.0%52.9%55.9%58.8%61.8%64.7%67.6%70.6%73.5%76.5%79.4%82.4%85.3%88.2%91.2%94.1%97.1%100.0%
Load season: 2017 - 2018
3.0%6.1%9.1%12.1%15.2%18.2%21.2%24.2%27.3%30.3%33.3%36.4%39.4%42.4%45.5%48.5%51.5%54.5%57.6%60.6%63.6%66.7%69.7%72.7%75.8%78.8%81.8%84.8%87.9%90.9%93.9%97.0%100.0%

We will use the normalized ratings values as ml features, thus we create the feature list.

[6]:

features_names_list = rating_norm_features(ratings_list)
features_names_list

[6]:

[['HratingnormWinloss[normalization=True]',
  'AratingnormWinloss[normalization=True]'],
 ['HratingnormColley', 'AratingnormColley'],
 ['HratingnormMassey[data_limit=20]', 'AratingnormMassey[data_limit=20]'],
 ['HratingnormEloWin[HA=0_K=40_ks=400]',
  'AratingnormEloWin[HA=0_K=40_ks=400]'],
 ['HratingnormEloPoint[HA=0_K=40_ks=400]',
  'AratingnormEloPoint[HA=0_K=40_ks=400]'],
 ['HratingnormKeener[normalization=True]',
  'AratingnormKeener[normalization=True]'],
 ['HratingnormOffenseDefense[tol=0.0001]',
  'AratingnormOffenseDefense[tol=0.0001]'],
 ['HratingnormMarkov[b=0.85]', 'AratingnormMarkov[b=0.85]'],
 ['HratingnormAccuRATE', 'AratingnormAccuRATE']]

We have selected the Naive Bayes classifier and we start making predictions from the 4th week. We apply the anchored walk-farward procedure with window size = 1 which means that every week we make predictions by using previous weeks data for training set. For example for the 4th week, the training set is consisted of the 1st, 2nd and 3rd week. Note that every season we restart the walk-forward procedure.

[7]:

results = Predictions(data, outcome, start_from_week=4).ml_pred_parallel(
    clf_list=[GaussianNB()], features_names_list=features_names_list)



=====Accuracy results=====

                                                                                                        Accuracy  \
GaussianNB()-[features: HratingnormWinloss[normalization=True] AratingnormWinloss[normalization=True]]  0.488875
GaussianNB()-[features: HratingnormColley AratingnormColley]                                            0.490464
GaussianNB()-[features: HratingnormMassey[data_limit=20] AratingnormMassey[data_limit=20]]              0.494473
GaussianNB()-[features: HratingnormEloWin[HA=0_K=40_ks=400] AratingnormEloWin[HA=0_K=40_ks=400]]        0.489193
GaussianNB()-[features: HratingnormEloPoint[HA=0_K=40_ks=400] AratingnormEloPoint[HA=0_K=40_ks=400]]    0.492689
GaussianNB()-[features: HratingnormKeener[normalization=True] AratingnormKeener[normalization=True]]    0.498093
GaussianNB()-[features: HratingnormOffenseDefense[tol=0.0001] AratingnormOffenseDefense[tol=0.0001]]    0.475207
GaussianNB()-[features: HratingnormMarkov[b=0.85] AratingnormMarkov[b=0.85]]                            0.496503
GaussianNB()-[features: HratingnormAccuRATE AratingnormAccuRATE]                                        0.492371

                                                                                                        Correct Games  \
GaussianNB()-[features: HratingnormWinloss[normalization=True] AratingnormWinloss[normalization=True]]           1538
GaussianNB()-[features: HratingnormColley AratingnormColley]                                                     1543
GaussianNB()-[features: HratingnormMassey[data_limit=20] AratingnormMassey[data_limit=20]]                       1521
GaussianNB()-[features: HratingnormEloWin[HA=0_K=40_ks=400] AratingnormEloWin[HA=0_K=40_ks=400]]                 1539
GaussianNB()-[features: HratingnormEloPoint[HA=0_K=40_ks=400] AratingnormEloPoint[HA=0_K=40_ks=400]]             1550
GaussianNB()-[features: HratingnormKeener[normalization=True] AratingnormKeener[normalization=True]]             1567
GaussianNB()-[features: HratingnormOffenseDefense[tol=0.0001] AratingnormOffenseDefense[tol=0.0001]]             1495
GaussianNB()-[features: HratingnormMarkov[b=0.85] AratingnormMarkov[b=0.85]]                                     1562
GaussianNB()-[features: HratingnormAccuRATE AratingnormAccuRATE]                                                 1549

                                                                                                        Wrong Games  \
GaussianNB()-[features: HratingnormWinloss[normalization=True] AratingnormWinloss[normalization=True]]         1608
GaussianNB()-[features: HratingnormColley AratingnormColley]                                                   1603
GaussianNB()-[features: HratingnormMassey[data_limit=20] AratingnormMassey[data_limit=20]]                     1555
GaussianNB()-[features: HratingnormEloWin[HA=0_K=40_ks=400] AratingnormEloWin[HA=0_K=40_ks=400]]               1607
GaussianNB()-[features: HratingnormEloPoint[HA=0_K=40_ks=400] AratingnormEloPoint[HA=0_K=40_ks=400]]           1596
GaussianNB()-[features: HratingnormKeener[normalization=True] AratingnormKeener[normalization=True]]           1579
GaussianNB()-[features: HratingnormOffenseDefense[tol=0.0001] AratingnormOffenseDefense[tol=0.0001]]           1651
GaussianNB()-[features: HratingnormMarkov[b=0.85] AratingnormMarkov[b=0.85]]                                   1584
GaussianNB()-[features: HratingnormAccuRATE AratingnormAccuRATE]                                               1597

                                                                                                        Total Games
GaussianNB()-[features: HratingnormWinloss[normalization=True] AratingnormWinloss[normalization=True]]         3146
GaussianNB()-[features: HratingnormColley AratingnormColley]                                                   3146
GaussianNB()-[features: HratingnormMassey[data_limit=20] AratingnormMassey[data_limit=20]]                     3076
GaussianNB()-[features: HratingnormEloWin[HA=0_K=40_ks=400] AratingnormEloWin[HA=0_K=40_ks=400]]               3146
GaussianNB()-[features: HratingnormEloPoint[HA=0_K=40_ks=400] AratingnormEloPoint[HA=0_K=40_ks=400]]           3146
GaussianNB()-[features: HratingnormKeener[normalization=True] AratingnormKeener[normalization=True]]           3146
GaussianNB()-[features: HratingnormOffenseDefense[tol=0.0001] AratingnormOffenseDefense[tol=0.0001]]           3146
GaussianNB()-[features: HratingnormMarkov[b=0.85] AratingnormMarkov[b=0.85]]                                   3146
GaussianNB()-[features: HratingnormAccuRATE AratingnormAccuRATE]                                               3146