World Cup Follow-Up: Update of Winning Probabilities and Betting Results(blog.wolfram.com)
blog.wolfram.com
World Cup Follow-Up: Update of Winning Probabilities and Betting Results
http://blog.wolfram.com/2014/06/26/world-cup-follow-up-update-of-winning-probabilities-and-betting-results/
17 comments
I haven't used it but I have found most of these demos to look very staged and there is too much magic going on. Isn't the whole point of this to blur the lines between language and cloud so you are eternally dependent on their system and have to pay them when you need to seriously scale or deploy something?
it isn't magic, but simply the fact that the language is "dynamically typed" (or more accurately, that the whole system works symbolically). the "magic" is due to its UI compositing system using the same exact semantics as the rest of the language (or rather, it is the same language). this makes graphics-making and UI-making super easy.
have a look at the Mathematica stack exchange[1] site to see more code. there isn't any contrivance there.
[1] http://mathematica.stackexchange.com/questions/tagged/graphi...
have a look at the Mathematica stack exchange[1] site to see more code. there isn't any contrivance there.
[1] http://mathematica.stackexchange.com/questions/tagged/graphi...
Wolfram language is very close to Lisp, which makes it an ideal tool to represent data and algorithms. I like the fact that they use a function representation that is similar to Prolog, so that you are free to choose between Lisp and Prolog techniques as desired.
I'm not sure I understand the data this model presents. I focused on Belgium (being Belgian) and their next competitor (the US).
Even though Belgium comes out slightly ahead in the "chance of victory" graph, and in the "most probably game tree" they are picked as the more likely winner, the lower graphs (chance to reach, chance of knock-out) show the US as having a higher chance of making it to the quarter finals than Belgium.
Isn't this inconsistent?
Even though Belgium comes out slightly ahead in the "chance of victory" graph, and in the "most probably game tree" they are picked as the more likely winner, the lower graphs (chance to reach, chance of knock-out) show the US as having a higher chance of making it to the quarter finals than Belgium.
Isn't this inconsistent?
Indeed you are right, the figure showing the "chance to reach, chance to knock-out" was accidentally an outdated one (from a test simulation with a lower number of trials). The real figure will be updated soon.
They are quite close though, good job noticing that!
--
it has been updated now
I figured as much, seeing as the chance to reach the group of 16 was below 1.0 for all teams. Looking forward to seeing the new charts!
Belgium probably has a better chance to beat the last two teams given that they make it to the quarterfinals. The US might have a better chance of getting to the quarterfinals but a high chance of being knocked out after that.
The "Winning Probabilities" chart is calculated "starting from the round of 16". But the "chance-to-reach/chance-of-knockout" graphs below seem to have been calculated at an earlier time - it shows the probability of reaching the round of 16 as less than 1 for each team.
I think the US lost their last group match, but Belgium won theirs, so maybe something like shifted the model probabilities between the time of the 2 graphs.
I think the US lost their last group match, but Belgium won theirs, so maybe something like shifted the model probabilities between the time of the 2 graphs.
The US lost, but it was Germany that was knocked out.
No, we (Portugal) got knocked out, since the US had the same points and more goals scored.
No. The US lost, but both Germany and the US qualified for the next round, with Germany being group lead (Germany is 1G, US is 2G).
Not necessarily. You could say that Belgium has a lower chance of making it to the quarter finals, but, if they do, they have a great advantage, for example.
I'm not saying this is the case, but it explains how you could have a lower probability to reach the quarter finals but a higher probability to win overall.
I'm not saying this is the case, but it explains how you could have a lower probability to reach the quarter finals but a higher probability to win overall.
The probability of winning all games is inherently conditional on the probability of reaching the quarter finals. To equate this to common examples of probability, winning at the knockout stage is like flipping a biased coin and getting 4 heads in a row (= four matches won).
As such, there is no way the probability of winning can exceed the probability of reaching the quarter finals.
As such, there is no way the probability of winning can exceed the probability of reaching the quarter finals.
This is a different situation.
We have
We have
P(Belgium reaches quarters) >= P(Belgium wins overall)
P(USA reaches quarters) >= P(USA wins overall)
P(USA reaches quarters) >= P(Belgium reaches quarter)
P(USA wins overall) >= P(Belgium wins overall)The coin won't get any advantage from playing in its home field, for example. I specifically said that I'm not arguing that this is the case here, but one can conceive of an example where it would.
I cannot conceive of an example. In fact I argued that it is impossible because of the fundamental rules of conditional probability.
Probability of losing is not transitive. If A is likely to lose to B, and B is likely to lose to C, it does not follow that A is likely to lose to C.
Unless we're using ELO to determine these probabilities.
Aw, you're right. I didn't read closely enough to understand their model.
Comparing this to 538's (quite different) Analysis [1]:
#1
* 538 - Brazil - 36.0%
* WOLF - Brazil - 32.2%
#2
* 538 - Argentina - 17.0%
* WOLF - Netherlands - 23.5%
#3
* 538 - Germany - 12.0%
* WOLF - Germany - 21.6%
This paints a pretty interesting picture about trying to use statistics and math to predict sporting tournaments like this. Seems like nobody has been very successful at this yet. Obviously it gets "easier" as the tournament progresses, though.
* [1] http://fivethirtyeight.com/interactives/world-cup/
#1
* 538 - Brazil - 36.0%
* WOLF - Brazil - 32.2%
#2
* 538 - Argentina - 17.0%
* WOLF - Netherlands - 23.5%
#3
* 538 - Germany - 12.0%
* WOLF - Germany - 21.6%
This paints a pretty interesting picture about trying to use statistics and math to predict sporting tournaments like this. Seems like nobody has been very successful at this yet. Obviously it gets "easier" as the tournament progresses, though.
* [1] http://fivethirtyeight.com/interactives/world-cup/
This is interesting but not surprising: the fact that scoring in (association) football is a relatively rare event means that it is less predictable at the individual match level than other team sports such as rugby, American football and basketball that feature more frequent scoring events; i.e. there is a greater likelihood of the underdog snatching a surprise win.
That is absolutely incorrect. Heavy underdogs in the group stages of this world cup had frequently had moneylines of +1300 and higher (ex Netherlands vs Austrailia). There is NEVER an NFL game in a typical season where the moneyline (ie without pointspread) on the underdog will pay out that high. So, the is a much SMALLER likelihood of the underdog snatching a surprise win.
Many pro sports leagues like the NFL are designed to have parity amongst the teams, to make it more exciting. There is no way to ensure parity in national teams. If there was an international American football competition, it would have even less parity than international basketball competitions (where the US is extremely heavily favored in nearly every game).
Perhaps the underlying process is just so noisy, better models are not really possible. It would be interesting to see confidence intervals on those predictions - I am relatively confident they would overlap. My intuition tells me that the confidence intervals would be huge and there is very little statistically significant differentiation between the best and worst teams in the tournament - upsets happen in soccer many times every tournament.
I think how successful a model is depends on your purpose too - if you have a model that predicts only 60% of games correctly, while technically you're not very "good", you're doing much better than those gambling on the sport (usually the favorite in sports betting wins at a rate of 52-54% I think) or probably conventional 'sports analysts' (no data to support the second point, that's an educated guess based on the gambling statistic).
I think how successful a model is depends on your purpose too - if you have a model that predicts only 60% of games correctly, while technically you're not very "good", you're doing much better than those gambling on the sport (usually the favorite in sports betting wins at a rate of 52-54% I think) or probably conventional 'sports analysts' (no data to support the second point, that's an educated guess based on the gambling statistic).
Costa Rica and Algeria were given no chance of qualifying, while Spain, Portugal and England were right up there with Brazil.
http://blog.wolfram.com/2014/06/20/predicting-who-will-win-t...
Just goes to show, when it comes to sport, you can throw away most of these models until they start taking into account news and gossip from tabloids which probably has more bearing on team performance than raw numbers.
http://blog.wolfram.com/2014/06/20/predicting-who-will-win-t...
Just goes to show, when it comes to sport, you can throw away most of these models until they start taking into account news and gossip from tabloids which probably has more bearing on team performance than raw numbers.
Their analysis is not quite different if they agree on 2 out of 3. Also, both are good, it's pretty much the prediction anyone into the World Cup would make at this point.
Someone would bet Argentina #2 if the have faith Messi will play what he's used to, or Netherlands if they consider the actual performance in this cup. This reflects 538's stronger bias for player skill score, while Wolfram seems to adjust more for the past results.
Someone would bet Argentina #2 if the have faith Messi will play what he's used to, or Netherlands if they consider the actual performance in this cup. This reflects 538's stronger bias for player skill score, while Wolfram seems to adjust more for the past results.
My simpler, Elo-based model gives (currently):
Brazil: 24.2% Germany: 23.0% Netherlands: 19.0% Argentina: 15.7%
Brazil: 24.2% Germany: 23.0% Netherlands: 19.0% Argentina: 15.7%
France is also really good.. need to be on the top ones
They are quite different indeed. It would be interesting to compare, at the end of the tournament, the likelihoods of both models on all matches. This would give a pretty good idea on how good were the models.
I think you need to factor some other datapoints too:
- probable weather - some teams play differently depending on conditions
- yellow and red cards - some major player could miss a match. If team A is missing a defender, might be that is more vulnerable and thus, may lose a match.
and so on.
- probable weather - some teams play differently depending on conditions
- yellow and red cards - some major player could miss a match. If team A is missing a defender, might be that is more vulnerable and thus, may lose a match.
and so on.
I find comments like this unconstructive, and they always surface when people present forecasts. It's easy to play the shoulda-woulda-coulda game with data. "You should account for the breaking strengths of each team's players' ACL tendons. It could affect the outcome if they get injured!" Okay, so what? Did you do it? Does the data exist? Do you have any quantitative evidence of a statistically significant link between the weather and team performance? If you do, do you also have the evidence to show that the game forecast is not rate limited by the error of the weather forecast?
I'm not saying you have to write a thesis to suggest new data sources, but I am saying that "you need to do XYZ" is not a productive piece of advice without the evidence to back it.
I'm not saying you have to write a thesis to suggest new data sources, but I am saying that "you need to do XYZ" is not a productive piece of advice without the evidence to back it.
It's obvious that weather is a huge factor and if you think this is just a "shoulda-woulda-coulda game", then maybe you need to take a step back and admit you have no domain knowledge?
Also, if you're doing statistical analyses of your own, hopefully this isn't how you react to domain experts giving you advice on factors you should consider. If you react the way you just did, it's quite likely people will just shut up and you'll be left floundering the darkness.
Also, if you're doing statistical analyses of your own, hopefully this isn't how you react to domain experts giving you advice on factors you should consider. If you react the way you just did, it's quite likely people will just shut up and you'll be left floundering the darkness.
Good points. Now I am not going to excuse for expriming my thoughts on a public forum, but I will give right to you that one need to have data to sustain his points.
Now strictly to subject, would be indeed a nice exercise to take a team performance ( under any scoring formula ) and correlate it with the weather at that time. This data ( team data and weather data) exists. If we find any correlation - not sure. Could be that for some teams it doesn't exist - they play good no matter the atmospheric conditions.
The second exercise is the yellow cards, but this is indeed hard to factor since is very dependent of the coach strategy ( we cannot assume that the same player always plays). So, yes, OK, let's leave that out.
Now strictly to subject, would be indeed a nice exercise to take a team performance ( under any scoring formula ) and correlate it with the weather at that time. This data ( team data and weather data) exists. If we find any correlation - not sure. Could be that for some teams it doesn't exist - they play good no matter the atmospheric conditions.
The second exercise is the yellow cards, but this is indeed hard to factor since is very dependent of the coach strategy ( we cannot assume that the same player always plays). So, yes, OK, let's leave that out.
Though I agree with you on the weather bit, there is already a history of yellow and red cards from the tournament already, which could have huge implications on the tournament ahead. (For eg, Luis Suarez getting booted is one.)
It would be useful to have a database of football punishments so teams and fans could see whether a player has been dealt with fairly or not. Perhaps one already exists?
Also: Local support. For example, Colombia have a HUGE support in the stadium, they say are playing as a local all the time.
Plus: History against X. If a team have played against another a lot of times it build some model in how perform. Latin america teams know more about each others than teams abroad.
I see in this model not much discussion in how latin-teams have the upper hand here: The climate, the people, the shared-history, the kind of game, etc
Plus: History against X. If a team have played against another a lot of times it build some model in how perform. Latin america teams know more about each others than teams abroad.
I see in this model not much discussion in how latin-teams have the upper hand here: The climate, the people, the shared-history, the kind of game, etc
Indeed, it would be interesting to add more features. However, for international games, I saw that it was quite hard to improve accuracy beyond what I already have. One other approach would be to create alternative ranking systems (modified Elo...) and use them as features.
Elo ratings (such as those kept at eloratings.net) actually work quite well. Ultimately, they work like a weighted moving average.
The thing that I find hard to predict and build into my models is style of play. By style I mean: spatially-intensive, high-time-inpossession-the-ball-time (e.g. Spain with tiki-taka and Germany to some extent) versus time-intensive, opportunity-seeking/opportunity-creating (such as Brazil, Argentina, etc.)
Why? Because passing-intensive teams seem to display more of an own effect -- they fall or rise on the strength of their team, since it's an intricate, very technical and collaborative style. The results of opportunity-seeking teams are much more dependent on the strength of the adversary -- i.e. much more Elo-like.
Ideally, I'd be able to infer from the data a (exponentially biased to recent games) own-team/spatial play dependence factor as opposed to a strength-of-opponent/opportunity-seeking factor. In principle if all victories were explainable by a combination of those two variables the Elo residual/surprise would be a measure of this, but hey, teams get better/worse at opportunity-seeking too, even teams specialized in tiki-taka.
The thing that I find hard to predict and build into my models is style of play. By style I mean: spatially-intensive, high-time-inpossession-the-ball-time (e.g. Spain with tiki-taka and Germany to some extent) versus time-intensive, opportunity-seeking/opportunity-creating (such as Brazil, Argentina, etc.)
Why? Because passing-intensive teams seem to display more of an own effect -- they fall or rise on the strength of their team, since it's an intricate, very technical and collaborative style. The results of opportunity-seeking teams are much more dependent on the strength of the adversary -- i.e. much more Elo-like.
Ideally, I'd be able to infer from the data a (exponentially biased to recent games) own-team/spatial play dependence factor as opposed to a strength-of-opponent/opportunity-seeking factor. In principle if all victories were explainable by a combination of those two variables the Elo residual/surprise would be a measure of this, but hey, teams get better/worse at opportunity-seeking too, even teams specialized in tiki-taka.
I tend to agree with you, these factors would not improve the already good accuracy, however due to KO phase the games are all-or-nothing ( unlike other types of international games ), so some little factor ( extreme heat or a missing player ) could incline to a victory or not.
Anyway, good work!
Anyway, good work!
In the case of missing a player, take Uruguay for example: they just lost their best player after he bit (yes, à la Mike Tyson) the shoulder of a player on his last game. Fifa banned him for some 8 games, so he'll be absent for the rest of the cup. Uruguay just took a big blow.
You know what would be interesting, to perform this analysis to all past world cups, looking at the group results and computing the possible outcomes of the brackets. And then, compute some statistical indicator like the p-value (sorry I am very weak in statistics).
These statistical analyses never work well for football and this one is another example of that happening.
For example, except for Spain in South Africa, no European country has ever won the world cup outside of Europe. Why is this? A large part of this is due to the weather. And unsurprisingly, South African weather in the southern winter isn't terribly different from what the Europeans are used to. This stat alone should've predicted a lot of the European "upsets" and should trigger your suspicions about 2 out of the top 3 favorites being European.
And even if you were unconvinced about this prior to the tournament, having watched teams struggling in Manaus should've convinced you that these statistical analyses ignore important information that is obvious to even semi-casual fans.
For example, except for Spain in South Africa, no European country has ever won the world cup outside of Europe. Why is this? A large part of this is due to the weather. And unsurprisingly, South African weather in the southern winter isn't terribly different from what the Europeans are used to. This stat alone should've predicted a lot of the European "upsets" and should trigger your suspicions about 2 out of the top 3 favorites being European.
And even if you were unconvinced about this prior to the tournament, having watched teams struggling in Manaus should've convinced you that these statistical analyses ignore important information that is obvious to even semi-casual fans.
I am still unconvinced. The data set is incredibly small, incredibly noisy, and after the South African tournament only convincing if you cherry pick the right results.
The last time an Europen team didn't make it to the finals was in 1950. If climate really did matter that much, you wouldn't expect that. In addition to 2010 that you chose to ignore, the US world cup final went to the penalties and the 1986 Mexico final was tied 2-2 between Argentina and West Germany until something like 10 minutes before the end. (And a non-European team was in the finals largely thanks to the most famous refereeing error in the history of the sport).
That evidence looks really weak.
The last time an Europen team didn't make it to the finals was in 1950. If climate really did matter that much, you wouldn't expect that. In addition to 2010 that you chose to ignore, the US world cup final went to the penalties and the 1986 Mexico final was tied 2-2 between Argentina and West Germany until something like 10 minutes before the end. (And a non-European team was in the finals largely thanks to the most famous refereeing error in the history of the sport).
That evidence looks really weak.
Yeah but Brazil has continental proportions.. so you can find hot/cold weather conditions depending on the location..
Im Brazilian and my childhood city was between -3C to -10C on winter, more in the South Brazil(Paraná)
I Think half of the stadiums in the world cup have mid to cold wheater conditions
Belo Horizonte, São Paulo, Curitiba, Porto Alegre
Tropical Wheater:
Rio, Cuiaba, Natal, Fortaleza, Recife, Manaus(this is probably the worse)
Lets not forget that some of those latin americans nations are used to cold wheater conditions, like Uruguay and Argentina(as in South Brazil)
So it depends..
Im Brazilian and my childhood city was between -3C to -10C on winter, more in the South Brazil(Paraná)
I Think half of the stadiums in the world cup have mid to cold wheater conditions
Belo Horizonte, São Paulo, Curitiba, Porto Alegre
Tropical Wheater:
Rio, Cuiaba, Natal, Fortaleza, Recife, Manaus(this is probably the worse)
Lets not forget that some of those latin americans nations are used to cold wheater conditions, like Uruguay and Argentina(as in South Brazil)
So it depends..
>Can we at least conclude that our probabilities outperform bookmakers?
More like outperform the market?
Not sure if more precise probabilities outweigh the risk of ruin, if there's unbalanced amount of money backing each option.
More like outperform the market?
Not sure if more precise probabilities outweigh the risk of ruin, if there's unbalanced amount of money backing each option.
In gambling, bookmakers = the market. Just like a lot of demand (unbalanced amount of money) buying up a stock pushes up the price, the same will cause bookmakers to adjust their lines.
It's very, very hard to make a model that outperforms bookmakers in the long run. If you find some sort of inefficiency in the betting market, maybe. But it turns out that most often the betting market is inefficient at the points where casual betters outnumber the "sharps," which tends to be the points at which the bookmakers have the highest juice on their odds, so you're not likely to come out ahead anyway.
do you happen to know where one could find data on this? Not just bookmakers' odds, but volume of bets? This would make for interesting research
Betfair has an API. This can be used to return the odds and volumes matched for all sports on their Exchange platform.
https://developer.betfair.com/
https://developer.betfair.com/
It's not the complete picture, but more expensive players tend to be better than cheaper players. So more expensive teams tend to be better than cheaper teams. I see this is completely ignored.
Also team Elo has issues, as not a lot of games are played, former achievements weigh in too heavily.
Nonetheless, this is a really nice effort. Let's see how well it works.
Also team Elo has issues, as not a lot of games are played, former achievements weigh in too heavily.
Nonetheless, this is a really nice effort. Let's see how well it works.
Can someone explain the Belgium vs USA Tree/Graphs for me. The most likely tree shows Belgium beating the USA but in the graphs the USA has a higher chance of getting to the Round of 16?
It looks like they switched the graphs for Belgium and the US, since most other sources[1][2] give Belgium a slight edge and even their own most likely outcome graph shows Belgium as the favorite.
[1] http://fivethirtyeight.com/interactives/world-cup/ [2] http://rogerkaufmann.ch/dsaINTe_r.htm
[1] http://fivethirtyeight.com/interactives/world-cup/ [2] http://rogerkaufmann.ch/dsaINTe_r.htm
The figure was outdated. It has been fixed now.
This article makes the rather optimistic assumption that bookie prices efficiently and purely represent probabilistic models. This isn't quite the truth, as many bookies can and do take a position against the weight of money (punters often bet disproportionately and according to some psychology). There's also the juice to consider, particularly for underdogs, where value is quickly eroded.
It would be interesting to see if the predictions would be different if the official FIFA ranking[1] was used instead of ELO.
[1] http://www.fifa.com/worldranking/rankingtable/
[1] http://www.fifa.com/worldranking/rankingtable/
> There have been some surprises: from 10 of our favorite teams, 3 have been eliminated (Portugal, England, and, most surprisingly, Spain)
So Italy, that won the world cup 4 times, wasn't one of the 10 favorites??
So Italy, that won the world cup 4 times, wasn't one of the 10 favorites??
I hardly think winning in 1934, 1938 or 1982 has any impact on present day results. Even claiming that the 2006 win has much bearing is a stretch since they have a different head coach and only 4 of the same players.
Then England should not be a favorite either.
The favorites are based on the team's recent performance, not on World Cup victories. For instance, the Netherlands have never won a World Cup, but their recent performance prior to this World Cup is undeniably better than either England or Italy, so they are ranked higher.
England were never a favourite. They were a 20/1 going in to the tournament.
Unfortunately for Italy, it did not have very good match records before the world cup, and was thus not in the top 10 of the previous post (http://blog.wolfram.com/2014/06/20/predicting-who-will-win-t...).
They're bracket prediction just picks the higher ranked team every time...
yes, and?
I'm not sure exactly what they're referring to, but they may mean that in a series of Monte Carlo simulations, the higher ranked team doesn't win every time, they win with a higher probability. Which means that (probably) more of the outcomes result in them winning, but (probably) not all of them. The article did say they were doing Monte Carlo simulations but I don't see where they're not, except for when they define the most likely bracket (in which case choosing the higher-ranked team every time is correct).
This guy is doing the same calculations with slightly different predictions than Wolfram
http://rogerkaufmann.ch/dsaINTe_r.htm
http://rogerkaufmann.ch/dsaINTe_r.htm
Costa Rica knocked out both England and Italy, and yet it places below USA? Nonetheless, it's an interesting approach.
I would guess that world ranking plays into the algorithm a bit. USA entered the World Cup ranked 13th in the world, while Costa Rica were placed 32nd (from their Elo Ratings (http://eloratings.net/)). Both teams, if they make it into the quarter finals, would probably face buzzsaws (Netherlands for CRC and Argentina for USA).
USA knocked out both Portugal and Ghana and split results against Los Ticos in qualifying while finishing on top of CONCACAF.
The US has a 1.2% chance of winning.
"So you're saying there's a chance..."
"So you're saying there's a chance..."
Or put another way they don't just focus on making a general purpose programming language or even a math focused language, they actually work to make the language inter operate well with real world data sets.
In my opinion they've passed Matlab and R in terms of being able to take a raw data set and quickly ask and answer questions about it and as someone who does this for a living I'm very happy Mathamatica exists, although R still wins in terms of cost:)