Deconstructing The Eurovision Results Through An Essential Analysis Of Italy’s Voting

Deconstructing The Eurovision Results Through An Essential Analysis Of Italy’s Voting

In this year’s post-Eurovision discussions, the new voting system and the effects that ranking all the songs, not just the Top Ten, is featuring heavily. Why are we getting such diverse results from the combination of the jury and the televote? Can you really finish second in a televote and still score no points?  John Egan crunches some numbers to find out.

The Old Rules Of Thumb Are Useless

What would have happened if this year’s Eurovision Song Contest did not have the new ranking system? Taking the Italian televote and jury votes detailed below, we can see the impact that the new system has.

The votes given out on the night with the new system were 12 Denmark, 10 Malta, 8 Norway, 7 Greece, 6 Lithuania, 5 Ukraine, 4 Moldova, 3 Hungary, 2 Spain, 1 Romania. If the old system was used we would have had 12 Denmark, 10 Romania, 8 Norway, 7 Malta, 6 Moldova, 5 Ukraine, 4 Greece, 3 Spain, 2 Lithuania, 1 Russia.

Denmark gets 12 regardless. Moldova would have got 2  more points, and Russia and Spain each would have got one more point. Lithuania would have received 4 less points, Malta and Greece 3 less points each. The big issue here is Romania. Topping the televote Cezar would have scored 10 points in 2012, but this year he scored just a single point.

Over the years, everyone has built up expectations of what happens in the voting. If a country gets a strong televote, it’s going to get something in the final scores, even if the jury don’t like it. Top the televote, and you’d likely score 5 or 6 points, even if the jury ignore the song. But that has all changed and with the new ranking system these rules of thumb do not apply.

This could be the core of the problem we are seeing now in the media. As the accusations fly around Europe that votes have gone missing, people are forgetting that the new ranking system does not work in the same way or give the same results as the older system. It’s very likely that if the 2012 system was in use, we’d be booking flights to Kiev or heading back to Baku, rather than nipping over the Oresund Bridge to Copenhagen (and perhaps onward to Jutland and Herning).

The simple answer for the EBU is to release all the split votes at a country level. Without this transparency it’s going to be very hard to convince some parties that everything is on the level and above board when the results do not meet the expectations that have been built up over the years.

Show Us The Numbers

During the EBU press conference in Malmo, we learned that the votes of the jury and the televotes would not be released individually. Events Supervision Sietse Bakker reiterated this week that “…the decision not to reveal the detailed split results is a fine balance between transparency and protection of fairness in the future.” That doesn’t mean we’re not going to do some digging and try to work them out ourselves, although it’s going to take some time and explanation.

RAI, the Italian host broadcaster, has graciously released their televote figures (PDF link). Aside from the 21 per cent of spoiled votes during the Grand Final (we assume this is Italians trying to vote for Italy, ‘Viva Italia!‘), our friends in Rome have given us some data – real, actual data – from which to extrapolate how the new aggregate voting system might have worked.

It is, however, incomplete data, so there will be some estimation and deduction involved. Exploring this data has highlighted my limitations for conducting such analyses: I am not a mathematician or statistician. Rather, I’m a social researcher who uses statistics in my research, but who relies on statistical consulting for analyses requiring anything outside of my core toolkit. I suspect others might be able to program this analysis and get the same answer in mere second. It took me about an hour. Or two.

marco_mengoni_italy_2013

There’s Something About Mary

Nothing Stays The Same Forever

After the Eurovision Song Contest left Baku, we found out from some host broadcasters the top 10 jury and top 10 televote rankings. These were allocated 12, 10, 8, 7 points (as per the traditional ESC Borda count), added together, and ranked from most to least points. The entry with the most points got the eventually douze points; in the event of two songs having the same aggregate score, the one with more televote support was ranked ahead of the other.

I call this the Borda-unBorda-reBorda system. More complex to explain than necessary and each shift between Borda count versus raw scores will diminish the accuracy of the result. Not horribly so, but in ways that could make a difference.

This year each participating country did something that created more granularity, and almost certainly more representative. Each jury ranked the entries in both their voting semi-final and the Grand Final from first to last. The results from the televote were also ranked from first to last. Then these scores were added together to produce a raw score. In this instance a lower score is better: finishing 1st is better than 15th, so a raw score of 8 is better than one of 20. These raw scores are ranked from best (lowest) to worst (highest), and the top 10 are then given the traditional ESC Borda score. Again, if there’s a tie in raw score, the song with the higher televote ranking gets put ahead.

jko_scores_eurovision2013

It was looking quite good for the UK at this point…

Welcome To The Grand Final Jungle

With twenty six entries in this year’s Grand Final, the jury and the televote both generated a list ranking the songs  from 1 through 25. Therefore the best possible score is 2 by placing first in both the jury and the telvote ranking; and the worst possible score is 50, were a song to finish 25th in both rankings. If two songs have the same aggregate score, the one with a higher televote score is ranked higher.

In a Eurovision Song Contest of perfect harmony, where the juries and televoters agreed about every entry, the ideal aggregate scores for the top 10 would be 2 (1+1), 4 (2+2), 6, (3+3), 8 (4+4) etc., through to 50 points for 25th place (25+25). This would be the ideal mapping, and we’re going to come back to this in a moment.

Thanks to RAI, we have the complete rankings for the Italian Grand Final televote, and we know the final point allocations for Italy.

There were a few obvious outliers. Romania won the televote but only got 1 point on the final scoreboard for an overall tenth place. Spain was only 16th in the televote, but got 2 points for an overall finsih of ninth. Lithuania was 20th in the televote and got  6 points for a fifth place finish.

Clearly there’s a marked gap between what the juries and televoters thought; in particular, the jury must have ranked Lithuania much higher than the televoters for it to be ranked fifth overall. At the same time, Norway was 12th with televoters, but third in overall points—also indicative of a higher televote score. Conversely, Russia were 5th and Azerbaijan were 9th with the televoters: neither got any points from Italy, which indicates the jury ranked them both very low.

All of these things help us put the puzzle together.

farid_mammadov_azerbaijan_2013

Rob Brydon’s Man In A Box goes to Eurovision

The Jury And The Televote Did Not Agree

Whilst I worked with the entire rankings (one through 25), the following table shows the televoting top 10 from Italy. It shows the televote rank, estimated jury rank (calculated by me), and estimated final raw score. Ideal refers that mystical perfect Eurovision, where the televoters and juries rank all 25 entries exactly the same.

The final rank is the ranked order the top 10 for points. It is derived from the official final score, as indicated in the final voting table provided by the EBU.

  Televote Rank Jury Rank Raw Score Ideal Final Rank Final Score
Denmark

6

2

8

2

1

12

Malta

7

4

11

4

2

10

Norway

12

1

13

6

3

8

Greece

4

10

14

8

4

7

Lithuania

10

5

15

10

5

6

Ukraine

3

13

16

12

6

5

Moldova

2

15

17

14

7

4

Hungary

8

11

19

16

8

3

Spain

16

3

19

18

9

2

Romania

1

21

22

20

10

1

Russia

5

18

23

22

11

0

Georgia

11

12

23

24

12

0

Iceland

19

6

25

26

13

0

Azerbaijan

9

17

26

28

14

0

Belgium

21

7

28

30

15

0

Armenia

14

16

30

34

17

0

Sweden

23

8

31

32

16

0

Netherlands

13

19

32

36

18

0

France

25

9

34

38

19

0

UK

24

14

38

40

20

0

Belarus

17

22

39

42

21

0

Ireland

15

25

40

44

22

0

Germany

18

23

41

46

23

0

Estonia

22

20

42

48

24

0

Finland

20

24

44

50

25

0

The results from the first Semi-Final televote (in which Italy votes) also help. Russia (2nd), Belgium (10th),  and Netherlands (11th)  were ranked decently by Italy’s semi-final televote, but ranked 5th, 21st, and 13th in the Grand Final. Belgium, in particular, dropped a fair bit. Regardless, none of them got any points from Italy.

Here’s the Italian televote ranking comparing songs in both the first Semi-Final and Grand Final:

Italy Semi-FinalTelevote Rank Final Televote Rank Sem-Final Points Final Points
Ukraine

1

3

12

5

Moldova

2

2

8

4

Lithuania

3

10

10

6

Denmark

5

6

6

12

Russia

6

5

4

0

Belarus

7

17

7

0

Belgium

8

21

2

0

Netherlands

9

13

1

0

Estonia

11

22

5

0

Ireland

13

15

0

0

Only four entries – Ukraine, Moldova, Lithuania and Denmark – scored points from Italy in both the semi-final and Grand Final. Denmark only scored 6 points in the semi-final, but earned 12 points in the Grand Final. But its televote ranking was about the same in both events. That means the Italian jury on Saturday night ranked Denmark much higher than it did a few nights earlier. With twenty five rather than fifteen entries on Saturday night, this seems rather strange… Unless Denmark’s jury final performance on Monday night was markedly inferior to their subsequent one on Friday night.

For the remaining six entries only Russia was in the televote top 10 on Saturday night – yet they received null points. That means the juries must have placed several songs ahead of Russia. Conversely, Russia’s semi-final televote rank (6th) and scores for the semi-final (7th place, 4 points) are close to the ideal, and imply the jury broadly agreed with the ranking of the televote.

Broadly speaking, aside from a handful of entries that were ranked in the top 10 for both the jury and televoters in Italy, the results don’t match up to the ‘ideal’ very well, leading to some rather marked omissions from the score table after the Italian top ten was calculated.

cezar_romania_2013

Run away, nymphs, run as fast as you can

Arithmetically it’s clear that it all works, and the scores add up. But Eurovision isn’t just about the numbers, it’s about fair play and emotion. It’s very hard to justify a situation where a song topping the televote walking away with a single point (in the case of Italy’s votes for Romania), or a second place song in the televote not scoring anything (in the case of Azerbaijan’s votes for Russia).

The Simple Answer To All This Lies With The EBU

Although the traditional 12, 10, 8-1 points are still being used, this year’s scoring for the Eurovision Song Contest is a radical departure from previous years. The changes have not been communicated well and there is a level of confusion and recrimination in the air. This is not good for the brand of the Contest, and the longer these questions go on, the more damage the brand will suffer.

The situation has been ongoing since the end of the Contest, five days ago. It is time for strong leadership and a decisive move to clear up all these questions once and for all. Announcing that Digame are satisfied with the result, and that Price Waterhouse Cooper are also happy with result is not enough. Distribtuing a ‘combined jury score table’ will not be enough. Trying to hide away the countries that did not reach the voting threshold in an attempt to negate ‘power voting’ will not be enough.

What is needed is sunlight. The EBU hold the data for all the televotes and all the jury votes. Individual broadcasters are releasing some of that data to the public, which allows the media to partially reconstruct the Contest scores.

Every year The X Factor in the UK release all the percentages of the public votes for the entire contest. If publishing the data is good enough for Simon Cowell, it should be good enough for Jon Ola Sand.

Release the data.


Appendix A: How The Table Was Worked Out

Lithuania, Denmark, Greece and Norway were the code breakers here. In the Grand Final, Lithuania finished 10th in the televote, but were 5th overall; Norway were 112th in the televote but finished 3rd overall. Lithuania were two rankings behind Norway (3rd versus 5th) in the final score. Greece’s final score was between them in 4th place: Greece was well ahead of Lithuania and Norway in the Grand Final televote (4th versus 10th and 12th). Norway must have beaten Lithuania and Greece because of its higher jury rank.

Denmark and Malta, however, were ranked ahead of Norway in terms of raw scores. In terms of Grand Final televote they were ranked 6th and 7th compared to Norway’s 12th. That means Norway must have been ranked higher than either of them with the juries.  If Norway won the televote, their raw score is 13: 12 for televote and 1 for jury. In that scenario Denmark and Malta each would have earned raw scores of 12 or lower (even if they tied Denmark is ahead because of their televote ranking). So their lowest jury rankings were 6th (Denmark) and 5th (Malta).

Greece, however, was 4th with televoters and ended up 4th overall – which aligns nicely with the ideal. If Norway was 3rd overall with 13 points, Greece’s score is 14 or higher—because Norway was well below Greece in the televote they cannot have the same raw score. Turns out a raw score of 14 for Greece cracked it.

We have two more outlier: Russia and Romania. Russia finished 5th with the televoters, but got zero points. Therefore the aggregate score of Russia has to be at least one point below the  raw score of the 10th ranked entry: Romania.  Our 10th song is Romania, who won the televote. So we know its score is 1 (televote) plus something else. Russia’s score must therefore be equal to or lower than Romania’s.

At this point it became a substitution and sorting game: try entering a ranking, sort the spread sheet by raw scores, and see if it lines up with the final votes from Italy. In this exercise getting the top 10 right is not quite good enough: we needed to be reasonably confident that nothing from middle of the televoting table was ranked surprisingly high by the Italian jury.


23 Responses to “Deconstructing The Eurovision Results Through An Essential Analysis Of Italy’s Voting”

  1. Peter says:

    Nice article.
    And according to escxtra, we now have question marks over Montenegro’s votes to Ukraine as well.

    Full transparency for the voting is the only sensible option if they want us to trust their results, and trust that certain country’s alleged attempts to game the system are being handled correctly. This would mean that they would have to release exact voting figures as well the number of votes that Digame disregarded through “power voting”. For example, did the supposed missing Russian SMS votes from Azerbaijan get lost through vastly different phone votes, or did Digame filter them out because they were considered fraudulent?

    Of course, this might help afore-mentioned countries improve their attacks for next year, but it would also make it painfully obvious to everyone else that it was happening, instead of hiding behind a vague press release and hoping no one notices. If they have confidence in their system, release the data and prove it. If they don’t have confidence, either do something to fix the problems or put someone in charge who can, and then release the data.

    Unfortunately, the clue was in your article:
    “It is time for strong leadership and a decisive move to clear up all these questions once and for all.”
    That’s the EBU stuffed then!

  2. AJD says:

    At the risk of asking a stupid question – what is the voting threshold rule? I’ve read it in several places but never found what it means:

    “Trying to hide away the countries that did not reach the voting threshold in an attempt to negate ‘power voting’ will not be enough.”

  3. Ewan Spence says:

    THere are certain targets that need to be passed to allow a televote to be seen to be representative, although the exact criteria have never been released. For example there may be a requirement to have X number of votes cast, a minimum number of votes for each country, or other areas.

  4. Matt says:

    So, if for example. One country has a unusually high vote, but everyone else has trigged the threshold but is about 10 to 20% above the threshold, that result will be thrown out as it shows signs of possible voting fraud. As a result, the jury vote is left to be the country’s points on the night.

    For the purpose of this example:

    1000 votes are needed to trigger the country’s threshold.

    If everyone has anywhere between 1000 and say 1200 (I’m just making a rounded number for this exercise). Then one country has 10,000 votes. Would the tele-vote be discredited and the only the jury vote be used??

    Have I got the general idea or am I right off the mark??

  5. Eric Graf says:

    “It’s very hard to justify a situation where a song topping the televote walking away with a single point (in the case of Italy’s votes for Romania), or a second place song in the televote not scoring anything (in the case of Azerbaijan’s votes for Russia).”

    Oh, I disagree with this. There’s no overriding reason that anyone should get points for anything. It’s a contest. There are rules.

    You have to draw a line in the sand somewhere. Yes, it was drawn differently this year. The participants may only now be realizing HOW differently. But they knew the rules when they signed up.

    I do agree that the full rankings, jury and televote, by country, should be released, to prove that the rules were followed. If all these kerfluffles are as big as the press is making them out to be (a big if), then they sure do need to do something.

  6. Roy van der Merwe says:

    I have to study this more carefully. I think the best will be to draw a lot, one year we do ust televoting and one year just jury voting and nobody knows ahead of time which result will be used.

  7. Philip Newton says:

    Very interesting article, and really highlights the difference this year.
    I agree that the countries may not have really realised HOW different the scoring was going to be, and are just realising.
    Full transparency is the only way forward.

  8. Robin Tremmel says:

    This is a total farce! Here is the petition that floating around FB to get ALL the jury votes published from EVERY country..
    Please sign it-we need to get these results out.

    https://www.change.org/en-GB/petitions/european-broadcasting-union-release-full-split-results-for-all-countries-in-eurovision-2013?utm_campaign=share_button_action_box&utm_medium=facebook&utm_source=share_petition

  9. Robin Tremmel says:

    Britains average jury age 62! Tony Blackburn on the jury FF sake! I can hardly believe it! Lindsay de Paul-It was like the Bonnie Tyler fan club-come to think of it-she is younger than them :)

  10. Peter says:

    @Robin: Where did you hear Lynsey de Paul was on our jury? She wasn’t one of the names Norton read out in his commentary:
    Rita Campbell
    Natalie Green
    Melissa James
    Tony Hatch
    Tony Blackburn

    The two Tonys are 70, but as far as I can tell from a quick google, the other three are much younger, which would make the average age much lower than 62 as well.

    Tony Blackburn even tweeted a photo of all 5 of them!
    http://ow.ly/i/29e6m

  11. Robin Tremmel says:

    My mistake-the average age is about 59.Blackburn and Hatch are at the cutting edge of pop right enough..

  12. John Egan says:

    It’s difficult to precisely say Matt. In recent days the EBU has claimed Digame’s system looks for bulk voting from clusters of mobile phones and excludes those votes. But how they would differentiate the cultural affinity/diaspora voting coming out of the Balkans versus clusters of people in Lithuania paid to vote for Azerbaijan (allegedly) is difficult to see. Unless this explains why none of the ex-Yugos got out of this year’s semi-finals: thousands of televotes in some countries were excluded as bulk votes.

    When Monaco were in the Contest a few years back they never registered enough televotes. Apparently San Marino is a jury only result, because they don’t have their own country code for telephones–they piggyback on Italy. Any televotes cast in San Marino in the Grand Final count towards Italy’s. Which seems…inconsistent.

  13. Robin Tremmel says:

    Peter-my mistake-I got the info off a comment on Eurovision times.I should be more careful with my sources next time.I still think the British Jury looks dodgy to say the least..

  14. Peter says:

    Why do you have to be at the cutting edge of pop to understand music and work out the quality of a song and the staging?

    It’s not the job of the jury to say “I could have a bit of a boogie to Cascada, let’s give her 12″, it’s their job to score the quality of the writing, or how well the performers can sing the song, how vocally challenging is the song, or what the overall performance is like.

    So what if the two Tony’s are 70, they have a lot of experience in the industry and will know how to pick apart a song, even one written in the modern era and aimed at a younger audience. And that’s before you consider the other three jurors. For example, what part of Rita Campbell’s CV makes her sound unqualified for the job, exactly?
    http://www.ritacampbell.co.uk/about/
    She’s written songs for a wide-range of different artists in different styles, she’s also a singer, and a vocal coach. She seems like a perfect person to put on a jury to me.

    Don’t forget that the EBU rules state the following regarding the juries:
    “There shall be a balance among the members of each National Jury to ensure sufficient representativeness in terms of gender, age and background.”
    http://www.eurovision.tv/upload/press-downloads/2013/Public_version_ESC_2013_Rules_ENG_FINAL.pdf
    Judging by the names on the list and Tony Blackburn’s photo, it sounds like the BBC got it spot on. And this is from someone who normally has a problem giving the BBC credit for most of their Eurovision choices!

  15. Z. says:

    Eric, concerning the contest itself I agree with you. Concerning the televoters paying money for their votes I think the new rule is questionable.

    Robin, the German jury was 26 years on average … Maybe that’s why we didn’t give any points to Anouk.

  16. Robin Tremmel says:

    A lot of people didnt like Anouks song Z.I personally think she has a great voice but no way was that a eurovision song-she should have gone with something more like nobodys wife.I think she was fortunate to get 9th.
    I would like to see the results of the British jury-as well as all the others.The Italian jury was all men-thats not even in the rules.

  17. Michael UK says:

    The new system was brought in to stop diaspora / neighbour voting which was becoming a farce . Italy always gave romania 12 due to Romanians living there . That was clearly unfair and it happens in many countries . The jury would have voted for Romania if the song had been good enough

  18. Michael UK says:

    In recent years Ireland has given 12 points to Lithuania but we all know its not Irish people voting , it’s the immigrant Lithuanian population, and for the same reason turkey has a 48-60 point head start . Whilst its understandable to want to vote for ones homeland it’s clearly unfair on the other countries competing . Something had to be done . As a Brit I wouldn’t want to vote for the UK if I was living abroad , I’d feel a bit of a cheat I d rather win fairly or not at all !

  19. Robin Tremmel says:

    Ireland always gives points to the UK and vice-versa.In fact ireland has given the most points to the UK than any other country.They obviously think our songs are fantastic :)
    We give (the uk) the most votes over the years to Ireland.Its not really much different to what the romanians are doing now in Italy.Also Italian is very much like Romanian-they are very close-so I suppose some Italians are voting for Romania for that reason also.
    The problem for Britain and Ireland is we dont have more neighbours.Maybe Tony Hatch knows a good tune about this :)

  20. michael UK says:

    @robin
    the italian .spanish and portugeuese votes for romania and moldova are very consistent and predictable and i dont think its to do with the language its diaspora, and ireland hardly ever give the uk 12 points , this year was a rare event , the uk giving less to ireland than they gave us.

  21. Rob says:

    Blackburn & Hatch – are you kidding me? Well qualified? Yes, they really have their finger on the pulse of music in 2013.

  22. Mariska says:

    Hi John, Nice analysis, thanks. I just don’t get how you calculated the jury results for all the countries that ended up getting no points in the final. Could you explain?

  23. [...] ESCIn­sight: Deconstruc­ting The Euro­vi­sion Results Through An Essen­tial Ana­ly­sis Of Ita… [...]

  24. John Egan says:

    Thanks Mariska. The bottom of the table is the hardest to figure out, in fact.

    To a certain extent I only tried to figure out the highest Romania might have been with the jury and still end up 10th overall (with a single point)–despite winning the televote.

  25. [...] zwischen Jurys und Publikum gravierende Auswirkungen zeitigt, wie John Egan von ESC Insight am Beispiel Italiens ermittelt hat, wo der Sender das Ergebnis des Televotings veröffentlichte. [...]

  26. [...] nicht nur die Plätze 1 bis 10 der Jury und des Televotings, sondern alle Ränge. Auf der Seite escinsight.com wird es gut erklärt: Früher war es so, dass der Rumäne auf jeden Fall sehr viele Punkte aus [...]

  27. [...] way the jury and televote scores were combined. There’s a (lengthy-ish) discussion of this on ESC Insight so I won’t repeat it all here, but essentially rather than just the top 10 televote songs and the [...]

Leave a Reply