Press play to listen to this article
If a general election was held in your country today, how would you vote?
Every week polling firms across Europe ask this question to a sample of their national population to measure support for each of the political parties. Some call people on their landlines and cellphones, others conduct face-to-face interviews or use online panels.
The results of those contacts — taking into account sample size and who answered the questions — can then be extrapolated to estimate national opinion. But even if this weighting is done carefully and correctly, any single result can be misleading if interpreted in isolation. Projecting from one sample taken with one method to estimate the intentions of a whole country is fraught with pitfalls. That’s where POLITICO’s Poll of Polls comes in.
To boost statistical reliability, we aim to amalgamate all well-conducted polls on a specific question into a single estimate of national political sentiment. Combining the result produces a more robust estimate of the actual state of opinion, diluting the impact of outlier results and showing trends more clearly.
You can see the results for each European country here. Every data point in each chart represents one result from one poll, giving readers a sense of the variance that comes with polling data. We’ve applied the approach not only to general elections but also to leader approval ratings and other polling conducted regularly on the same subjects, for example on the question of Scottish independence.
Which polls do we include?
There are two major requirements that polls must meet in order to be included. The first is transparency: A poll must at a minimum include information about how the poll was conducted; when the fieldwork was done; who conducted it; who commissioned it and paid for it; and the sample size.
Second, more crucially, all polls must be based on a sample of people that is under the polling firm’s control, which ensures it is as representative as possible of the population the firm wants to draw conclusions about. That means surveys on social media or on websites where you can click and vote as often as you want are not polls. The statistician Nate Silver has dubbed these online surveys “clickers†to distinguish them from the polls that are relevant for political analysis.
A poll can be representative if every individual in the population has the same chance to be polled. That’s the case in theory in random digit dialing — although polling companies encounter difficulty when trying to reach people without landline phones. Another option is to weight the answers based on the demographic characteristics of the sample and the overall population.
The size of the sample alone does not make a poll more or less representative, but a larger sample size reduces the uncertainty by providing more information. A rule of thumb is that a good poll should have a sample size of at least 800 people according to the Association of Market and Opinion Research Institutes of Austria. The Poll of Polls database does include polls with sample sizes smaller than that, but they get weighted down, and there are relatively few polls with such small samples.
As long as polls meet the minimum threshold of transparency and a track record of published polls, we include almost all polls and let our statistical model take care of weighting polls with larger sample sizes or those conducted more recently. Polls commissioned by political parties for internal purposes, which are sometimes leaked to the public, are not included as they usually do not meet the transparency requirements.
How do we combine the polls?
Using a statistical model called the Kalman filter, we aggregate polls using the last election result as a starting point. From that we take every poll — along with its sample size and margin of error — as a new piece of information to plot a trend line. Larger polls with a smaller margin of error have a bigger influence on the Poll of Polls value; so do more recently conducted polls compared to those that are older.
Kalman filtering is a statistical technique used to combine a series of measurements taken over time to narrow in on a more accurate estimate. It is often compared to the GPS localization in navigation apps on smartphones: When you open the app, your position is uncertain, indicated by a large circle around your presumed location, but as more GPS signals come in, the position becomes more reliable and the blue dot moves closer to your actual location.
Can the Poll of Polls model correct for polling biases?
Yes, to some extent. By pooling more data, the Poll of Polls figure is less prone to random sampling error than an individual poll. If one polling company’s methodology is subject to a systematic methodological bias, then its results will be diluted by other polls.
If several national polls have systematically under- or overestimated support for one party (as has been the case in some recent elections, such as for the League in the 2018 Italian election or the Austrian People’s Party in the 2019 election) then the resulting Poll of Polls will be off beam too. But after tracking and aggregating polls for more than 30 national elections since Poll of Polls was founded, we can be confident that amalgamation is more reliable than any single polling series.
How accurate is it?
Every poll comes with a sampling error, not least because the entire population isn’t polled so the polling result will never exactly match the actual result. But in order to taste a soup you don’t have to empty the pot — a representative spoonful is more than sufficient to assess the taste. In fact, the sampling error turns out to be reliably stable: Studies suggest the average historic polling error (by how much any individual poll misses the actual election result of a party) lies between 2 to 2.5 percentage points. An analysis of more than 30,000 national polls from 351 general elections in 45 countries between 1942 and 2017 concluded that this polling error was also relatively stable over time. So polling is becoming no less accurate either.
In the most recent national elections in the EU, the POLITICO Poll of Polls showed its worth. While the average polling miss for individual polls in our database is around 2.3 percentage points, the average difference between the election result of a party and our final POLITICO Poll of Polls is around 1.8 percentage points.
What about the European Parliament?
POLITICO is the only outlet offering a daily updated seat tracker that translates polling trends into seats in the European Parliament — providing a barometer of the political weather across Europe.
Analyses that compare national polls with polls asking about voting intention in a European election show that in most cases there is no significant difference between the two (the U.K. was a special case where European Parliament election results were often very different to national polling trends at the time). That makes it possible to use polls that ask about national voting intentions to estimate support for the political groups in the European Parliament.
In the run-up to a European election, when polling firms start asking about the voting intentions for the EU legislature specifically, we incorporate that data into the Poll of Polls model for the European Parliament to improve accuracy. The 2019 European Parliament election proved the accuracy of our seat estimation model and until 2024 you can track the developments for the different groups in the European Parliament on the dedicated site.
Who came up with Poll of Polls?
Poll of Polls was founded in 2017 by Cornelius Hirsch and Peter Reschenhofer. It was acquired by POLITICO Europe in 2019 ahead of the European Parliament election. Additional features are available exclusively to users of POLITICO’s Pro platform, a subscription service that provides in-depth reporting, data and actionable intelligence for policy professionals. More information here.
