Why the polls didn't expect Brexit to happen, and what could have been done
By Taran Volckhausen
Nine months after voting in a furiously contested referendum, the United Kingdom has now formally started the process of leaving the European Union. Theresa May, the British Prime Minister, has authorized triggering Article 50 of the Lisbon Treaty, which will ultimately lead the UK out of the EU within two years' time.
"This is an historic moment for which there can be no turning back," May said to the British House of Commons. "Britain is leaving the European Union."
Brexit referendum result upset expectations
A year ago, few experts would have expected for the UK to be leaving the EU like it is today. A poll of UK voters on the day of the election by the country's second largest market research organization, Ipsos-MORI, predicted that Remain would win over Leave by a margin of 8 points with 54% to 46% in favor in the EU. Another polling company, YouGov, also predicted victory for Remain albeit with a smaller 4-point lead of 52% to 48%.
Following the announcements of these two polls on the day of the Brexit referendum, the value of the pound went up to $1.50 USD, setting a high for 2016. Within hours, however, the results of the vote would come in with punishing effects for the value of the pound, and the political fortunes of many leaders in British Parliament, including former-PM David Cameron.
While most British pollsters said the results would be close, they predicted a narrow victory for Remain. The UK voters went on to shock the pollsters by voting to leave the EU by a margin of 52% to 48%, with the Leave side receiving over a million more votes than Remain.
Why did the pollsters get it wrong?
While there have been various theories pushed forward about why the pollsters failed to accurately gauge public opinion in the months and weeks preceding the Brexit vote, one reason that stands out is the reliance on many of the traditional polling companies put on phone interviews.
Looking at polls throughout the campaign, it became clear that results deviated based on the medium that pollsters used to conduct surveys. For instance, internet poll averages pointed to a 1% lead for Leave, while phone interview polls showed Remain leading by 2.6% on average.
At one time, random-dial phone interviews were considered the gold standard of polling practices. However, with the rise of mobile technology and declining response rates, many polling companies have gravitated toward online polling.
Online polling has risen as another option. However, there isn't a random way to select voters on the Internet because participation is more voluntary than receiving a phone call. These changes in technology led to a dilemma for polling companies in previous elections.
How could public opinion tracking be done better?
The internet has completely transformed the way citizens communicate with each other as a society. Between millions of articles and billions of social media posts that are created every day, there is an enormous amount of information being produced online.
By using sentiment analysis, which relies upon natural language processing (NLP), machines can now read and derive meaning from the information that is being created online. Through machines, millions of articles and posts that are uploaded every day to social media sites like Facebook and Twitter can be analyzed in real-time and conclusions can be drawn.
For example, in the weeks leading up to Brexit, there was a great deal of people talking to each other about the topic on Twitter. In fact, there were 54 million tweets about Brexit in the six weeks before the vote. Contrary to polling data, the #EURef Data Hub, a partnership between the Press Association and Twitter, showed that the Leave side was leading Remain throughout the campaign.
As traditional polling methods continue to suffer setbacks, the advantage of analyzing data from social media is becoming more relevant, and is making it easier to get an accurate picture of public opinion.
Now that Brexit has come to pass, it's still important to collect information on how it will continue to affect the British economy and the public. For instance, Article 50 will almost certainly impact consumer confidence in the U.K. and the Eurozone.
Instead of waiting for the British consumer confidence index to be released, the finance industry can use a sentiment analysis tool like the Vector API to sample consumer and producer opinions on Brexit across multiple social media channels.
Vector is a natural language processing application that performs information extraction on millions of news stories per day. It provides high value to any quantitative researcher, adding a collaborative-authoring workflow in perfect synergy with the most powerful and unique faceted search in the business. For more information, please visit www.indexer.me email@example.com.
Vector website: http://www.indexer.me
Press kit: http://www.indexer.me/media-kit.html
Social Media: LinkedIn@Indexervector, @indexervector, @indexerme, @jofletcher
Indexer is a tech start-up in the artificial intelligence space and has a focus on computer vision and natural language processing technologies.