Derry Girls

Derry Girls & Data Gathering Methodologies

The divisions in Northern Irish society are brilliantly lampooned in a now famous scene from the hit Channel 4 comedy Derry Girls when a group of teenagers are asked to explain the differences between Catholics and Protestants. ‘Protestants keep their toasters in cupboards’ and ‘Protestants hate Abba’ are among the nuggets from the now iconic scene. In the last while another division may have emerged in what is a similarly obscure domain: data gathering methodologies.

Opinion polls and surveys on constitutional matters have particular importance in Northern Ireland due in part to the vague wording of the ‘Border Poll’ provision of the Good Friday Agreement;

The Secretary of State shall exercise the power [to call a Border Poll]  if at any time it appears likely to him (sic) that a majority of those voting would express a wish that Northern Ireland should cease to be part of the United Kingdom and form part of a united Ireland.

Victims campaigner Raymond McCord took a legal action in 2018 seeking clarification of this clause. McCord wanted to know the exact criteria that would determine whether or not a poll was to be held. He lost the case. Nonetheless, an outworking of the McCord case, is a general acceptance that election results, the Census, Assembly votes and opinion poll data are among factors that would be considered by the Secretary of State when deciding whether or not to call a vote. Hence, the particular significance of opinion polls.

Party support &Census?

Party support in elections may seem the most obvious barometer of whether a ‘Border Poll’ should be called. But allegiance can be fickle when it comes to referendums. In both the UK (Brexit) and the Republic (Nice and Lisbon Treaties), there is very recent experience of referendums where voters did not vote along party lines. In an analysis of the Northern Ireland parties, John Coakley argues that supporters of the two main unionist parties remain ‘rock-solid’ in support of the Union (97% in the case of the DUP, and 93% for the Ulster Unionists), but Sinn Féin (82%) and SDLP (60%) are less consistent supporters of Irish unity. So, party support alone may not be the ideal gauge.

The Census is an alternative data source. The 2021 Census will be closely watched to see if those from a Catholic background outnumber those from a Protestant background. As the Derry Girls informed us, ‘Catholics are Irish and Protestants are British’, so this should be a good gauge? However, a new question on nationality was introduced to the 2011 Census that paints a complex picture when it comes to identity. The 2011 data revealed that 52% of Catholics identified as Irish, 27% identified as Northern Irish, and 11% identified as British. Meanwhile, 66% of Protestants identified as British, 15% as Northern Irish, and 11% as British and Northern Irish.  Questions on religion and nationality in the Census are simply not specific enough to fully ascertain opinions on the constitutional status of Northern Ireland. There was a suggestion from Mitchel McLaughlin that a Border Poll-type question should be included in the Census. However, there is a particularly contentious history with regard to the Census in Northern Ireland. More generally, there is a reluctance from census collectors to include attitudinal questions. There will certainly be no constitutional question on the 2021 Census, but it will be interesting to see if there is pressure to include one in 2031.

Online Polls show higher levels of support for unity

This brings us to opinion polls. There tends to be only a small number of such polls per year in Northern Ireland. When new data is released, it can dominate the local news cycle for days. The most recent poll, against the backdrop of Covid and Brexit, attracted considerable coverage in Britain and the South . This Lucid Talk Poll for the Sunday Times found a majority (50.7%) in favour of holding a ‘Border Poll’ in the next five years . On Irish unity, the results were close (42.3% in favour of Irish Unity, 46.8% wanting to stay in the UK). These results were similar to another Lucid Talk Poll last year. Moreover, in September 2019, a poll from former Conservative party peer Lord Ashcroft found a majority in favour of Irish unity. All the aforementioned polls were mostly based on online data collection. In terms of predicting recent Northern Ireland election results, this methodological approach has proven sound. However, after the the latest Lucid Talk poll, Arlene Foster speaking to Sky News cautioned that online polls always come with a ‘health warning’.  Similarly, David Trimble when pressed on the poll pointed towards the results of a face-to-face based survey, the Northern Ireland Life and Times Survey (NILTS).

The methodogical debate- do ‘shy nationalists’ dislike face-to-face surveys?

The NILTS’ latest report (based on fieldwork from September 2019 to February 2020) found just 22% in favour of Irish unity. Similar findings were reported in other face-to-face interview surveys conducted as part of a Northern Ireland General Election Survey. Like the NILTS, this study was an academic-led exercise with Jon Tonge of University of Liverpool Lead Investigator. Here, interviews found that only 29% of respondents said they would vote for a united Ireland, with 52% saying they would vote for Northern Ireland to remain part of the UK.

It is pretty clear that online polls find more support for unity compared to face-to-face surveying. Of course, the divergent findings have led to pretty heated debates in the online world. University College London’s Working Group on Unification Referendums has produced a really worthwhile report on the mechanics of a Border Poll that includes a neat summary of the arguments relating to methodologies;

There has been much speculation as to the reasons for this divergence. One view is that traditional surveys may underestimate support for unification because of what is sometimes called as a ‘shy nationalist’ effect. On this view, some people are reluctant to ‘admit’ to a stranger in person that they would like to see Irish unification. It is indeed true that interview-based surveys yield many more ‘Don’t know’ responses than do online polls. On the other hand, people do seem willing, in the same surveys, to acknowledge that they support nationalist parties or hold strong views on other contentious matters.

Conversely, others are concerned that respondents to online polls may be unrepresentative of the wider population. Such polls draw from ‘panels’ of people who sign up to complete surveys by particular companies. The people choosing to do this may be unusually engaged citizens: that may be a reason for their lower ‘Don’t know’ response rates. Particular groups may also be over- or under-represented on the panels. On the other hand, online polling companies weight their results to compensate for any over- or under-representation, and have methods to protect against large-scale ‘gaming’ of the process.

The report’s conclusion on the matter is too worth quoting;

The overwhelming weight of expert scholarly opinion regards interview-based surveys based on probability sampling as the ‘gold standard’ for social research. Importantly for the current case, this approach is least susceptible to the risk of manipulation by anyone who might wish to skew the result. Furthermore, the danger of a ‘shy nationalist’ effect under interview-based surveys can be addressed. All of this suggests that a Secretary of State would be well advised to give greater weight to rigorously conducted interview based surveys than to online polls when judging whether a referendum under the Northern Ireland Act might be required. Nevertheless, other types of polling (such as online polls) that have been proven to accurately predict voting behaviour should also be given serious consideration as a means of assessing voting intention.

More data please!

Out of all the online discourse on the issue over recent years, one nugget of wisdom emerged that has largely been overlooked. Peter Donaghy, on Slugger O’Toole, observed in 2019 that since the EU referendum, there have been just nine polls on the question of a Border Poll. Over the same period, there were 56 polls regarding Scottish independence. There simply isn’t enough data being gathered. As was confirmed in the McCord case, the Secretary of the State can fund opinion polls and surveys on the subject. This is something that can happen quite quickly and in some way plug the information gap.

Of course, questions on storage habits with regard to toasters may have to be made optional, given the obvious sensitivities.