A word on our political polling methods...

Thanks for your queries about Horizon's party vote polling:

1.  Why are you using your internal database/self-selecting surveys, rather than random polling like everyone else?

Horizon does not conduct self-selecting surveys.

A self-selecting survey is defined as one in which anyone can choose to participate.

(Please refer to this paper from  specialist analysts at the  University of Auckland's Department of statistics: https://osf.io/pjzfb/). 

Horizon uses two of its own specialist research panels: HorizonPoll, representing the general population 18+, and the Horizon Research Māori Panel, representing the Māori population 18+. We also use other third-party specialist research panels for source diversity.

For the May 2023 survey we used three panels, including one external, operated by a reputable international market research company.

Who is using online panels:

Kantar and Curia and others also use online panels for their political polls: Kantar’s sample included 502 online in May and Curia’s 200 online. These panels could be the same as the external ones we use from time to time, though this is not publicly reported. Kantar owns a panel. 

http://www.reidresearch.co.nz/TV3+POLL+RESULTS.html. 

Of 1000 respondents, 700 were interviewed by phone and 300 online.

This means Horizon, Curia, Kantar and Reid use online panels in party vote polls. You will need to confirm what Talbot Mills does.

Careful quota sampling

Horizon stratifies its sampling to 18+, and we then quota sample to achieve a representative population sample.

Careful progressive sampling each day by various demographic criteria ensures we achieve a representative sample, including sometimes harder-to-reach segments (like those aged 18-24, and some ethnic groups). It also ensures we achieve appropriate numbers of 2020 supporters of political parties.

Not every one of our tens of thousands of panel members receives an invitation to do a single survey. This avoids one of the risks online panels might face, of people with a particular interest joining in order to do surveys on particular subjects (like politics).

We also have security systems which guard against respondents coming from the same IP addresses and/or giving identical answers to questions. This also guards against robot responses. We have not had an incidence of this at any time since the first panel was established in 2005.

To further ensure the sample is representative, we can weight on up to 6 demographic criteria. We use an internationally standard iterative rim weighting method and almost no changes were made to the May final sample by weighting. This reflects on the high quality of the sample.

2. Why are you weighting a political poll based on the 2020 election result?! That’s another significant contributing reason for the bias towards Labour and against National, right?

We regard this as an excellent practice. Horizon does this to ensure the sample does not inadvertently contain disproportionate numbers of people who vote for a particular party or parties. This reduces, not increases, the potential for bias.

Results would be less reliable if, say, in the course of random sampling, supporters for a party were over-sampled, and this was not corrected for.

In the May survey our methodology ensured we achieved representative samples of voters for various parties.

This means we can produce results on current party voting preferences. It allows us to track in detail the changes in intended vote among 2020 party voters.

Both in May 2023 and other surveys, we have tracked significant falls in support for Labour - and can determine the extent of party loyalty and for which party its 2020 voters would vote if an election were held tomorrow. For example, before the 2020 election, we tracked a 16% rise in Labour support, compared with 2017 voter support. In May we found a 17% drop in Labour support compared with 2020, with 9% of Labour's 2020 vote going to National.

For National we find a 71% loyalty rate in May, with 14.8% of its 2020 vote going to ACT.

Incidentally, we also check results without weighting by 2020 party vote. In May there was no significant difference.

Regarding your other comments:

Horizon reports intended vote by those who are both enrolled and 100% likely to vote. This is not the same as for other polls. There would be little value in appearing in polls of polls. We have declined a request from one of the major media companies to include us in its poll of polls.

Horizon's political issues surveys have an excellent record of accuracy.

One of our surveys of the flag referendum options, involving running four iterations, varied by just 0.71% from the official result:

https://horizonpoll.co.nz/page/424/most-accurat

Our final 2020 survey on the cannabis referendum found that 67,900 votes could be the majority voting against. The final result was 67,662 against, a difference of just 238 votes.

We have an excellent record of accurate research results when these can be compared with official and other publicly available statistics. These include precisely projecting the number 18+ who would get the COVID-19 vaccine and our research in areas like KiwiSaver, for example, where our results for the number of adults in the scheme match official data.

I note in your email that you have colour-highlighted our reference to conducting the party vote research in the public interest. I’m not sure if this is of interest to you. We think that doing this without charge in the public interest, not with a pecuniary one, is a good thing: One of our aims is to listen New Zealanders - and make them heard. 

Thanks for questions and for letting us know your views.

Kindest regards