More unboring methodological analyses

Alright, Ontario, we’re on day 16, 13 more to go. And also, happy Valentine’s Day! While consistency might be a beautiful thing in love, it’s boring as hell in politics. The horserace remains stable: PCs 49%, OLP 24%, NDP 19%, and Greens 8%. 

If you voted in yesterday’s poll, thanks! I’ve listened and will dig into why my polls are a bit more bullish on Ford relative to some other pollsters. Much of this will be an echo of the analysis I did regarding the volatile federal polling we’ve been seeing. 

I’ll start with a mea culpa. There’s one methodological choice that I might live to regret, and that’s limiting my reporting to the 4 major parties and omitting an “other” vote choice. I find respondents often over-report that they will vote for an “other” party. For example, in other polling, I’ve seen the “other” category be anywhere from 0-6%. There was a fairly high “other” vote in 2022 (5.5%), but typically, it’s much lower (ex. 1.7% in 2018). This slightly inflates my vote for each of the 4 parties, but not by a lot. Again, I might regret this, but we’ll see! 

To offer an apples-to-apples comparison, I’ve allocated the vote (removed “other”) from other pollsters' results. So, am I high on Ford? It depends. In the first week of the campaign, Innovative Research, Ipsos, Léger, and I had the PCs leading by pretty much the same margin (25-27%). Angus Reid had their lead at 18%, Mainstreet at 12% and Liaison Strategies at 13%. As you well know, my polling has shown a very stable race. My average PC advantage last week was 26%, and it’s 25% this week. Other pollsters are also finding a stable race, but it’s stable to their baseline. 

A lot of this points back to what I laid out in my federal analysis: mode effects. The pollsters I pointed to that show similar PC advantages to my own are all polling online. You can see our collective trend line below. The highlighted PC numbers are mine. We see a bit of volatility but a pretty consistent trend line. The PC numbers hover around 50%. Notably, no one has the Liberals above 30%. We also see a tighter 2nd-place race between the Liberals and NDP. 

Next, we have IVR polls. These pollsters are finding a tighter race between the PCs and Liberals, with the Liberals mostly in the low 30s. I say they’re finding a closer race, but not a close one. The PCs would still win, but there would be a big difference in seat count. 

Our only live-caller phone pollster, Nanos Research, has a pretty consistent trend with the IVR polls. The PCs are at 45%, the Liberals at 32%, and the NDP at 17%.

So why are we seeing differences? I don’t know for sure, but I have hunches.

I have used all of these polling methodologies before, and know there’s one area where online polling performs very well, and that’s in reaching younger audiences. Below is my sample frame from today’s release. In any polling, you want to weigh your data as little as possible. Weighting is extremely important to ensure you have a representative sample, but ideally, we’re getting pretty close to our targets through our data collection alone. You can see that my gender, age, intersecting gender and age, and regional targets are pretty much bang on. The one area of concern, and this is true of any polling methodology, is it’s easier to reach higher-educated voters, so that has to be accounted for in weighting.

In my experience, phone polling, whether IVR or live-caller, tends to have a hard time getting young people to participate. That means you’re weighing more heavily on fewer younger voters and weighing down your older sample. I don’t know if that has been the case for these other pollsters and I really don’t want to be nitpicking their individual methodologies.

And as I mentioned in my federal article, we could be seeing some non-responses bias where certain types of voters aren’t answering polls. To be fair, this could happen for either phone/IVR or online polls. We’ll simply have to wait and see how things evolve over the next 13 days.