A poll of Guelph’s by-election was recently conducted by an up-start Winnipeg-based polling firm KLR VU run by Allan Bruinooge, brother of Conservative MP Rod Bruinooge.

KLR VU Poll (through 13 Aug)
Frank Valeriote, Liberal — 34.6 to 38.6%
Gloria Kovach, Conservative — 24.4 to 28.4%
Mike Nagy, Green — 17.2 to 21.2%
Tom King, NDP — 15.8 to 19.8%

The above ranges reflect the margin of error of the poll. KLR VU surveyed 3,396 people on two days, July 27 and August 13, which represents a margin of error of +/- 2%, 19 times out of 20 (note: typically a survey of 3,396 people yields a margin of error of +/- 1.7%, so we assume KLR VU is rounding this to 2%).

The poll has sparked controversy due to its methodology, its perceived Conservative connection and also because it’s only other public poll — on whether Dr. He.nry Morgentaler should receive the Order of Canada or not — showed very different results from the mainstream pollsters (Ipsos and Angus Reid). David Akin has the best run-down of the controversy.

On the question of methodology, we are less concerned about the use of touchtone polling in general — where questions are pre-recorded, voters are called randomly, and asked to press ‘1′ or ‘2′ etc in response to different questions. No doubt this type of survey has some selection bias in that some people will simply hang up because there is no live person on the other end. More importantly, since there is no screening, it does not ensure that the respondent is an actual eligible voter (although realistically there is no way to ensure this with traditional polling either — essentially we have to take the person’s word for it). The selection bias is probably not that significantly different than traditional polling (a live person asking the questions), where people can also refuse to participate. At issue is whether supporters of one party/candidate are more or less likely to not participate than others. It seems unlikely — that is, Conservative voters are probably just as likely to hang up as Liberal voters as NDP voters and so on. Likewise, are people who are not eligible voters more likely to participate using touchtone polling than traditional polling? Probably not. So, we’re not prepared to discount the use of touchtone polling in general.

Somewhat more concerning in this poll is the large time gap between the initial survey and the final survey — 17 days apart. Given that polls are snapshots in time, a 2+ week gap certainly raises eyebrows at to its reliability. Typically, polls are most reliable when they are conducted over a short, contiguous period of 2-3 days. However, it’s probably worth noting that another mainstream pollster, Environics, also often conducts polls over a large period. For example, it’s latest survey was conducted June 13 to July 5, a gap of 22 days (although it is probable the surveys were more evenly distributed than the KLR VU poll). Given that Environics is an established pollster, the time gap in the KLR VU poll may not be significantly out of line with the mainstream, although it is not good polling practice.

Many NDP supporters, in particular, have questioned the poll’s validity, no doubt because the results show its candidate running behind the Green Party candidate. The NDP views Tom King as a star candidate and believes it can win the riding. However, it should come as no surprise to see the Greens doing well in Guelph — Guelph Greens won more than twice the province-wide Green share in the last federal and provincial elections (indeed out-polled the NDP in the latter), and the Greens are putting serious money and bodies into the riding. Compare the KLR VU results with our analysis from 14 August:

DemocraticSPACE Estimate (through 14 Aug)
Frank Valeriote, Liberal — 28-32%
Gloria Kovach, Conservative — 26-30%
Tom King, NDP — 20-24%
Mike Nagy, Green — 18-22%

The KLR VU results are slightly lower than our estimate for the NDP (the high end of the KLR VU range of 19.8% is just below our low end of 20%) and higher than our estimate for the Liberals (the low end of the KLR VU of 34.6% is just above our high end of 32%), but are within our estimated ranges for the Conservatives and Greens. So the results are not wildly different from our estimated ranges.

The jury is of course still out on KLR VU, given that it just doesn’t have a track record for us to judge its reliability. However, the results as reported are not completely out of line with what we might expect to see, plus or minus a couple of points for each party.

It’s important to keep in mind that this is, however, just one poll (although with a large sample size). Only with a few polls from different firms can we be more confident in the polling numbers. So stay tuned…