We had said on several previous occasions (for don’t say that is general after the battle) that there were problems with the surveys: It was not clear that you were properly analyzed the voter likely, the rejection rates (here link), that the primary had shown a number of surprises (here link). More in general, we emphasized the fact that the methods have to be coordinated with the social reality they study, their validity depends on the status of that reality (link)
The results of the November 19, showed this clearly. The Radio of the University of Chile had to show a picture that is clear enough.
The main survey had several important errors for Piñera (10 points in several cases) and Sanchez (12 to 6 points), and with a strong underestimation of Kast. More still, the overall impression -a vote very high Piñera, of the important differences between Guillier and Sanchez – was clearly erroneous.
Now, does that imply that there is a problem of bad faith? Do the surveys right we were tricked? And no. The CIRCLE that is not of right, was between surveys with a more problems. UDD-The Mercury -the people who use that interpretation, I would say that is right – was quite good.
What is wrong with all the polls? Do not serve for nothing? Both Criteria as UDD obtained estimates that were fairly decent. In general, the surveys were not wrong with Guiller (only the CIRCLE), and nobody gave many votes to Goic (who also was the one who criticized prior to the survey). Adimark, among the great, had minor errors with Piñera and Sanchez (and he stopped doing surveys before the other).
In other words, there are important problems to solve (how do you calculate voter likely?); but not everyone was wrong.
Here it is important to make two clarifications are crucial. A attitude and the other methodological.
Let us start with the attitude of the reaction. Here’s Mackenna of the CEP on Twitter today
To see another reaction,Izikson of CADEM through the same social network:
Between Mackenna and Izkinson is the difference between the serious and the mocking of the population. Between the one who truly believes that the things there are to do it well and who gives the same. The difference between the minimum professionalism and the seller of smoke.
A survey is, in the end, an instrument for quantitative, the minimum is to go beyond the order. The impression left by a survey is not only an issue of order. An error of 10 points is not a matter of ‘do more accurate estimates’.
In the end, based on election results and analysis it is politically possible to estimate better. In my office we did the exercise of bet in trying to predict the number of votes for Piñera. The data that we use to do this were (a) how many people voted 1a back 2013, (b) vote primary right-2017, (c) voting Evelyn Matthei in 2a back 2013. Based on this, the average of the office were 2.6 million votes, concentrated the options between 2.3 and 2.5 million (the minimum level of voting right is what got Matthei in 2a back 2013 and then you can get more because it is stronger candidate). The outliers thought of 3 million because they followed rather to the surveys. This is important because the same Izkinson has retrucado to the fact you only gave the order with that order, it just seems obvious given the surveys. Without them, it is also possible to obtain estimates.
Then, a good survey needs to deliver something more than a good result at the lump. In particular, a good electoral survey.
The second theme is methodological. It is known that the survey CADEM does not comply with any requirement of the methodological minimum. Use surveys at the point of influx to a survey policy, it is tacky (and again, that it is well-known and had already been said on this blog already 2015). And yet, they were used -not just the media, even Kenneth Bunker -that develops in TresQuintos predictions bayesian – used. Izkinson made several defenses of his method. This is even worse because the other surveys (Criteria for example, that we now recall because their results were relatively decent) were vapuleadas for using an Online Panel -the same Bunker on Twitter declared presentable. What appeared as showing his clear unsuitability was the high vote of Bea Sanchez. It is well inconsistent to declare unusable a methodology problem while at the same time accept other.
Remember that online panels have problems -the bias is not solved automatically through ponderadores. However, they have been used in other contexts (and in some studies to be quite successful with signs clearly biased -there’s a study in 2015 with Xbox users that delivers good results to the survey american, link here). Even in the chilean case, apart from Criteria other surveys using panels (private) gave reasonable results. We know that the traditional methodologies have problems (response rates for example), and then there’s that test new methodologies in the current context. Now, to make these tests it is necessary to do things with seriousness and care.
And we know very well that seriousness and care is not what characterizes the survey CADEM. Problems with surveys are there, the fail will be; but the minimum is to recognize them to be able to solve them. Quit defending the indefensible shows that there is not much interest to do the right thing. If this debacle at least is achieved that is not acceptable to do any thing, something you can get in clean.