FAULTING KENYAN OPINION POLLS

The Standard
Nairobi, Kenya
September 22, 2008
By ROBINSON OCHARO

Competitive politics to day is very much hinged on opinion polls. This is very much so especially for strategizing and campaigning.

The polls provide the public with an opportunity to express themselves before casting their vote, express their levels of satisfaction with the government after they have voted, guiding political parties on political preferences and providing a back – up to media reports. Success of opinion polls depends on inter-alia sampling, choices of opinion items (indicators), data collection techniques and the actual data classification, presentation and analysis.

As a general rule, samples picked for polling must be a representative of the universe otherwise the finding will not be externally valid: That is, they cannot apply to those subjects in the universe who were not represented in the sample. Secondly, items chosen for polling must have epistemic correspondence to the concept being studied.

That means each item/indicator selected should appear to have a definite relationship to the attitude being measured.

Otherwise the study ends up with an opinion that is not reproducible (The final response/outcome is not predictable). Finally, data editing, classification, presentation and analysis should be done by professionals following the professional ethics of research. Therefore the biggest test opinion polls are faced with is that of validity. A classic example in which an opinion poll failed the validity test is that of the study conducted prior to the 1948 USA presidential election. It was predicted that Republican Thomas E. Dewey would emerge victorious. This in return influenced the media who declared before the votes where counted that Dewey had won the elections.

After actual official vote count, it finally turned out that the 31st president of the United States of America was Harry Truman. This truth came out after newspapers carried headlines such as "Dewey Defeats Truman." This error could have been caused by one or several of the following factors: Sampling, choice of opinion indicators, researcher bias, timing and rate of polling.

It is in this background that I mirror the following two issues of the recent Gallup polling on the professionalism of polling: Firstly, the sampling. The Gallup study picked a "representative sample of 2,200 across all provinces in Kenya".

This is perfect until one asks the basis of representation in this particular study. This justifies the reason why any consumers of the Gallup study findings should conclude that with a postulation that the opinion expressed in the said study should, first and foremost, be seen as an opinion of the 2,200 Kenyans interviewed – unless it is established beyond doubt that the sample picked is a representative of the Kenya population who have a potential of forming an opinion on the issues covered in the study.

In the case of Kenya for an opinion (especially on politics) from a sample to be externally valid, it must capture key variables such as ethnicity, age, education level, gender, social status and residence (rural/urban). These variables cut across the Kenya voter. One would be assuming external validity in these findings if the young and the old, the male and female, the educated and the illiterate, the rural and the urban, those with high social status and those with low social status in Kenya were well represented in the sample.

Secondly, there is the issue of choice of indicators. As a general rule, unless one is rating, an opinion scale should have multiple indexes to be used in locating the opinion of the respondent on an issue.

For example a question "Do you believe the election was honest?" cannot be used to measure opinion. Rather, certain key indicators should be identified and used to measure the respondents’ opinion on the honesty of the electoral process of 2007.

In this case the researchers should have asked the important question of what constitutes honest and how the respondents would be able to differentiate between honesty and dishonesty? Above all items used for polling should have epistemic correspondence to the concept being studied. Under the item mediation for example the Gallup study asked the following questions:

Do you approve or disapprove of Kofi Annan’s efforts in forming a coalition government?

Do you approve or disapprove of UN efforts to stop the violence in Kenya after the election?

Do you approve or disapprove of US efforts to stop the violence in Kenya after the election?

I believe that the formation of the coalition government, the role role of the UN and the US in stopping the violence should have been separate, each with a set of indicators for measuring the Kenyan opinion on their success or failure/acceptance or rejection.

Interesting, though, the following items were in the Gallup study:

1. Do you think there was one ethnic group that was responsible for/behind the post-election violence or the responsibility should be shared by more than one ethnic group? Which one?

2. I am going to read a statement. Please tell me whether you agree or disagree with it. In the future, I can coexist peacefully in my city or local community with all Kenyans, regardless of their ethnicity or tribal affiliation.

As for the first item, it is a psychological fact that such question appeals directly to the self-serving bias of the respondent. A bias in which human beings tend to see themselves better than those of the out-group and therefore, automatically associate bad things with the out-group and good things with the in-group.

The second item is likely to yield misleading responses as it does not measure the actual social distance between groups: It does not separate overt behaviour from covert behaviour. For example a respondent simply saying that in future he/she can coexist peacefully with another Kenyan from a different ethnic group does not in psychological terms mean that there is no conflict.

Human beings are known to share physical space without overt conflict but inwardly are in conflict. So, the presence of a trigger event like incitement will make that which has for long been covert, overt. Gallup can find tips on measuring social distance from the Emory Bogardus’ Social desistance scale.