The experiment had a number of weaknesses, some of which I discussed in my previous article and others of which are discussed in the e-mail messages and replies that I just mentioned. Most of these weaknesses were things that I couldn't do anything about after the fact. One, however, was something I could try to address by re-analyzing the data. To quote from my my previous article: I don't think my scenario was a great one. I think it was a bit too heavily slanted to suggest that Turkey was the target, regardless of which map you were looking at. (This is not pure speculation, but is something that is also supported by the data.)
You can reread my previous article for a better description of this problem, but basically, it had to do with the fact that the analysis implicitly assumed that the scenario I presented was balanced. That is, it assumed that France and Turkey were roughly equally likely Italian targets, and therefore that the results of a calculation I did would be either positive or negative if France or Turkey was the target. What I didn't consider was the fact that an unbalance in the scenario would mean that more people would think Turkey was a target, regardless of which of the two maps they saw. This had the effect of shifting the true center of peoples' opinion away from zero. Thus, instead of drawing a line based on whether the result of that calculation was less than or greater than zero, I should really have drawn a line based on whether the result was less than or greater than x, where x was a number offset from zero.
As indicated in my quote from my previous article, I thought the data seemed to support the idea that the scenario was biased to indicate that Turkey was a more likely target. It turns out that this was true. If the scenario had been balanced, I would have expected more people to think France was the target when units were pointed towards France, and more people to think Turkey was the target when units were pointed towards Turkey. As it turns out, more people thought Turkey was the target regardless of which map they saw.
But interestingly, the effect I had originally hypothesized is there at a gross level. Even though the average response indicated that Turkey was a target regardless of which map was seen, the ratings (on a scale from 1 to 5) showed that people more strongly thought that Turkey was a target when units were pointed towards Turkey, and more weakly when units were pointed towards France. Reexamining the data showed that the difference in average response between the two maps was over a fifth of a point.
So if the scenario had been balanced, that fifth of a point difference would have been centered about zero, with one tenth of a point below the zero mark (negative) and one tenth above (positive). However, the actual center was a bit under 0.2, so that one tenth of a point in either direction still left you on the positive side. (I'm talking average scores here... individual responses could and did go to the negative side). One way to correct for the data would be to take into account this offset by looking for values above and below that number a bit less than 0.2, instead of above and below zero.
Vindication for my hypothesis? Not quite. I didn't go and generate new graphs with the correction I described, because what that's really doing is adding an offset. In other words, every bar in all the graphs would be shifted by an amount a bit less than 0.2. These corrected charts would better support my hypothesis than did my original ones, but they would not unquestionably prove my hypothesis. Though better, they would not be conclusive. I believe this to be primarly due to weaknesses in the experimental design and execution, but I cannot say with certainty that the reason is that, rather than the possibility that my hypothesis simply was incorrect.
It would certainly be interesting to have the resources (time, subjects) to conduct a better-designed, non-electronic experiment under more controlled conditions. Perhaps I'll submit a grant proposal to the National Institute of Mental Health under the guise of trying to understand paranoid people... a title along the lines of "The Emergence of Paranoia as a Survival Trait in a Small Highly Competitive Population." Thanks for taking an interest in my mental ruminations.
|
Simon Szykman ([email protected]) |
If you wish to e-mail feedback on this article to the author, click on the letter above. If that does not work, feel free to use the "Dear DP..." mail interface.