Mental Rotation

Shepard & Metzler (1971) introduced the concept of mental rotation into cognitive science with what has become one of the best-known experiments in the field. This fame may, in part, be because the experiment is associated with a set of memorable graphical images (e.g., figure 1), some of which made it onto the cover of the issue of Science where it was first published. However, it is also a remarkably elegant experiment, that produced some strikingly clear results. The findings seemed directly to refute the Behaviorist doctrine, still holding considerable sway amongst psychologists at the time, that thought processes depend entirely upon language. By suggesting that analog representations have an important role to play in thinking, the findings also raised prima facie difficulties (though not necessarily insurmountable ones) for the (digital) computer model of the mind that lay at the heart of the newly emerging field of cognitive science (Block, 1983a).

Figure 1
Some of the stimulus figure pairs used by Shepard & Metzler (1971).
A- Identical objects differing by a rotation in the plane of the page
B – Identical objects differing by a rotation in depth
C – Mirror-image objects (also rotated in depth)

Shepard & Metzler presented their subjects with pairs of drawings of three-dimensional, asymmetrical assemblages of cubes, as shown in figure 1 A, B, and C. In each pair, the right-hand picture either showed an assemblage identical to that shown on the left, but rotated from the original position by a certain amount, or else it showed an assemblage that was not only rotated, but was also the mirror image of the one to the left (figure 1 C). The experimental task was to tell, as quickly as possible (by pressing a button) whether the two objects depicted were in fact identical (except for rotation), or were mirror images. Shepard's hypothesis was that the task would be done by forming a three-dimensional mental image of one of the depicted objects, and rotating this whole image, in the imagination, to see whether it could be brought into correspondence with the other picture. The experimental results clearly supported this idea, because it was found that, for each subject, the time taken to confirm that both objects of a pair were, in fact, identical, increased in direct proportion to the angular rotational difference between them. It was as if the subjects were rotating their mental image at a steady rate (although this might be different for each subject), so that the further they had to go to bring their image into correspondence with the reference picture, the longer it would take them. On post-experimental questioning, most of the subjects confirmed that this was indeed how they believed that they had done the task. (Interestingly, it made no difference whether the rotation was in the plane of the page, or in depth.)

Despite the elegance of this experiment, and the clear-cut results (nice linear plots of reaction time against rotational angle), Shepard's interpretation of it as evidence for irreducibly analog and intrinsically spatial processes in thinking, and for what he called a “second order isomorphism” between image and object (Shepard, 1975, 1978b, 1981, 1984), certainly did not go unquestioned. Some researchers challenged Shepard's contention that his results show that images are rotated as a whole, rather than their parts being compared in a piecemeal fashion (Hochberg & Gellman, 1977; Pylyshyn, 1979a; Yuille, 1983). More radically, some have doubted whether the rotation task really involves imagery at all (Marks, 1999). After all, unlike in the experiments with imagery mnemonics, the subjects were never explicitly told to use imagery in performing the comparison task, and alternative explanations of the result (i.e., the linear relation between reaction time and angle of rotation) are conceivable. For example, Just & Carpenter (1976; Carpenter & Just, 1978), who tracked subjects' eye movements whilst they did a version of the Shepard & Metzler task, argued that the linear increase in reaction time arose not from the inner rotation of an image, but from a need to make more eye movements between the two pictures (in order to compare their features) the more they were rotated relative to one another. In the iconophobic culture that still prevailed in psychology in the early 1970s such an “imageless” interpretation of the results was still very much a live option. Of course, it meant that any claims by subjects to have formed and rotated mental images had to be discounted, but Behaviorism had long since taught psychologists to give very little weight to this sort of subjective, introspective data.^[1]

However, the case for mental rotation does not rest solely on this one celebrated experiment. Shepard and his students (especially Lynn Cooper) were subsequently able to demonstrate mental rotation and other related effects in quite a number of different experimental designs (see Shepard & Cooper et al., 1982) mostly designed to block alternative interpretations of the results that would avoid the need to postulate rotating imagery. Most of these other experiments did not involve the comparison of two simultaneously visible pictures, thus leaving no scope for the sort of eye movement explanation that Just & Carpenter had suggested. For example, Cooper & Shepard (1973) presented their subjects a letter of the alphabet rotated out of its normal, upright orientation, and asked them, again, to indicate whether the letter was in its normal form or its mirror image (backward) (see figure 2). Once again the response time was found to increase the further the stimulus letter was rotated from upright (although, for reasons that need not detain us, the relationship was not so neatly linear as in the earlier experiment). The implied explanation is that the subjects rotate their image of the non-upright letter that they are shown into its canonical upright orientation, in order to compare it to their memory of how the letter would normally look.

Figure 2
Rotated letter stimuli of the sort used by Cooper & Shepard (1973).

Cooper (e.g., 1975, 1976) also did several experiments on the mental rotation of complex irregular polygons, such as those in figure 3, and some of this work depended on the subjects being explicitly told to form and rotate mental images of the polygons (whose shape they had previously memorized). Once again, a linear dependence of reaction time upon angle was found, suggesting smooth, regular rotation of an image. An advantage of the explicit imagery instructions used in these experiments is that it forges a more explicit link between the measured results and imagery as a conscious experience under voluntary control. On the other hand, it opens up the very real possibility that the results might be explained away as the result of “experimental demand” (essentially, the subjects deliberately trying to produce the results that they believe the experimenters want, regardless of the actual underlying cognitive processes involved – see Supplement: The Problem of Demand Characteristics in Imagery Experiments). This is well known to psychologists to be a very real problem in many areas of experimental psychological research (Orne, 1962), and it has been shown to be particularly problematic in imagery research of this type (Intons-Peterson, 1983). However, when these experiments of Cooper's are taken together with the other rotation experiments that did not use explicit imagery instructions, and gave little foothold for experimental demand to influence the results (e.g., Shepard & Metzler, 1971; Shepard & Cooper, 1973) (and other experiments not discussed here: see Shepard & Cooper et al., 1982) they make a powerful case for the reality and robustness of the mental rotation phenomenon.

Figure 3
Examples of irregular polygons used in mental rotation experiments by Cooper (1975, 1976).

Mental rotation experiments of the original Shepard & Metzler (1971) design (or variants of it), are almost unique amongst imagery experiments in that they depend neither on verbal reports from the subjects, nor on explicit verbal instructions to use imagery in performing the experimental task. They can thus be adapted for use with animal subjects, raising the possibility of providing direct evidence for the occurrence of imagery in non-human animals. The results of attempts to do this, however, with both birds (pigeons) and various species of monkeys, have been mixed, to say the least, and largely disappointing (Hollard & Delius, 1982; Rilling & Neiworth, 1987, 1991; Georgopoulos et al., 1989; Hopkins et al., 1993; Vauclair et al., 1993; Delius & Hollard, 1995; Köhler et al., 2005; Burmann et al., 2005; Nekovarova et al., 2013). Remarkably, the most plausible evidence that any animals can use the mental rotation strategy, and thus probably experience imagery, comes from work with sea lions (Mauck & Dehnhardt, 1997; Stich et al., 2003).

On the other hand, the effect does not seem to depend specifically upon visual imagery: it has been demonstrated in congenitally blind human subjects (Marmor & Zaback, 1976; Carpenter & Eisenberg, 1978), who are generally believed not to experience visual mental imagery, and are thus presumably rotating haptic or kinesthetic images. Mental rotation abilities have also been studied in infants (Örnkloo & von Hofsten, 2007), the elderly (Dror et al., 2005), and in people with neurological problems (Courbois et al., 2004; Hinnell & Virji-Babul, 2004), and age, sex, and even sexual-orientation related differences in its performance have been investigated (e.g., Richardson, 1994;; Terlecki & Newcombe, 2005; Levin et al., 2005; Quaiser-Pohl et al,. 2006; Maylor et al., 2007; Joanisse et al., 2008). Furthermore, the mental rotation concept has been applied to practical issues ranging from the improvement of surgical technique (Conrad et al., 2006) to the understanding and treatment of dyslexia (Russeler et al., 2005).

Controversy continues about the underlying mechanisms of mental rotation (as of imagery in general), but these have recently been investigated by several neuroscientific techniques, such as eye movement measurements (de'Sperati, 2003), direct recording from electrodes implanted in the brain (Georgopoulos et al., 1989), functional magnetic resonance imaging (fMRI) (e.g., Cohen et al., 1996; Richter et al., 2000; Creem et al., 2001; Koshino et al., 2005; O'Boyle et al., 2005), EEG (Prime & Jolicoeur, 2010), and transcranial magnetic stimulation (Ganis et al., 2000). Some of this research has been focused on “motor imagery” (Jeannerod, 1994) rather than visual imagery. Indeed, it has been suggested that motor processes and motor areas of the brain may be involved in mental rotation quite generally (Kosslyn, 1994; Wexler, Kosslyn & Berthoz, 1998; Wohlschläger, 2001; Tomasino et al., 2005; Amorim et al., 2006; Eisenegger et al., 2007; but see Flusberg & Boroditsky, 2011).