‘Figure3 Examples of subjects’ right hand trajectory in Bimanual Circle-Line condition. Note the increased ovalization for healthy controls (A) and for patients with AHP (B), but not for hemiplegic (HP; C) or patients with motor neglect (MN; D).’

One source of such cases is research on anosognosia for hemiplegia. Patients with anosognosia for hemiplegia will sometimes deny, and appear in some ways unaware of, a severe paralysis of one or more limbs on one side of their bodies. Some such patients lack concurrent awareness of failures to move their plegic limbs but do not suffer from severe sensory deficits or neglect, and cannot move their hemiplegic limbs at all. For our purposes it is useful to focus only on these patients.

On the leading, best supported explanation, in these cases anosognosia for hemiplegia arises from deficiencies in monitoring action \citep{berti:2005_shared,berti:2008_motor}. To illustrate, consider a patient who was asked to brush her hair holding a brush in her paralysed hand. Although she was unable to move the hand, she proceeded to move her head as if her hair was being brushed and then reported having successfully brushed her hair \citep[pp.\ 173--4]{berti:2008_motor}. How could a deficit in monitoring action explain this? When a subject with anosognosia for hemiplegia is asked to perform an action involving her hemiparetic limb, motor representations occur as they might do in ordinary subjects \citep{berti:2005_shared,garbarini:2012_moving}. However, in ordinary subjects monitoring processes reliably ensure that any failures to act are detected; motor representations adjust accordingly \citep{Haggard:2005sc}. By contrast, in these cases of anosognosia for hemiplegia, there is damage to the monitoring processes or to capacities underlying them. A consequence is that motor representations are isolated from information relevant to failures to act. This is why some patients with anosognosia for hemiplegia sometimes act as if their hemiparetic limbs were actually moving.

How is anosognosia for hemiplegia relevant to our concern with experiences revelatory of action? Consider an anosognosic patient like those just mentioned and a patient with hemiplegia but no anosognosia. Suppose each is asked to draw simultaneously with both hands, where the unaffected hand was supposed to continuously draw a vertical line and the paralysed hand to continuously draw a circle. There will be a difference in their experiences. The anosognosic patient will sometimes judge that she is performing a bimanual action; this indicates that she has an experience which reveals the goal of drawing both lines and circles. By contrast, the non-anosognosic patient with hemiplegia will report performing a unimanual action and not the bimanual action, of course; this confirms that, as expected, she has no experience revealing the goal of drawing both lines and circles. What could explain this difference in experience between the two patients? The sensory information available to each patient should be the same: after all, hemiplegic individuals can of course only actually move one hand, and the patients we are concerned with do not have relevant sensory deficits. But there is a difference between the patients' motor representations. In the anosognosic patient, deficient monitoring means that motor representations should occur much like those that would occur were she not hemiplegic: there will be motor representations concerning the movements of left and right hands. By contrast, in the hemiplegic but non-anosognosic patient intact monitoring ensures that these motor representations do not occur or do not persist: there will only be motor representations concerning the movements of the unaffected hand. The predicted difference in motor representation can be confirmed by measuring how straight the lines drawn are: in the anosognosic patient's case only, the attempted straight line will show interference of the sort that, ordinarily, would be expected only if the other hand were actually drawing a circle \citep{garbarini:2012_moving}. So comparing hemiplegic patients with and without anosognosia yields a pair of cases fitting our criteria: there are differences in which goals experiences reveal, and these differences appear to be determined by differences in what is represented motorically.

The difference in differences ...

Here is the argument. Consider two sequences of sensory encounters: (a) a sequence of sensory encounters with two phonetic events that do not differ with respect to category (both are realisations of /d/, say), and (b) a sequence of sensory encounters with two phonetic events that do so differ (one is a realisation of /d/ the other of /g/, say).11 Let the events encountered in the first sequence differ from each other acoustically in the same way and by the same amount as the events encountered in the second sequence differ from each other. (That it is possible to find two such pairs of events follows from the fact that we enjoy categorical perception of speech.) The two sequences are depicted in Fig. 3. Now:

Repp and Knoblich (\citeyear{repp:2007_action}) asked expert and non-expert pianists to press two keys in sequence, where the first key was sometimes to the left, and sometimes to the right, of the second key. The key presses produced an \emph{ambiguous tone pair}, that is, a pair with the property that the first tone is sometimes perceived as lower in pitch than the second whereas at other times it is perceived as higher in pitch \citep{deutsch:1987_tritone}. The tones always occurred in the same order regardless of which key was pressed first. By asking subjects to report how they perceived the relative pitches of the tones, Repp and Knoblich found that, for the expert pianists, the direction of the key presses influenced the perceived direction of the change in pitch. Could what influences experience in this case be not a motor representation but merely the occurrence of a movement, or perhaps even the perception of a movement of the subject's own fingers? Against these possibilities, note that the effect was not observed in non-expert pianists: for them the direction of movement did not measurably influence the perceived pitches. Since the direction of movement was the same for both groups, if the influence were due to movement only we would expect it to occur irrespective of piano-playing expertise. Instead it seems likely that differences in expertise between the two groups of subjects affected how the movements they performed were represented motorically, and that these differences in motor representation are in turn what explains their perceptions of relative pitches.

It is not only in performing action that motor representation can influence experience: the same can occur in observing action. Thus in another experiment, Repp and Knoblich (\citeyear{repp:2009_performed}) compared observing someone else perform a sequence of key presses with performing the same sequence oneself. They found the same effect on experiences of an ambiguous tone pair in expert pianists regardless of whether they were observing or performing the action. Sometimes, which judgement about pitch an experience provides a reason for depends on what is represented motorically; and this dependence is systematic, of course, for it reflects how pianos work. These studies, and others like them,% \footnote{ See also \citet{zwickel:2010_interference} who investigate effects of action on visual experience of motion, and \citet{schutz-bosbach:2007_perceptual} for a review. } provide relatively direct evidence that motor representation can shape experience.

what is an object index? Formally, an object index is ‘a mental token that functions as a pointer to an object’ \citep[p.\ 11]{Leslie:1998zk}. If you imagine using your fingers to track moving objects, an object index is the mental counterpart of a finger \citep[p.~68]{pylyshyn:1989_role}.

The interesting thing about object indexes is that a system of object indexes (at least one, maybe more) appears to underpin cognitive processes which are not strictly perceptual but also do not involve beliefs or knowledge states.

Part of the answer to the objection is that motor representations of outcomes may structure our experiences of objects. The existence of affordances suggests that this is at least possible.

So here is the idea: What is experienced is an object, not an action or an outcome. So the fact that the action lies in the future and the outcome has not yet occurred is no objection. But the motor representation of the outcome structures the experience of the object in some way. So the overall character of the experience of the mug differs when an action is represented motorically compared to when it does not.

And this difference is structural in the same sense that the difference object indexes make is structural. It’s about how elements of experience are organised rather than about any particular sensory modality.