A Puzzle about Action

Click here and press the right key for the next slide (or swipe left)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Is this lever pressing habitual or instrumental aciton?

habitual

Stimulus is the layout of this room.

Rat (=Agent) is rewarded with food

Room-LeverPress (=Stimulus-Action) Link is strengthened due to reward

Thf LeverPress (=Action) will occur in this room (=Stimulus).

Problem: different hypotheses, same prediction

What if we devalue the food?

Explain devaluation (poison, or satiation)

habitual

Stimulus is the layout of this room.

Rat (=Agent) is rewarded with food

Room-LeverPress (=Stimulus-Action) Link is strengthened due to reward

Thf LeverPress (=Action) will occur in this room (=Stimulus).

instrumental

Lever pressing (=Action) leads to food (=Outcome).

Thf LeverPress-Food (=Action-Outcome) Link is strong.

Rat (=Agent) has strong positive Preference for food.

Thf LeverPress (=Action) will occur.

Devaluation affects Preference, so changes what the instrumental hypothesis predicts.

Devaluation does not affect the Simulus-Action link. (It’s the fact that food was preferred in the past that matters: because of this, getting food was rewarding and so strengthened the Simulus-Action link.)

What if we devalue the food?

Instrumental : it will reduce lever pressing (to none)

Habitual : it will have no effect on lever pressing

‘Mean lever-press rates during the extinction (left-handpanel) and reacquisitiontests(right-handpanel) followingthe devaluation of either the contingent (group D-N) or non-contingentfood (group N-D).’

Dickinson, 1985 figure 3

What if we devalue the food?

Instrumental : it will reduce lever pressing (to none)

Habitual : it will have no effect on lever pressing

(a) Rat’s behaviour is instrumental (explained by their Preferences). (b) Hypotheses about processes underpinning decisions are scientifically testable.

‘the laboratory rat fits the teleological [instrumental] model; performance of this particular instrumental behaviour really does seem to be controlled byknowledge about the relation between the action and the goal’

\citep[p.~72]{Dickinson:1985qp}

Dickinson, 1985 p. 72

But there is a complication ...

‘we did not conclude that all such responding was of this form.

Indeed, we observed some residual responding during the post-re-valuation test that appeared to be impervious to outcome devaluation and therefore autonomous of the current incentive value,

and we speculated that this responding was habitual

and established by a process akin to the stimulus-response (S-R)/reinforcement mechanism embodied in Thorndike’s classic Law of Effect (Thorndike, 1911).

\citep[p.~179]{dickinson:2016_instrumental}

Dickinson, 2016 p. 179

Dickinson, 1985 figure 3

The puzzle: \begin{enumerate} \item If the action is habitual, why is it modulated by devaulation? \item If the action is instrumental, why does it still occur (albeit less frequently) after devaluation? \end{enumerate}

puzzle

If the action is habitual,
why is it modulated by devaulation?

If the action is instrumental,
why does it still occur (albeit less frequently) after devaluation?

Solution is to stop thinking that actions can be just one or the other. \emph{The instrumental/habitual distinction concerns proceses, not actions!}

Keyboard Shortcuts?

A Puzzle about Action

Keyboard Shortcuts`?`