Keyboard Shortcuts?

×
  • →Next step
  • ←Previous step
  • ↓Skip this slide
  • ↑Previous slide
  • mShow slide thumbnails
  • nShow notes
  • hShow handout latex source
  • NShow talk notes latex source

Click here and press the right key for the next slide (or swipe left)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

 

Motivational States

motivational states

When we think about motivational states, the most familiar kind is ...

primary motivational states

not directly modifiable by learning

  • hunger
  • thirst
  • lust
  • disgust
  • ...
... preference. These states change over time in under the influence of learning (and fashion, ...) I prefer chocolate over rhubarb right now, but might later have the reverse preference.

preferences

changing, influenced by learning (and fashion, ...)

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...
Another familiar kind of motivational state is what animal learning theorists call ‘primary motivational states’.
These do not change over time and are not modifiable by learning, or at least not readily modifiable.
They inlcude hunger, thirst, lust and disgust.

Can your primary motivational states diverge from your preferences?

 

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Devaluation - standard procedure:

Training: Rat is put in chamber with Lever; pressing Lever dispenses sucrose (novel food).

Devaluation: Rat is taken into another chamber, poisoned, and then exposed to sucrose.

Extinction Test: Rat returns to chamber with Lever; pressing Lever does nothing.

Dickinson, 1985 figure 3; Balleine & Dickinson, 1991 figure 1 (part)

‘Mean lever-press rates during the extinction (left-handpanel) and reacquisitiontests(right-handpanel) followingthe devaluation of either the contingent (group D-N) or non-contingentfood (group N-D).’
‘Experiment I: Mean number of lever presses ... during the extinction test session ... The various groups received either immediate (Groups IMM/SUC and IMM/ H20) or delayed (Groups DELjSUC and DEL/H2O) toxicosis [delayed did not cause aversion] and were re-exposed either to the sucrose solution (Groups IMM/SUC and DEL/SUC) or to water (Groups IMM/H2O and DEL/H20).’
Pavlovain conditioning, primary motivational states can have a direct effect on actions.

‘The dissociation between lever pressing and magazine entries produced by re-exposure is [...] problematic for the incentive learning account.

To recapitulate, this explanation assumes that instrumental performance is mediated by some “representation” of the relationship between the instrumental action and reinforcer that also encodes the current incentive value of the reinforcer. The represented incentive value can only be changed, however, after aversion conditioning by exposure to the reinforcer.

Given this account, the question immediately arises as to why re-exposure is necessary for a change in lever pressing but not magazine entries’

\citep[p.~293]{balleine:1991_instrumental}

Balleine & Dickinson, 1991 p. 293

Why do

the two actions,

lever pressing

and

magazine entry,

dissociate in this way?

conditioning

Pavlovian (classical)

Results in stimulus-stimulus links (e.g. bell-food)

The animal responds to the first stimulus as if the second were present

Acquired through exposure to contingencies

Subject to overshadowing and blocking (u.a.)

Overshadowing occurs when more than one stimulus is associated with an event: e.g. a light and a tone. The salience of the tone affects how much the animal learns about the correlation between light and event: a salient tone reduces the amount of learning. (31- 32)
Blocking : if an animal has previously learnt about a correlation B -> E2, it is much less likely to learn about a correlation A -> E2 when presented with a situation in which E2 is preceded by A and B. (48)

Operant

Results in stimulus-action links

The animal responds to the stimulus by performing the action

Acquired through being rewarded when acting in the presence of the stimulus

Involved habitual processes

habitual

Action occurs in the presence of Stimulus.

Agent is rewarded [/punished]

Stimulus-Action Link is strengthened [/weakened] due to reward [/punishment]

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

instrumental

Action leads to Outcome.
 

Action-Outcome Link is strengthened.

Agent has strong [/weak] positive [/negative] Preference for Outcome

Will Action occur? It depends on the strength of Action-Outcome Link and Agent’s Preference.

conditioning

Pavlovian (classical)

Results in stimulus-stimulus links (e.g. bell-food)

The animal responds to the first stimulus as if the second were present

Acquired through exposure to contingencies

Subject to overshadowing and blocking (u.a.)

Overshadowing occurs when more than one stimulus is associated with an event: e.g. a light and a tone. The salience of the tone affects how much the animal learns about the correlation between light and event: a salient tone reduces the amount of learning. (31- 32)
Blocking : if an animal has previously learnt about a correlation B -> E2, it is much less likely to learn about a correlation A -> E2 when presented with a situation in which E2 is preceded by A and B. (48)

Operant

Results in stimulus-action links

The animal responds to the stimulus by performing the action

Acquired through being rewarded when acting in the presence of the stimulus

Involved habitual processes

Why do

the two actions,

lever pressing

and

magazine entry,

dissociate in this way?

Because

magazine entry but not lever pressing ‘is under the control of Pavlovian ... contingencies’

and Pavlovian contingenies enable primary motivational states directly influence action.

Balleine & Dickinson, 1991 p. 294

‘A possible resolution to this discrepancy lies with the differing contingencies controlling lever pressing and magazine entry. There is evidence that simple anticipatory approach to a food source, such as magazine entry, is primarily under the control of Pavlovian as opposed to instrumental contingencies (e.g. Holland, 1979),thus raising the possibility that incentive learning is necessary for instrumental but not Pavlovian reinforcer revaluation effects. There is, in fact, independent evidence that accords with this analysis’ \citep[p.~294]{balleine:1991_instrumental}

 

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Devaluation - standard procedure:

Training: Rat is put in chamber with Lever; pressing Lever dispenses sucrose (novel food).

Devaluation: Rat is taken into another chamber, poisoned, and then exposed to sucrose.

Extinction Test: Rat returns to chamber with Lever; pressing Lever does nothing.

Dickinson, 1985 figure 3; Balleine & Dickinson, 1991 figure 1 (part)

Pavlovain conditioning, primary motivational states can have a direct effect on actions.

Aversion does not directly influence preferences.

‘The pattern of results accords [...] with a role for an incentive learning process in the reinforcer devaluation effect;

not only must consumption of the reinforcer be paired with toxicosis,

the animals must also have an opportunity to contact the reinforcer after aversion conditioning if there is to be a change in instrumental performance’

\citep[p.~293]{balleine:1991_instrumental}

Balleine & Dickinson, 1991 p. 293

[To introduce the term ‘incentive learning’]
A moment ago I asked, What happens if we poison the subjects but do not re-expose them to the food?

Can your primary motivational states dissociate from your preferences?

The two kinds of motivational states can dissociate

motivational states

primary motivational states

not directly modifiable by learning

  • hunger
  • thirst
  • satiety
  • disgust
  • ...

preferences

changing, influenced by learning (and fashion, ...)

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...
What kinds of processes in individual animals guide actions?

What kinds of processes in

individual animals

guide actions?

Two conclusions:

1. two kinds of process -- habitual vs instrumental

2. two kinds of motivational state -- primary vs preferences