Chapter Outline

Chapter Outline

1. THE ACQUISITION OF AN APPETITIVE RESPONSE

   1. Skinner's Contribution: Skinner demonstrated that reinforcement has a significant impact on behavior. Central to Skinner's theory is the concept of contingency which expresses the specific relationship between a target behavior and a reinforcer. A reinforcer is any event used to increase the frequency of behavior.

   2. The Distinction Between Instrumental and Operant Conditioning: Skinner considered instrumental conditioning as a situation requiring a response for reinforcement. However, the environment of the conditioning situation constrains the opportunity to make the instrumental response. In contrast, in operant conditioning there are no constraints placed upon the subject regarding response execution. Skinner developed the operant chamber to study conditioning more conveniently.

   3. Types of Reinforcers: Skinner identified several types of reinforcers. A primary reinforcer has innate reinforcing properties; a secondary reinforcer develops its reinforcing properties through its association with primary reinforcement. The ability of a stimulus to function as a secondary reinforcer depends upon (1) the strength of the primary reinforcer with which it has been paired, (2) the number of pairing of the secondary and primary reinforcers, and (3) the contiguity of the presentations of secondary and primary reinforcers. Skinner also distinguished between positive and negative reinforcers. A positive reinforcer is an event that has the ability to increase the frequency of a behavior which precedes the event. A negative reinforcer is an event whose termination after the occurrence of a behavior increases the frequency of the behavior.

   4. Shaping: Shaping is a technique for training an animal to emit a desired behavior by first selecting a high-frequency operant behavior and then slowly changing contingencies until the desired behavior is learned. Shaping is also known as the successive approximation procedure.

      1. Training a Rat to Bar Press

         A rat's operant rate of bar pressing is not at zero, because the animal, through exploration, might engage in bar pressing. However, shaping ensures rapid acquisition of the desired behavior.

      2. Shaping Social Discourse

         Parents can employ shaping to effectively teach social skills to their children.

   5. Schedules of Reinforcement: The contingency relationship between an operant response and the probability of reinforcement is known as a schedule of reinforcement. Many such schedules have been studied, including two major classes: (1) ratio schedules of reinforcement that require a certain number of responses to produce reinforcement, and (2) interval schedules of reinforcement that reinforces a response only after a certain amount of time has elapsed since the last reinforced response.

      1. Fixed-Ratio Schedules

         A fixed ratio schedule of reinforcement requires a fixed number of responses to be made to obtain reinforcement. In other words, every nth response is reinforced, regardless of how much time has elapsed since the last reinforcement. This schedule is characterized by a postreinforcement pause, which refers to a temporary cessation of behavior following reinforcement, followed by a resumption of responding at the level characteristic of that ratio schedule.

      2. Variable-Ratio Schedules

         A variable ratio schedule is one in which an average number of responses produces reinforcement. It is exactly like a fixed ratio schedule except that the actual number of behaviors required to produce reinforcement changes after each reinforcement.

      3. Fixed-Interval Schedules

         The fixed interval schedule involves a contingency in which reinforcement for a response is available only after a specified period of time has elapsed since the previous reinforced response. This schedule produces the scallop effect, which refers to a pattern of behavior characteristic of the fixed interval schedule. Response rate slows after reinforcement and then slowly increases as the time approaches when reinforcement will be available.

      4. Variable-Interval Schedules

         The variable interval schedule is the same as a fixed interval schedule, except that the interval changes after each reinforced response. The scallop effect typically does not occur in this schedule.

      5. Differential Reinforcement Schedules

         When reinforcement depends on both time and number of responses, the contingency is called a differential reinforcement schedule.

      6. Differential Reinforcement of High Responding Schedules

         The differential reinforcement of high responding (DRH) schedule requires a high rate of behavior over a specified time period before a response is reinforced.

      7. Differential Reinforcement of Low Responding Schedules

         In the differential reinforcement of low responding (DRL) schedule, a certain interval of time must pass without a response. The first response following the end of the time period is reinforced.

      8. Differential Reinforcement of Other Behaviors Schedules

         A differential reinforcement of other behaviors (DRO) schedule provides reinforcement only when there is a total absence of a particular response in a specified period of time.

      9. Compound Schedules

         In a compound schedule, two or more of the above schedules are in effect, one at a time, and animals must complete whatever schedule is in operation at any one time.

   6. How Readily Is an Instrumental Response Learned?

      1. The Importance of Contiguity

         Reward leads to the conditioning of the response when it follows immediately after the response. Delaying the reward interferes with conditioning.

      2. The Effect of Delay

         A delay between the operant response and its reinforcement has a pronounced negative effect on learning rate.

      3. Delay of Reward and Conditioning in Humans

         As is true for nonhuman animals, humans also learn more slowly as the delay between response and reinforcement increases.

   7. The Impact of Reward Magnitude

      1. The Acquisition of an Instrumental Response

         Many studies have shown that the greater the reward magnitude, the faster the acquisition of the instrumental response.

      2. The Performance of an Instrumental Response

         The greater the reward magnitude, the higher is the performance level of an operant or instrumental response.

      3. The Importance of Past Experience

         The shift from high to low reward magnitude momentarily reduces the level of instrumental performance below that which occurs when the low reward magnitude is used consistently. Crespi (1942) called this outcome the depression effect. When the shift moves in the other direction, from low to high reward magnitude, the level of instrumental behavior is momentarily higher than found when the high reward magnitude is used consistently. Crespi used the term elation effect to describe this outcome. Later, Zeaman (1949) substituted the term positive and negative contrast for the elation and depression effect. Contrast effects are more likely the result of changes in motivation or emotion (e.g., frustration in negative contrast) than in learning.

      4. The Influence of Reward Magnitude in Humans

         As they do in nonhuman animals, increases in reward magnitude tend to have a positive effect upon performance in humans.

2. EXTINCTION OF AN OPERANT OR INSTRUMENTAL RESPONSE

   1. The Discontinuance of Reinforcement: Omitting reinforcement after an operant or instrumental response leads to extinction.

   2. Spontaneous Recovery: If an operant response is extinguished by removing the reinforcer, a succeeding "rest" period might be followed by spontaneous recovery of the response. This outcome parallels spontaneous recovery in Pavlovian conditioning. Presumably, extinction occurs because nonreinforced trials in extinction increase inhibition.

   3. The Aversive Quality of Nonreward: Amsel claimed that nonreward elicits an aversive frustration state. Cues that are associated with frustration become aversive, and can motivate escape behaviors to terminate those cues.

   4. Activation of an Instrumental Behavior: Omitting an expected reward can not only lead to extinction of the response that previously produced the reward, it can also facilitate other instrumental appetitive responding.

   5. Resistance to Extinction: The following three factors influence the resistance to extinction of an operant or instrumental response.

      1. The Influence of Reward Magnitude

         When the level of acquisition training is low, a large reward produces greater resistance to extinction than does a small reward. However, with extended acquisition, a small reward during acquisition produces greater resistance to extinction.

      2. The Influence of Delay of Reward

         When a constant delay of reward is experienced during acquisition, resistance to extinction is not affected by the length of the delay. However, variable delay of reward during acquisition increases resistance to extinction in some situations.

      3. The Importance of Consistency of Reinforcement

         Compared to continuous reinforcement, intermittent reinforcement in acquisition produces greater resistance to extinction. This outcome is the partial reinforcement effect (PRE). Two theories that explain the PRE reasonably well are Amsel's frustration theory and Capaldi's sequential theory. Amsel suggested that nonreward elicits a frustration state. Continuously reinforced animals quickly stop responding in extinction because frustration is aversive and inhibits responding. But animals that are intermittently reinforced often are rewarded for responding while they are experiencing frustration from prior nonreward, and thus learn to respond when those cues are present. Thus, during extinction, nonreward elicits responding in partially reinforced animals, but suppresses responding in continuously reinforced animals.

         According to Capaldi, if reward follows a nonrewarded trial, an animal will associate the memory of nonreward (SN) with the response leading to reward. The PRE occurs because during extinction, continuously reinforced animals have more novel cues present (SN) than partially reinforced animals. Thus, in continuously reinforced animals, there are fewer cues present in extinction that are associated with responding.

3. APPLICATION: CONTINGENCY MANAGEMENT

   Skinner proposed that faulty contingencies often produce behavioral problems. Behavior modification includes a broad array behavioral treatments. One of these is contingency management, which uses reinforcement and nonreinforcement to increase the frequency of appropriate behaviors and eliminate maladaptive behaviors. Contingency management involves three phases:

   1. The Assessment Phase: In the initial stage of contingency management, a determination of the behavioral problem is made.

   2. The Contingency Contracting Phase: In the second stage, a desired response is specified and the precise relationship between response and reinforcement is detailed. Reinforcement can be dispensed by another or by the individual performing the behavior.

   3. The Implementation of the Contingency Management Program: Contingency management programs have been found to be very successful in a wide variety of situations.