After studying Chapter 11, you should know and understand the following key points:

True Experiments

Characteristics of True Experiments

In true experiments, researchers manipulate an independent variable with treatment and comparison condition(s) and exercise a high degree of control (especially through random assignment to conditions).

Obstacles to Conducting True Experiments in Natural Settings

Researchers may experience difficulty obtaining permission to conduct true experiments in natural settings and gaining access to participants.

Although random assignment is perceived by some as unfair because it may deprive individuals of a new treatment, it is still the best and fairest way to determine if a new treatment is effective.

Threats to Internal Validity Controlled by True Experiments

Threats to internal validity are confounds that serve as plausible alternative explanations for a research finding.

Major classes of threats to internal validity include history, maturation, testing, instrumentation, regression, selection, subject mortality, and additive effects with selection.

Problems That Even True Experiments May Not Eliminate

Threats to internal validity that can occur in any study include contamination, experimenter expectancy effects, and the Hawthorne effect.

Contamination occurs when information about the experiment is communicated between groups of participants, which may lead to resentment, rivalry, or diffusion of treatment.

Threats to external validity occur when treatment effects may not be generalized beyond the particular people, setting, treatment, and outcome of the experiment.

The Hawthorne effect occurs when people's behavior changes simply because they believe someone (e.g., researchers, persons in authority) is interested in them.

Quasi-Experiments

Quasi-experiments provide an important alternative when true experiments are not possible.

Quasi-experiments lack the degree of control found in true experiments; most notably, quasi-experiments typically lack random assignment.

Researchers must seek additional evidence to eliminate threats to internal validity when they do quasi-experiments rather than true experiments.

The one-group pretest-posttest design is called a pre-experimental design or a bad experiment because it has so little internal validity.

The Nonequivalent Control Group Design

In the nonequivalent control group design, a treatment group and a comparison group are compared using pretest and posttest measures. If the two groups are similar in their pretest scores prior to treatment but differ in their posttest scores following treatment, researchers can more confidently make a claim about the effect of treatment. Threats to internal validity due to history, maturation, testing, instrumentation, and regression can be eliminated in a nonequivalent control group design.

Nonequivalent Control Group Design: The Langer and Rodin Study

Quasi-experiments often assess the overall effectiveness of a treatment that has many components; follow-up research may then determine which components are critical for achieving the treatment effect.

Sources of Invalidity in the Nonequivalent Control Group Design

To interpret the findings in quasi-experimental designs, researchers examine the study to determine if any threats to internal validity are present. The threats to internal validity that must be considered when using the nonequivalent control group design include additive effects with selection, differential regression, observer bias, contamination, and Hawthorne effects. Although groups may be comparable on a pretest measure, this does not ensure that the groups are comparable in all possible ways that are relevant to the study outcome.

The Issue of External Validity

Similar to internal validity, the external validity of research findings must be critically examined. The best evidence for the external validity of research findings is replication with different populations, settings, and times.

Interrupted Time-Series Designs

In a simple interrupted time-series design, researchers examine a series of observations both before and after a treatment. Evidence for treatment effects occurs when there are abrupt changes (discontinuities) in the time series data at the time treatment was implemented. The major threats to internal validity in the simple interrupted time-series design are history effects and changes in measurement (instrumentation) that occur at the same time as the treatment.

Time Series with Nonequivalent Control Group

In a time series with nonequivalent control group design, researchers make a series of observations before and after treatment for both a treatment group and a comparable comparison group.

An Extension of Single-Case Designs to Research in Natural Settings

Rationale

Both time-series designs and single-case experimental designs use observations before and after a treatment in order to make inferences about the effectiveness of the treatment. With both types of designs, researchers must consider history threats as a plausible alternative explanation for apparent treatment effects. Single-case experimental designs can be conducted as quasi-experiments when groups of individuals are tested using the ABAB design or multiple-baseline design.

The ABAB Design

When the ABAB design is used in natural settings, treatment is alternately introduced and withdrawn (baseline) for a group of individuals. The effect of treatment can be observed by comparing records for observations made during treatment and baseline periods.

Program Evaluation

Program evaluation is used to assess the effectiveness of human service organizations and provide feedback to administrators about their services.

Program evaluators assess needs, process, outcome, and efficiency of social services.

The relationship between basic research and applied research is reciprocal.

Despite society's reluctance to use experiments, true experiments and quasi-experiments can provide excellent approaches for evaluating social reforms.