Lecture 8

Data Collection III: Experimental Techniques

Summary Notes

The logic of conducting experiments

Experiments offer the best known scientific way to establish causality or causal relationship between variables. Three conditions have to be met for proving that and event A (presumed cause) actually causes event B (presumed effect):

  1. Co-occurrence (covariation): Both A and B must be observable and measurable, and that some sort of relationship exists between the cause and the effect (if A then B, if not A then not B)
  2. Sequence (temporal precedence): Cause must come first then the effect second (A precedes B)
  3. Elimination of alternative explanations: Eliminate other possible or plausible causes. Alternative hypotheses are tested.

Traditionally, an experiment involves (1) taking an action, and (2) observing its effect.

The experiment examines the effect of an independent variable (cause - an experimental stimulus or treatment) ON a dependent variable (effect). I.e. examine cause on effect. 

The Basic and the True Experiment

The basic experiment involves three pairs of components:

  1. Independent and dependent variables

  2. Pretesting and posttesting

  3. Experimental and control groups

The true experiment could do without the pretesting if subjects are randomly assigned to groups to ensure equivalency between groups.

Notations of experiments

Variables 

Variables come in different types depending upon their role in the experiment.

  1. Since the independent variable is the variable that influences another variable (the dependent variable), in this case it is the variable that is manipulated or systematically altered by the experimenter (also called experimental variable or predictor variable).

  2. Since the dependent variable is the one influenced by another variable (the independent variable), in this case it is the variable that is affected or is the outcome of the manipulation (also called outcome variable, response variable, criterion variable).

Another type of variable that is important in experimental designs is called the extraneous variable, which is any variable that may have an influence on the effect of the experimental manipulation. Sommer and Sommer (1997) explain three ways to control the effects of extraneous variables (read more on these three techniques in the required reading):

  1. eliminate or hold extraneous variables constant

  2. measure the extraneous variables to take them into account in the analysis

  3. use a control group or control conditions

The true experiment or simply the experiment has to achieve two conditions:

  1. Random assignment is possible: 

  2. Control over extraneous variables is possible

Between-subject design and Within-subject design:

 

In a two-level experiment (two levels or conditions of the independent variable) and when trying to decide upon how subjects are assigned to both experimental conditions, one is faced with two options. Either randomly assign a group of subjects to one condition and another group of subjects to the other condition (between-subjects design), or assign the same group of subjects for both conditions of the experiment (within-subjects design). 

 

Pre-experimental designs (NOT recommended)

  1. The one-shot case study (no pretest & no control group – misleading results)
  2. One-group pretest-posttest design (no control group – factor other than independent var. may be cause)
  3. Static group comparison (no pretest – initial status of groups unknown)

Hawthorne effect

Benefits of the control group

Random Assignment

Assigning subjects in random to both experimental and control groups to ensure equivalency of both groups in terms of the dependent variable. Random assignment is different from random selection (in sampling).

Experimental Variations

Experimental research designs may differ in several ways. The following are some of those variations:

  1. One group pretest-posttest

  2. Control group pretest-posttest (two group)

  3. Randomized Solomon four-group design

  4. Randomized control-group posttest only

  5. Nonrandomized control-group pretest-posttest

  6. Counterbalanced treatments

  7. One-group time-series

  8. Control-group time-series

Quasi-experiments

Quasi-Experimental research designs occurs when random assignment of subjects to groups is not possible.

Threats to Internal and External Validity

It is time to differentiate the difference between internal validity and external validity. 

Threats to Internal Validity (Sources of Internal Invalidity)

  1. History (effect of historical outside event)
  2. Maturation (subjects growing older, tired, hungry, etc.)
  3. Testing (Pretesting or posttesting influencing behavior)
  4. Instrumentation (measurement instrument between pretest & posttest, comparable)
  5. Statistical regression (extreme measures of dependent variable)
  6. Selection biases (equivalent, comparable groups)
  7. Experimental mortality (subjects dropping off)
  8. Causal time-order (did stimulus cause dependent var. or dependent var. caused changes in stimulus; rare)
  9. Diffusion or imitation of treatments (passing info from experimental group to control group)
  10. Compensation (effect of compensation on control group)
  11. Compensatory rivalry (deprived control group try to beat special experimental group)
  12. Demoralization (deprived control group give up)

.Solving Threats to Internal Validity

Use the basic experiment with random assignment.

Threats to External Validity (Sources of External Invalidity)

  1. Interaction between the testing situation & the experimental stimulus (testing interaction with the stimulus) – groups took pretest in experiment, groups will not take pretest in real life.

.Solving Threats to External Validity

Use Solomon 4-group design with random assignment of subjects into the groups.

Back to top

Topics for Discussion

The following items are topics for discussion during this lecture. Students should prepare their thoughts and ideas around these topics.

 

Back to top

Required Readings

  1. Sommer, B. & Sommer, R. (2002). A Practical Guide to Behavioral Research: Tools and Techniques, 5th ed. Oxford: Oxford University Press. (Chapter 6, pp.82-102, Experimentation).

Back to top

Favorite Links

Back to top

Student Work

Back to top

Assignments

Back to top

References

Back to top

Additional Links

Back to top | Previous lecture | Next lecture

Copyright © 2000 by Hisham S. Gabr. All rights reserved.

This page last revised: 09/05/03