EXPERIMENTAL AND QUASI-EXPERIMENTAL RESEARCH DESIGN
"Children are born scientists. They spontaneously experiment." (Buckminster Fuller)
Experiments are the Cadillacs or top guns of research design, the most expensive and powerful techniques you can use. Before even considering an experimental design, you need to ask the following:
Is it possible to precisely categorize the people, places, or things in your study?
Is is possible to select random people, places, or things in your study?
Is the process of random selection into experimental and control groups ethical and legal?
Criminal justice is one of those fields where you rarely find experimental designs, for the third reason above. The subjects we often use are defendants, prisoners, or agency personnel, and it's either unethical or illegal to benefit one group (with an experimental treatment) while depriving another group (the control group) of the treatment. You can get away with experiments in other social sciences, like education and psychology, if you're experimenting with relatively harmless educational techniques or some simple perception or memory test. In general, if there's any risk of harm from delivering or withholding services to anybody in your sample, you cannot use an experimental design.
In those situations where you can use an experimental design, there are certain procedures you need to follow. First of all, you need to randomly select a control group. This group must be statistically equal to the other group, a randomly assigned treatment group. Both groups must come from the same population, and "statistically equal" means that, even though selection was by randomization, you apply your knowledge of population characteristics to ensure that matching has occurred so that any extreme case in one group (someone 6'7" tall, for example) has a match or counterpart in the other group. Randomization will take care of some of this for you, as will ad-hoc statistical controls, but the procedure known as matching (which is more often associated with quasi-experiments) will take you a lot further. Matched subjects, or subjects, in the control group are NOT to be exposed to some treatment, intervention, or change that you introduce or manipulate. You can have more than one control and treatment group. You can have full and partial treatment groups, or treatment groups based on different variations of some treatment, intervention, or change. It's important that all your groups have about the same number of subjects in them, and as a general rule, you should have no less than 25 subjects in each group. You should also (as a matter of ritual) conduct a pretest on your dependent variable, at least, across all groups to develop a statistical baseline. The experiment is complete when you take a final measure, called the posttest, but you can make multiple posttests at any time during the experiment. Your findings should be interpreted primarily from differences in a posttest score between experimental and control groups.
The term blind experiment means that you've gone out of your way to make sure the subjects don't know which group they're in, the experimental or control group. The term double blind experiment means that you and your research helpers don't even know who's in what group. These precautions help protect your study from the Hawthorne Effect and the Placebo Effect. The Hawthorne effect refers to the tendency of subjects to act differently when they know they are being studied, especially if they think they have been singled out from some experimental treatment. The Placebo effect refers to the tendency of some subjects to think they are "cured" or sufficiently treated when they know about the research. By using placebos in your control group, you can neutralize or balance out the Hawthorne Effect, as well as possibly satisfy some ethical and legal qualms about withholding services from a control group.
The choice of which research design to use is related to the manifestation of the phenomenon within the population. The appropriateness of a design is also associated with a number of across-the-board concerns that affect your research from beginning to end, as the following table illustrates:
|Population characteristics||Experimental aims||Sampling design||Experimental design|
|Homogeneous random variation||Estimating and comparing population parameters||Simple random sampling||Completely randomised design|
|Heterogeneous with systematic and random variation||Estimating and comparing means||Stratified random sampling||Randomised block design|
|Trends||Analysis of pattern and process||Systematic sampling||Response
|Factorial structure||Estimating and comparing means for combinations of factors||Factorial designs||Factorial designs|
|Dependence relationship||Prediction of a value from a single predictor||Simple random sampling||Regression analysis|
|Prediction of a value from more than one predictor||Multiple regression|
The word "quasi" means as if or almost, so a quasi-experiment means almost a true experiment. There are many varieties of quasi-experimental research designs, and there is generally little loss of status or prestige in doing a quasi-experiment instead of a true experiment, although you occasionally run into someone who is biased against quasi-experiments. Some common characteristics of quasi-experiments include the following:
matching instead of randomization is used. For example, someone studying the effects of a new police strategy in town would try to find a similar town somewhere in the same geographic region, perhaps in a 5-state area. That other town would have citizen demographics that are very similar to the experimental town. That other town is not technically a control group, but a comparison group, and this matching strategy is sometimes called nonequivalent group design.
time series analysis is involved. A time series is perhaps the most common type of longitudinal (over time) research found in criminal justice. A time series can be interrupted or noninterrupted. Both types examine changes in the dependent variable over time, with only an interrupted time series involving before and after measurement. For example, someone might use a time series to look at crime rates as a new law is taking effect. This kind of research is sometimes called impact analysis or policy analysis.
the unit of analysis is often something different than people. Of course, any type of research can study anything - people, cars, crime statistics, neighborhood blocks. However, quasi-experiments are well suited for "fuzzy" or contextual concepts such as sociological quality of life, anomie, disorganization, morale, climate, atmosphere, and the like. This kind of research is sometimes called contextual analysis.
One of the intended purposes for doing quasi-experimental research is to capture longer time periods and a sufficient number of different events to control for various threats to validity and reliability. The hope is that the design will generate stable, reliable findings and tell us something about the effects of time itself. In fact, for a noninterrupted time series, the independent variable is usually time itself, for example, if you were monitoring rises and falls in crime rates and attributing it to changes in society over time. Almost all quasi-experiments are somewhat creative or unusual in what they attribute the cause of something to, and this is the case because we aren't using a true experiment where we manipulate some independent variable in order to assess causality. Instead, at best, we have a statistical baseline and some interventions that have occurred naturally (like the passage of a law) or were created by the researcher (such as some public relations campaign).
In quasi-experiments, the word "trend" is used instead of cause, and we are interested in finding the one true trend. Unfortunately, this kind of research often uncovers several trends, and the major ones are usually developed into "syndromes" or "cycles" while the minor ones are just referred to as normal or abnormal events. Say, for example, during the course of your research a bunch of college students from Florida State on spring break descended upon your town and started partying wildly. You might call this the "Florida State syndrome" or something like that. Say, for example, a series of full moons came close together during the course of your study. You might call this the "full moon cycle." The point is that neither of these would be the true trend, but they might be trends nonetheless.
Because quasi-experimental (as well as experimental) research designs tend to involve many different, but interlocking relationships between variables, it's advisable that the researcher engage in modeling the causal relationships. This allows identification of spurious and intervening variables, as well as a number of other variable relationship types like suppression effects. Spurious variables should be thrown out; intervening variables require multiplying the effects of two variables (interaction terms); and suppression refers to when part of a variable affects part of another variable even though the bivariate relationship is nonsignificant. Models also permit elaboration and specification. Elaboration is the process of reclassifying or subclassifying your variables, sometimes even switching around your independent and dependent variables. Specification is the process of making your dependent variable more narrow (e.g., applies only to left-handed, lower-class black males) and then multiplying some of your independent variables into a new, more powerful interaction term which has to be interpreted as some new kind of variable, not the additive sum of the original variables. A variety of causal modeling techniques exist (see Asher 1983), from the fairly simple use of crosstabulation tables to run partial correlation analysis to the more sophisticated, and rarely-seen technique of path analysis which is essentially a regression run of each variable on every other variable. In some undergraduate courses like this, students sometimes analyze crosstabs for almost the whole semester; that's how important some instructors think modeling is as a heuristic device for teaching research methods.
Instructions: For each of the following research studies, indicate whether the experimental procedures were carried out were correct or incorrect, and ethical or unethical. You must also tell me, in your own words, why you think so.
1. 50 students in classes the instructor was teaching volunteered to participate in an experiment. The instructor then randomly assigned 25 of them to an experimental group and 25 of them to a control group.
2. A police trainer is planning an experiment to test the effectiveness of a new training program which is not very popular among police officers. In order to prevent bias from entering into the experiment, the trainer decides not to give a pretest prior to starting the experiment.
3. At a very violent inner-city high school, a justice researcher is testing a new anti-school shooting program where hidden, but functional metal detectors are set up at one entrance to the school, which is called the experimental zone. No metal detectors are set up at the other end of the school, called the control zone, but students are told they are hidden anyway.
4. Correctional researchers develop a new device to improve inmate safety. They test the device on 50 newly admitted inmates. A pretest determines that 1 out of every 5 newly admitted inmates is injured within a 1 month period. A posttest determines that 0 out of every 5 newly admitted inmates is injured within a 1 month period. The researchers claim their device is a success.
5. Court personnel assign every 5th defendant to the best lawyer in town, and every 10th defendant to a student intern. All the rest get public defenders. The experiment is single-blinded so all subjects think they're getting the same legal resources.
Instructions: Indicate by name (true experiment; interrupted time series; noninterrupted time series; or contextual analysis) what kind of research each of the following situations describes, and why.
1. A before and after evaluation of the effectiveness of a new gun control law.
2. Citizen attitudes toward the death penalty from 1950 to the present.
3. Changes in police arrest rates since the Rodney King incident.
4. Changes in the Uniform Crime Reports since 1940.
5. A program analysis of community policing.
6. A longitudinal analysis of victimization trends.
7. Public support for the FBI and ATF after the Ruby Ridge and Waco incidents.
8. A comparison of the rise and fall of drug use in mostly black and mostly white high schools.
9. The effect of the TV show America's Most Wanted on citizen attitudes toward criminals.
10. The effects of declining morality on crime rates.
Asher, H. (1983). Causal Modeling. Beverly Hills, CA: Sage.
Campbell, D. & J. Stanley. (1963). Experimental and Quasi-Experimental Designs. Chicago: Rand McNally.
Cook, T. & D. Campbell. (1979). Quasi-Experimental Design. Chicago: Rand McNally.
Hagan, F. (2000). Research Methods in Criminal Justice and Criminology. Boston: Allyn & Bacon.
Key, J. (1999). "Experimental Research and Design," Oklahoma St. Univ. website, last accessed July 17, 2011.
Lasley, J. (1999). Essentials of Criminal Justice and Criminological Research. NJ: Prentice Hall.
Neuman, L. & B. Wiegand. (2000). Criminal Justice Research Methods. Boston: Allyn & Bacon.
Rosenberg, M. (1968). The Logic of Survey Analysis. NY: Basic.
Senese, J. (1997). Applied Research Methods in Criminal Justice. Chicago: Nelson Hall.
Spector, P. (1981). Research Designs. Beverly Hills, CA: Sage.
Last updated: July 17, 2011
Not an official webpage of APSU, copyright restrictions apply, see Megalinks in Criminal Justice
O'Connor, T. (2011). "Experimental and Quasi-Experimental Research Design," MegaLinks in Criminal Justice. Retrieved from http://www.drtomoconnor.com/3760/3760lect04a.htm accessed on July 17, 2011.