Experiments and quasi-experiments: methods for evaluating marketing options; hospitality managers could achieve greater success

True field experiment: An experiment conducted in the real world. Field experiments use random assignment, but do not attempt to control all factors extraneous to the ones being manipulated. Field experiments are useful for answering applied hospitality marketing questions because they have high internal validity and high external validity.

Type-1 error: Concluding that the treatments being tested had an effect when they really did not.

Type-2 error: Concluding that the treatments being tested had no effect when they really did.--A.L. and M.L.

Adjusting Sample Size: A Temptation to Avoid

The procedure for deciding on a sample size described in the accompanying article is the correct method. However, the required sample sizes indicated by this method are usually large, and marketers often want to avoid the costs of working with such large samples. In those cases, marketers may be tempted to run an experiment with smaller sample sizes than the number recommended by standard procedure, analyze the results, and then add additional subjects if a practically meaningful but not statistically significant effect is found.

We advise against this two-step procedure for two reasons. First, the small initial sample sizes may result in chance reductions of the observed effect such that what is in reality a practically meaningful effect appears not to be so. In that case, marketers will not add subjects, and the statistical power needed to avoid this Type-2 error will not be available. Second, the decision to run the experiment with additional subjects only when there is a sizeable but not significant effect in the initial, small sample biases the final test with the larger sample and increases the probability of a Type-1 error. If marketers can afford the additional subjects required by the second step of this procedure, they should use that larger sample size in the first place.

Ann Lynn, Ph.D. is an assistant professor in the department of psychology at Ithaca College (alynn@ithaca.edu).

Michael Lynn, Ph.D. is an associate professor at the School of Hotel Administration at Cornell University (wml3@cornell.edu)

Print Get the Mag Weekly Updates