Mobile Edition
Print
Get the Mag
Weekly UpdatesRandom assignment. Assigning subjects to treatments so that each subject has an equal chance of getting in each treatment group provides the greatest assurance that treatment groups are similar prior to the implementation of the treatments. As long as sample sizes are large, this random assignment distributes the subjects' characteristics evenly across the different treatment groups. (23) The larger the sample being randomly assigned, the greater the similarity between the resulting groups, but samples of 20 to 30 subjects per treatment group are often sufficient to consider the different treatment groups as equivalent. (24)
Random assignment of individual consumers to treatments is easy when experiments are conducted in a laboratory or are conducted via post or e-mail. In those cases, the experimenter has control over which subjects get which treatment. In addition, many magazines and television-cable companies now have the ability to deliver distinct content to various (essentially random) subsets of their customers. This allows marketers to expose different people to different ads even though they are reading the same magazine or watching the same television show. Those people can then be contacted and asked to provide information used to compare the effectiveness of the different ads.
In some cases, random assignment of individuals to different treatments is not possible. For example, a restaurateur could not randomly assign individual dining parties in a field experiment that compares the effects on sales of playing two different styles of music over the sound system. In such cases, however, it is possible to use different units of analysis and to conduct true experiments by randomly assigning those units to treatments. A restaurateur could, for example, randomly determine which of two different styles of music are played each day for two months and could then compare the average daily sales under each style of music. In this case, any differences between days in the number and type of customers or other characteristics will be evenly distributed across the two treatment groups and any subsequent difference between me treatment groups in average daily sales can be safely attributed to the different styles of music. In general, researchers can assign many different units (e.g., individual consumers, multi-person dining parties, days, units of a restaurant chain) to treatment groups, but should make sure that those units are what are described by the outcome measures. (25)
If random assignment of individuals or other units of analysis is not practical, marketers can use a quasi-experimental design. To do this the marketer must try to anticipate all the variables that might affect the outcome variable and find naturally occurring units matched on those variables. Unfortunately, it is nearly impossible to anticipate all the relevant variables and find units that are perfectly matched thereon. Even if matched pairs could be found, it is possible that factors outside the experimenter's control could change one of the units during the course of the study and thereby create a new confound. For example, a competitor of one of two matched restaurants in a quasi-experiment could suddenly close, boost the other restaurant's sales, and confound the experiment. The internal validity of this simple quasi-experimental design falls far short of that for a true experiment with random assignment. There are a variety of more-complex quasi-experimental designs that help address different threats to internal validity, and marketers interested in conducting a quasi-experiment should consult experts about the options available. However, the internal validity of quasi-experiments is never as great as that of true experiments, so whenever practical, random assignment is the preferred method of assigning subjects to treatments.
External Validity
External validity is the extent to which an experiment's results apply or generalize to the real marketing environment of interest. External validity is threatened by differences between the real-world and experimental samples, treatments, measured behavior, or contexts. A common example of such a threat can be found in test marketing of new entree items by, for instance, McDonald's (e.g., the McRib sandwich). Restaurants (not only McDonald's) often label such items as special offers that are available "for a limited time only." The problem with that approach stems from the fact that the availability of the items will not remain limited if they are judged a success and permanently added to the menu. In other words, the experimental conditions differ from those to which the marketer wants to generalize the experimental results. This difference is important because limited availability increases demand for products. (26) Test markets that describe items as special offers available for a limited time generally i nflate the demand for those items and do not provide good estimates of the demand that item would generate as a permanent addition to the menu.
The way to ensure external validity is to make the features of the experiment similar to the features of the situation to which the experimental results will be generalized. Marketers should draw a sample that is representative of the actual consumers of the product or service, deliver the treatments to subjects in the same way and in the same context that they will be delivered in the marketplace, and measure the same outcome variable that managers want to affect in the marketplace. However, it is expensive and difficult (if not impossible) to make experiments similar in all respects to real-world situations of interest. Thus, marketers must often conduct experiments that differ in some ways from the situations to which they want to generalize the experimental results. For example, marketers often settle for nonrepresentative samples or measure attitudinal outcome variables when it is consumers' behavior that they ultimately want to affect. How much these differences affect the generalizability of the result s depends on the specifics of the case. Some things to keep in mind when evaluating the generalizability of results across samples and measures are discussed below.
People of different ages, sexes, and ethnicities, as well as people from different regions of the country or world, differ in terms of tastes, value priorities, and other factors that may affect their responses to marketing communications and offers. As a result, it is dangerous to draw conclusions about one group of people based on data about a different group of people. However, generalizing findings across groups of people can be reasonable when there are only small differences between the groups or when the differences that exist are unlikely to affect responses to the treatment. For example, researchers have found only small demographic and psychographic differences between the users of different brands within consumer-product and -service categories. (27) This suggests that marketers can run experiments on their own customers and safely generalize the results to all users of the product category. In addition, researchers have found that differences between African-American and Caucasian consumers do nor affect their responsiveness to point-of-purchase displays or price discounts. (28) This suggests that marketers can generalize findings about the effects of these tactics among one ethnic group to the other. The important thing to keep in mind is that differences between two groups of people do not necessarily make it inappropriate to generalize results from one group to the other. Only when those differences affect responsiveness to the experimental treatments is generalizability called into question.
The difficulty and expense of measuring purchase behavior in naturalistic experiments leads many marketing researchers to use self-reported attitudes, beliefs, or purchase intentions as outcome variables in experiments instead of using actual purchase behavior. As mentioned earlier, this practice is flawed because attitudes, beliefs, and intentions are weak predictors of actual behavior. (29) Consequently, treatments may affect attitudes, beliefs, and intentions, but not affect purchase behavior. Since purchase behavior is what marketers are ultimately trying to influence, that behavior should be used as the outcome variable in marketing experiments whenever possible.
When actual marketplace behavior cannot be measured in an experiment, researchers should measure consumer choice in an artificial situation that is structurally similar to the choice situation in the marketplace. Eric Marder developed an artificial-choice task (called STEP) that has similar choice options, information about each option, and ease of choosing each option as those that consumers face in the marketplace. He found that consumers' STEP choices closely parallel their marketplace choices. (30) Thus, choice tasks like STEP provide a reasonable alternative to marketplace choices when measuring the effects of marketing experiments.
Conclusion
To be as effective as possible, marketers should develop and test several potential courses of action before embarking on any of them. The best way to develop a variety of courses of action is to conduct exploratory or descriptive research. The best way to evaluate those options is to conduct a causal-research study that compares consumers' behavior when faced with various options. Experiments and quasi-experiments are underused, but are nevertheless powerful research tools that allow hospitality marketers to draw strong causal conclusions about the effects of pricing, design, and other changes on the amount of money customers spend, or the number of visits they make to an establishment.
Experiments should be designed to have statistical-inference validity, internal validity, and external validity. The perfect marketing experiment would have (1) a large sample size that was drawn randomly from the population the marketer wanted to make conclusions about, (2) random assignment of subjects to treatments, (3) treatments that are delivered in the same way and in the same contexts they would be delivered in the marketplace, and (4) measures of consumer choice, or purchase behavior, in the marketplace. Such an experiment would allow a marketer to identify with complete confidence the best option under consideration.
FEED