Using ex ante approaches to obtain credible signals for value in contingent markets: evidence from the field.

Accurately estimating the economic value of nonmarketed goods and services is essential for efficient public policy. While markets routinely provide signals of value for traded commodities, estimating values for goods and services that are not traded in markets provides a quandary for the policymaker. On the one hand, she can make use of market signals to estimate use values by utilizing revealed preference methods, such as travel cost or the hedonic approach. Alternatively, she may take a more holistic view and use a stated preference approach (e.g., contingent valuation), which to date is the only method that is capable of measuring the total economic value (use and nonuse) of a nonmarketed commodity. Yet, this approach presents its own set of challenges. In particular, some commentators have argued that contingent surveys are unreliable due to their hypothetical nature (e.g., Diamond and Hausman 1994).

Following Bohm's (1972) seminal work on estimating the demand for public goods, several dozen experimental studies have been undertaken to elucidate the relationship between hypothetical and real statements (see the literature review in List and Shogren [2002] and Harrison and Rutstrom [2005]). The weight of the evidence in this body of literature suggests that hypothetical bias--a divergence between behavior in real and hypothetical institutions--is often present, the implication being that it could be a significant problem for stated preference methods that use contingent markets. In response, economists have searched for ways to attenuate this bias. Following the recommendations of NOAA panel on contingent valuation (Arrow et al. 1993), Loomis, Gonzalez-Caban, and Gregory (1994) attempted to mitigate hypothetical bias by reminding respondents of their budget constraint and highlighting substitutes. While the authors find no evidence that these subtle changes in the survey instrument have an effect on subject responses, budget constraint and substitute commodity reminders have become standard practice for stated preference methods.

Other forms of ex ante adjustment of survey instruments were subsequently explored. (1) Cummings, Harrison, and Taylor (1995) introduced what has come to be known as "cheap talk" in the nonmarket valuation literature. (2) "Cheap talk" is a script that is instituted before value elicitation. The "cheap talk" script: (i) describes hypothetical bias and provides an example; (ii) reviews possible explanations for such bias; and (iii) encourages subjects to vote as if the valuation question were real (i.e., had real economic consequences). Cummings and Taylor (1999) test the "cheap talk" script against real and hypothetical referenda for contributions to public goods. They find strong evidence in support of this approach. In their trials where hypothetical bias was found, the bias largely disappeared when the "cheap talk" script preceded the hypothetical valuation question; voting behavior was not statistically different across real referenda and hypothetical referenda that included the cheap talk script.

The success of cheap talk has not been universal, however. List (2001) and Lusk (2003) find that a cheap talk script is effective in attenuating hypothetical bias only for certain classes of subjects--those with less market experience or less familiarity with the good being valued. Aadland and Caplan (2003) have been successful in attenuating hypothetical bias with a shortened cheap talk script, whereas previous research found that a shortened cheap talk script was not effective (Cummings, Harrison, and Taylor 1995; Poe, Clark, and Schulze 1997).

Other ex ante methods that have been introduced to attenuate hypothetical bias include a learning model in which respondents gain experience with the valuation mechanism in a real setting before the hypothetical setting is introduced. Bjornstad, Cummings, and Osbourne (1997) find that participation in a real referendum preceding a hypothetical one induces behavior in the subsequent hypothetical setting that is not distinguishable from behavior in a real referendum. Smith and Mansfield (1998) find similar results with a dichotomous choice mechanism.

While this lot of studies certainly has value, in contingent valuation surveys carried out in the field it is commonplace to present respondents with a realistic scenario, inducing them to believe that their responses have a degree of importance associated with them. Accordingly, in contingent markets it appears reasonable to assume that individuals' beliefs about whether their responses will actually be considered in policy circles varies--some may believe with a high degree of certainty that their responses are important, whereas others may have significant doubts. This characterization stands in stark contrast to that found in the studies cited above (i.e., real versus hypothetical statements of value).

This idea, identified as "realism" by Cummings and Taylor (1998) and "consequentialism" by Carson, Groves, and Machina (2000), suggests that stated preference survey designs that are "realistic" will induce subjects to truthfully reveal their preferences. (3) As discussed by Carson, Groves, and Machina, a binary choice referendum will be incentive-compatible assuming: (i) a weakly monotonic influence function (i.e., a higher proportion of supporting votes will not decrease the probability of provision), (ii) a coercive payment mechanism, and (iii) a closed valuation mechanism (i.e., the good cannot be provided in another way). The intuition is that if subjects believe that their responses have the potential to influence public policy, then there is no incentive for them to misrepresent their preferences. The "consequential" design approach can be applied in a straightforward manner: inform subjects that their responses matter in a probabilistic sense and they should truthfully reveal their preferences. (4)

The only papers of which we are aware that explore such a mechanism in an experimental setting are Cummings and Taylor (1998) and Carson et al. (2002), both of which utilize a referendum format. The results of Cummings and Taylor suggest that treatments utilizing low levels of probability (p [less than or equal to] 0.5) to link voting behavior to real economic consequences produce results not in accord with a binding referendum (p = 1.0), but voting behavior associated with higher probability levels (p = 0.75) cannot be distinguished from that of a binding referendum. On the other hand, Carson et al. find that subjects voting in probabilistic referenda (where probabilities of the referendum binding range from p = 0.20 to p = 0.80) do not behave differently than subjects voting in a binding referendum (p = 1.0).

While both cheap talk and consequentialism appear to have enjoyed a degree of success in gathering economic values that correspond to values obtained in binding elicitation mechanisms, to our knowledge no study has systematically compared responses across these ex ante methods with an incentive compatible instrument. What we offer in this article is precisely such a comparison. (5) To provide insights into the effectiveness of these ex ante methods within an otherwise identical protocol that is incentive compatible, we make use of a straightforward 2 x 4 experimental design with 256 subjects from an real marketplace--the sports card market. In order to foster incentive-compatibility, we incorporate the experimental design of Carson et al. (2002), which uses a majority voting mechanism that determines the transfer of n payments of a prespecified amount of money from the subjects to the experimental monitor, coupled with the delivery of n private goods to the subjects. The transfers of n pieces of sports memorabilia simulates the provision of a public good in the sense that either all n subjects pay the prespecified amount and receive an identical piece of sports memorabilia or none do. A coercive payment mechanism is utilized, and use of a private good ensures that the referendum is incentive compatible. Using identical written protocol, we conduct four distinct referenda: hypothetical, hypothetical-with-cheap-talk, consequential, and real.

Comparing behavior across these four treatments, we report two major findings. First, consistent with many other experimental results, our experimental evidence suggests that responses in the hypothetical referenda are significantly different from responses in the real referenda. Second, responses in the consequential and hypothetical with cheap talk treatments are, for the most part, statistically indistinguishable from responses in the real referenda. Yet, the data do hint that responses in the hypothetical with cheap talk treatments represent an upper bound on real responses. Our tentative conclusion is that accurate signals of value are most likely obtained from the subjects that view their decisions as being sufficiently consequential. However, since in the field the perception of consequences is subjective, the cheap talk design is likely to be a useful alternative, especially in those cases where the likelihood of successfully achieving consequentialism is small.

The remainder of this article proceeds as follows: Section 2 summarizes the experimental design; Section 3 discusses the results; Section 4 concludes with a discussion of how these results can potentially aid public policy decision making.

Experimental Design