Are revisions to USDA crop production forecasts smoothed?

Agricultural markets are inherently unstable, primarily due to a combination of inelastic demand for food and production technology that is subject to the natural vagaries of weather, disease, and pests. Price volatility causes many agricultural firms to rely on forecasts in decision making. Consequently, the U.S. Department of Agriculture (USDA) devotes substantial resources to agricultural situation and outlook programs (Offutt 2002). Crop production forecasts are an especially prominent example of this effort. These forecasts affect business decisions by farmers and agribusiness firms and also have an impact on government policy. Furthermore, it is a commonly held belief of market participants that USDA crop production forecasts function as the "benchmark" to which other production estimates are compared. The dominant role of USDA crop production forecasts is not surprising given the classic public goods problem of private underinvestment in information (Wolf, Just, and Zilberman 2001) and the critical role that public information plays in coordinating the beliefs of market participants (Morris and Shin 2002). In light of the importance of USDA crop production forecasts and their extensive impact throughout the agricultural sector, it is important to understand the accuracy and reliability of these forecasts.

Several studies examined the accuracy of USDA crop production forecasts (e.g., Egelkraut et al. 2003) and their market impact (e.g., Sumner and Mueller 1989). However, an important aspect of production forecasts generally has been overlooked in the previous literature: the process used to revise the forecasts across the forecasting cycle. For example, the National Agricultural Statistical Service (NASS) of the USDA typically releases five forecasts of annual corn and soybean production for a given marketing year starting in August preceding the marketing year and ending in January of the marketing year. Thus, production forecasts for corn and soybeans are revised four times each marketing year. (1) Only one previous study has examined the revision process for USDA crop production forecasts. Gunnelson, Dobson, and Pamperin (1972) analyzed first and second revisions for seven crops over 1929-1970 and reported, "While a relatively high percentage of the revisions was successful, the revised forecasts tended to under compensate for the errors in the previous estimate. Thus, for example, if first crop forecasts underestimated or overestimated crop size, the first revision was likely to exhibit similar characteristics" (pp. 641-42). This study did not provide quantitative evidence on the magnitude of the "undercompensation" or conduct formal statistical tests. (2)

Some analysts argue that USDA production forecasts are "smoothed" or "conservative." That is, monthly forecasts of the same event are thought to change too slowly compared with available information. For example, a prominent market advisory service made this statement with respect to the June 2000 winter wheat production forecast: "NASS is going to be particularly sensitive about making a drastic reduction in their July and August estimate. ARC anticipates USDA will take a conservative approach and slowly reduce production levels in July, August and September" (AgResource Company 2000). A related point was made at the 2004 USDA data users meeting, where a representative of a large agribusiness firm "commented he had noticed that if NASS corn forecasts go up from September to October they almost always go up from October to November" (USDA 2004, p. 8). Investigation of the efficiency of the revision process for USDA crop production forecasts will provide evidence about the validity of these perceptions.

Nordhaus (1987) developed a formal framework to determine the efficiency of fixed-event forecasts, which are a series of forecasts of the same terminal event, like USDA crop production forecasts. Note the difference from a conventional rolling-event framework, where a series of forecasts of different events is examined. Nordhaus argued that the fixed-event approach may be more powerful than the rolling-event approach, especially in detecting tendencies to systematically adjust forecasts. Rolling-event tests may be unable to detect inefficiency associated with systematic adjustments because forecast errors concerning terminal events occurring at different times are uncorrelated. However, these adjustments would be revealed by the fixed-event approach.

Previous studies of macroeconomic forecasts using Nordhaus' framework (e.g., Clements 1995, 1997; Harvey, Leybourne, and Newbold 2001) provide substantial evidence of systematic adjustments in forecast revisions. However, this approach has not been applied to agricultural forecasts. Detection of systematic adjustments in forecasts is of interest because it implies that (a) if forecast revisions are correlated, then forecasts do not efficiently incorporate all available information, and, therefore, may be improved, and (b) knowledge about systematic adjustments can be used by market participants to compute adjusted forecasts.

The purpose of this article is to determine the efficiency of the revision process for USDA corn and soybean production forecasts over the 1970/1971 through 2004/2005 marketing years. These forecasts are of particular interest because corn and soybeans account for about 80% of total U.S. grain and oilseed production. The analysis includes parametric and nonparametric tests of forecast efficiency based on Nordhaus' framework. Parametric tests use regression analysis to determine whether revisions in adjacent months are correlated. Nonparametric tests use contingency tables and Pearson chi-square tests to determine whether revisions in adjacent months are made in the same direction. The use of two different tests provides evidence on the sensitivity of results to the selected test. A simulation is performed to estimate the potential gain in forecast accuracy due to adjustment of USDA corn and soybean production forecasts for observed smoothing. Finally, potential sources of smoothing are identified in an interview of USDA officials responsible for compilation of USDA crop production forecasts.

USDA Crop Forecasting Process

All phases of the crop forecasting process are conducted by NASS, an agency within the USDA. Production forecasts for corn and soybeans are released in August, September, October, and November, with final estimates published in January. (3) Corn and soybean production forecasts are based on estimates of planted and harvested acreage and two types of yield indications, a farmer-reported survey and an objective yield survey. The acreage figures are obtained from the USDA's June Agricultural Survey, conducted during the first two weeks of June and reported at the end of June. The June Survey is based on a large farm operator list frame and a separate and independent area frame survey. These acreage estimates are used in subsequent production forecasts until there is evidence (survey or other) to alter the acreage estimates. The farmer and objective yield surveys use the same sampling, survey and estimation procedures from year to year. This allows yield and production forecasts to be compared over time.

The farmer-reported yield survey is conducted for states with significant corn and soybean production. In 2004, 33 states were surveyed for corn and 29 for soybeans. Farmers included in the yield survey are randomly selected from the list frame (essentially a list of names, addresses, and phone numbers) of individuals that responded to the June Agricultural Survey. This assures that farmers included in the yield surveys are growing the crop of interest. Farmers are asked monthly (August through November) for a subjective prediction of their final corn and soybean yields. While the list frame changes across years, reflecting change in farming arrangements, the same individuals are surveyed each month for a particular crop year. Farmer-reported yield surveys are conducted primarily by Computer-Assisted Telephone Interviewing (CATI), but some data are collected by mail and face-to-face interviews.

The objective yield survey for corn and soybeans typically has been conducted only for the seven most important production states, but that number was expanded to ten states for corn and eleven states for soybeans in 2004. These "speculative" states generate about 70% of U.S. production for each of the crops. The objective yield survey is based on an area-frame sampling design, where fields are randomly selected from the total land area used in the production of a given crop. Mirroring the procedure for the farmer-reported yield survey, fields for the objective yield survey are randomly selected from the larger number surveyed in USDA's June Agricultural Survey. Sample fields are selected with a probability proportional to their size.