Sampling strategy and survey administration

In order to achieve greater sample precision, it was decided to adopt a more focused sampling strategy rather than the more common 'shotgun' mail-out approach. The sampling strategy was designed to access a target population which was defined to be: CRC project leaders who had completed (or were soon about to complete) a CRC project that had been established with the intention of leading to some financial benefit (e.g., in the form of income or cost savings) for the CRC and/or at least one of its partners. Our interest here was in commercially-focused projects, as distinct from projects with a more academic or 'public good' knowledge focus.

The initial sampling frame of 456 potential respondents in selected CRCs (i.e., those most likely to be engaged in commercially-focused R&D) was derived from publicly-available sources (e.g., CRC annual reports and websites, reports in the mass media, etc.). The survey questionnaires were mailed out in April 2005, and in August 2005 a supplement to the initial sampling frame was identified from which a further 85 potential new respondents meeting the survey eligibility criteria were contacted. Thus, the total number of potential respondents was 541, and this covered all of the six CRC sectors. Also in August 2005 a website with the survey questionnaire was provided for those in the sample in an attempt to improve the response rate. Comparing key variables, there were no noticeable differences in the pattern of responses between the web-based and mail-out respondents and this allowed us to merge the two sub-samples for analysis.

Survey data analysis

Initial descriptive analysis of the sample data was conducted using SPSS for Windows (version 12.0.1), and to test the hypotheses structural equation modelling was conducted using PLS Graph version 3.0 (Chin, 2001). The partial least squares (PLS) method of latent structural modelling was chosen in preference to the more widely-used factor-based covariance approaches such as LISREL and AMOS because of the minimal demands PLS makes on measurement scales, sample size and distributional assumptions (Barclay et al. 1995; Chin 1998a, 1998b; Yi & Davis 2003). This approach was indicated because our achieved sample size was relatively small, and we knew from our initial descriptive analysis that there were distributional issues with the data. Further advantages of using the PLS method are that it can be used in an exploratory mode to suggest possible relationships in a model, thereby indicating propositions for later testing, and that it is better suited for analysis 'in situations of high complexity but low theoretical information' (Joreskog & Wold 1982: 270). Using PLS Graph we first assessed the measurement model, by examining the hypothesized links between the observed indicators and the latent constructs (all of the indicators were treated as reflective rather than formative because it was posited that the indicators or observed variables are manifestations of a construct, so any change in the construct will cause a change in the indicators), and then examined the structural model by estimating the hypothesized paths between the exogenous and endogenous latent constructs.

SURVEY FINDINGS

The achieved sample (n = 156)

At the end of the survey period, a total of 165 completed or partly-completed questionnaires had been returned by eligible respondents. Of these, 145 were paper questionnaires and 20 were responses to the web-based questionnaire. Of the 541 questionnaires that had been mailed out to potential respondents over the survey period, 215 were returned as incorrectly-addressed or ineligible (i.e., because the addressee did not meet the survey criteria as projects had not been completed or near completed or the data base contained inaccurate information). The survey response rate was therefore 165/326, or 51% of eligible respondents. However, of the 165 returned questionnaires, 9 had too much missing data and so were discarded as insufficiently complete leaving a usable sample of n = 156 giving a final response rate of 49%. Following the response wave extrapolation approach for estimating non-response bias (Armstrong and Overton 1977), an analysis of early versus late respondents was conducted by comparing the first 50 respondents with the last 50 on three descriptive variables: number of years as a CRC project leader, number of CRC projects completed as a project leader, and main employing organization. The statistical analyses revealed no significant differences between the two groups (suggesting that non-response was not a serious concern), and it was concluded that the achieved sample could reasonably be treated as representative of the population of interest.

The respondents were a diverse cross-sector of CRC project leaders, with around a half employed by a university and industry members being the smallest group in the sample (7%) which more or less reflects the situation with these projects generally. Experience as project leaders ranged from 1 to 20 years, with a median of 4 years, and the number of projects they had completed as a project leader ranged from 0 to 25 with a median of 2. Overall, these were reasonably experienced project leaders, with 81% having 3 or more years leadership experience and 58% having led 2 or more CRC projects to completion. We would thus conclude that these were sufficiently experienced project leaders to be considered as reliable informants on the nature and experience of their reported projects.

The sampled commercially-focused projects were also diverse, as shown in Table 1. A typical project in the sample contributed to one of the 3 main commercially-focused CRC sectors, involved at least 1 university and 1 industry partner, had 4 collaborating organizations, 6 dedicated project personnel, a budget of under A$1 million and a duration of no more than 4 years.

Assessing the measurement model

The first step in the analysis was to assess the proposed measurement model which was conducted, following conventional PLS practice (e.g., Fornell & Larcker 1981; Barclay et al. 1995; Chin 1998b; Hulland, 1999; Tenenhaus et al. 2005), by examining (a) individual item reliability for the construct measures, (b) the internal consistency of the measures, and (c) the discriminant and convergent validity of the construct measures. This process ensured that we had reliable and valid measures of the constructs before we proceeded to investigate the relationships among them in the model. After a first assessment and perusal of the resulting cross-loadings matrix, the initially-submitted measurement model was modified by the removal of 5 items, either due to low loading on their assigned construct (2 of the Contractual Trust items and 1 of the Goodwill Trust items, all of which had loadings of less than 0.40) or because of high cross-loadings with other constructs in the model (1 of the Competence Trust items and 1 of the Project Control Capability items). The results for the revised measurement model which was accepted as suitable for further analysis are shown in Table 2.

Individual item reliability was assessed by examining the loadings of each of the measurement items with their respective constructs in a matrix of item loadings by constructs (Appendix 2). The accepted rule of thumb here is to accept items with loadings of 0.70 or higher which indicate that there is more shared variance between the construct and its measures than error variance. In the revised measurement model all but 2 items met this criterion (their loadings were 0.66 and 0.67), but we accepted these as sufficiently reliable because they were very close to the rule of thumb and did not cross-load too highly with other constructs. We did not consider that these items were too laden with random error to render them unreliable, and so kept them in the model.

The indicator of internal consistency proposed by Fornell and Larcker (1981) was used to assess the composite reliability of the measures in the model. This measure is similar to Cronbach's alpha, but is considered superior in that it uses the loadings estimated within the causal model (whereas Cronbach's alpha assumes that each item in a measure contributes equally so the loadings are set to 1) and is not influenced by the number of items in the scale. The measure is interpreted in the same way as Cronbach's alpha, and a value of 0.70 is generally seen to indicate an adequate level of reliability (Nunally 1978). As shown in Table 2, all measures of the constructs in the revised model met this criterion (using both Cronbach's alpha and Fornell's composite reliability measure), and so were deemed to possess an acceptable level of internal consistency.

The two key elements of factorial validity, discriminant and convergent validity (together indicating how well the measurement items relate to their constructs; convergent validity is shown when each measurement item strongly correlates with its own construct, and discriminant validity when measurement items correlate weakly with the other constructs), were assessed following the procedures recommended by Gefen and Straub (2005). Firstly, the convergent validity of the first order measures was assessed by generating t values of the outer model loadings with a bootstrapping procedure. The t values for each of the measurement items were all much greater than 1.96 (they ranged from 8.67 to 57.11), demonstrating convergent validity of the measures used for the first order constructs. Secondly, discriminant validity was assessed using two procedures. The item (manifest variable) loadings on each of the first order constructs was determined using bivariate non-parametric correlation (with Spearman's rho) to generate the matrix shown in Appendix 2. This matrix shows that each of the measurement items loads most highly on the constructs to which they were assigned (in nearly all cases at 0.7 or higher), and with very few cross-loadings of higher than 0.50. The second procedure used the Average Variance Extracted (AVE, which is the average variance shared between a construct and its measures). The AVE for a construct should be greater than the variance shared between that construct and other constructs in the model, and is indicated in the correlation matrix when the square root of each construct's AVE is greater than the correlations of the construct to the other latent variables. As shown in Table 2, the diagonal elements in the correlation matrix (/AVE) range from 0.72 to 0.91 and all indicators load more highly on their own construct than on other constructs (i.e., the diagonal elements are greater than the off-diagonal elements in the matrix). On the basis of these analyses, we concluded that the measured constructs demonstrated an adequate level of discriminant and convergent validity.

Print Get the Mag Weekly Updates