Mobile Edition
Print
Get the Mag
Weekly UpdatesABSTRACT
The National Institutes of Health Stroke Scale (NIHSS) is commonly used in the assessment of stroke severity. Nurses, who use the tool infrequently, find it difficult to use due to the neurologic terminology embedded in the scale. For this project, we modified the NIHSS by replacing the neurologic terminology for each component of the original scale with plain English. No components were deleted or changed; the language was merely simplified. Testing showed the modified tool to be reliable (0.96) and valid (0.977) when compared with the NIHSS.
**********
The National Institutes of Health Stroke Scale (NIHSS) was originally designed as a research tool to measure stroke severity (Brott et al., 1989). Over time, it has developed into the gold standard for stroke assessment and measurement (Richardson, Murray, House, & Lowenkopf, 2006). The NIHSS is currently used to determine treatment options, anticipate discharge planning, and measure patient outcomes (Adams et al., 1999; Schlegel et al., 2003; Weimar, Konig, Kraywinkel, Ziegler, & Diener, 2004). Specialized training is required to use the scale, in part because of the specialized neurology terminology used in the body of the tool (Andre, 2002). Online training in the NIHSS, offered by the American Stroke Association, can take, on average, 2 to 3 hours per trainee (American Heart Association, 2004).
The NIHSS is used for both initial assessments of stroke severity and monitoring changes in a patient's condition. The NIHSS gives a numerical value to stroke severity, allows caregivers to compare scores over time, and gives a common language to the stroke community. The American Heart Association's Get With the Guidelines collects data on indicators of quality stroke care, including whether the initial NIHSS was done on each stroke patient (American Heart Association, 2008).
Background
Providence Portland Medical Center (PPMC) is a 483-bed hospital in the Providence Health System, located in Portland, OR. PPMC admits approximately 400 stroke patients per year. PPMC is nationally recognized by the Joint Commission as a certified Primary Stroke Center by the American Stroke Association for exceeding national benchmarks using "Get With the Guidelines--Stroke" having received the 2007 Gold Level Award for sustained performance and by the American Nurses Credentialing Center as a Magnet Hospital for Excellence in Nursing. This study was done with the support of the Providence Brain Institute.
The NIHSS was first used at PPMC in patient care as an assessment tool with the opening of a new neuroscience unit in fall 2003. Floor nurses on the general neuroscience unit were taught to use the NIHSS as part of a formalized stroke treatment and recovery program. NIHSS education was further expanded shortly thereafter to include the intensive care unit (ICU) and Emergency Department (ED) nursing staff. Training originally consisted of reviewing the NIHSS training tapes in formal education sessions (Stroke Group, 1998), and PowerPoint presentations. With availability of the online American Stroke Association NIHSS program (American Heart Association, 2004), training has gradually moved to this standardized venue.
Over time, the neuroscience nursing staff has become very comfortable, confident, and efficient in completing the NIHSS because they use this tool daily in their patient assessment. The ED and ICU use the tool less frequently, which, we believe, accounts for their continued complaints that the tool is difficult to use, time-consuming, and intimidating. Their lack of confidence in their use of the tool was highlighted by NIHSS scores that often differed significantly from those obtained by the stroke team physicians and nurses.
Tool Development
In spring 2005, at the request of the ED and ICU nursing staff, we modified the NIHSS language by replacing neurologic terminology with plain, common-use English for each component of the original scale. No components were deleted and no changes were made in scoring; only the language was simplified. Both the stroke neurologist and stroke nurse had formal training and testing in the NIHSS through the American Stroke Association and years of experience using the NIHSS before rewording the NIHSS. The reworded scale was then reviewed by the Providence Stroke Program staff, including a second stroke neurologist and two additional stroke nurses, all formally trained in the use of the NIHSS, with further simplifications made. The ICU and ED nurse educators then reviewed the wording, making suggestions for further improvement that were incorporated. A final tool, which we called the Stroke Scale in Plain English (SSPE) was finalized in fall 2005.
Methods
To test for reliability and validity, the study was designed to compare scores obtained using the NIHSS versus those using the SSPE. Six study sessions were held, including a total of 46 nurse participants. Using Benner's novice-to-expert concept (Benner, 1984), nurses with varying levels of expertise and experience in using the NIHSS were enlisted to participate. Sixteen nurses were novice users and were trained in the use of the NIHSS and tested on the same day. Fifteen nurses worked in the ICU or ED and were considered competent users, using the tool two to four times per month. Fifteen nurses worked on the general neuroscience unit and were considered expert users, using the scale daily
In the first study session, nurse participants watched five videotaped stroke patients (American Academy of Neurology, 1999) using the NIHSS to score the patients. This same group of nurses then watched the same five videotaped stroke patients again using the SSPE. In subsequent study sessions, we alternated which scale was used first to score patients to reduce the chance of bias.
To test the reliability of the SSPE, we calculated interrater reliability and internal consistency. We used the intraclass correlation coefficient (ICC) as a measure of interrater reliability. Using terminology of Shrout and Fleiss (1979), the form of intraclass correlation coefficient chosen was ICC (2,1); that is, we operated as if we selected raters randomly after which each rater rated each patient (target).
For internal consistency, we used two measures: Cronbach's [alpha] (Cronbach, 1951) and omega (Heise & Bohrnstedt, 1970). Cronbach's [alpha] is the most commonly used method of estimating reliability. It has, however, one drawback: unless the scale is unidimensional, [alpha] will underestimate reliability. When used with a multidimensional scale, [omega] gives higher reliability estimates, and factor analysis of the NIHSS showed that it was multidimensional (Lyden et al., 1999). To calculate [omega], the rating scales must be factor analyzed, for which we used principal axis factoring. To determine concurrent validity, defined as the relationship between a new instrument and existing, valid measures, we calculated the correlation between the SSPE and the NIHSS. A high correlation between the two would show concurrent validity
In addition, we used the validity and invalidity formulas of Heise and Bohrnstedt (1970). Theirs is a unique method because it demonstrates validity through factor analysis. Their validity formula gives the correlation between the composite score of the measure being tested and each factor derived from factor analysis. The invalidity formula measures the correlation between the composite score and a factor that the composite score is not supposed to measure. With a tool like the NIHSS or SSPE, one expects that the total (composite) score would have a high validity score and low invalidity score on whichever factor best represents the total score. Usually, this factor is the first, unrotated, extracted factor. Thus, to the extent that the first factor represents stroke severity, a high validity score and low invalidity score would contribute evidence of the validity of the scale. For our analyses, we used SPSS Version15 (SPSS, Inc., Chicago, IL) and Excel 2000 (Microsoft Corp., Redmond, WA).
Missing Values
The data set had a handful of missing values and a legitimate answer "untestable," which carried no rating score. In these cases, we decided to impute scores for the missing values and for the valid but nonscored ratings. The imputation was completed by regressing an item with missing values on the other scale items. We used the resulting regression equation to impute a value, rounded to the nearest legitimate score (Table 1).
Total scores were calculated for all raters, regardless of missing data. We did not change these original, total scores despite value imputation, and, as described above, we used the total score for estimating interrater reliability. We used imputation to facilitate factor analysis and [omega], [alpha], validity, and invalidity calculations.
Results
As Table 2 shows, the reliability estimates of the NIHSS and SSPE are high. Because of the multidimensional nature of the two rating scales, the reliability estimate of co exceeds that of Cronbach's [alpha]. Moreover, both scales had high interrater reliabilities that were comparable to [omega]. Finally, the reliability estimates between the scales were comparable, showing that the SSPE had the same reliability as the NIHSS.
As Table 3 shows, the correlation between the SSPE and NIHSS--concurrent validity--was very high. Moreover, the Heise and Bohrnstedt (1970) validity and invalidity scores for both measures strongly demonstrate the validity of these scales as measures of stroke severity.
Conclusions
From study results and using several different testing measures, the SSPE is a highly reliable tool and comparable to the NIHSS. The SSPE also has high concurrent validity with the NIHSS, given that that NIHSS measures stroke severity.
FEED