Clinical Hub,UW Health Clinical Tool Search,UW Health Clinical Tool Search,Questionnaires,Related

Development and Validation of the Nasal Obstruction Symptom Evaluation (NOSE) Scale

Development and Validation of the Nasal Obstruction Symptom Evaluation (NOSE) Scale - Clinical Hub, UW Health Clinical Tool Search, UW Health Clinical Tool Search, Questionnaires, Related

Development and validation of the Nasal Obstruction
Symptom Evaluation (NOSE) Scale
YUEH, MD, MPH, and MAUREEN T. HANNLEY, PHD, Houston, Texas, Durham, North Carolina, Milwaukee, Wisconsin, Seattle,
Washington, and Alexandria, Virginia
OBJECTIVE: The study goal was to validate a dis-
ease-specific health status instrument for use in pa-
tients with nasal obstruction.
sisted of a prospective instrument validation con-
ducted at 4 academic medical centers with 32
adults with nasal septal deformity.
METHODS: Prospective instrument validation oc-
curred in 2 stages. Stage 1 was the development of
a preliminary (alpha-version) instrument of poten-
tial items. Stage 2 was a test of the alpha-version for
item performance, internal consistency, and test-
retest reliability; construct, discriminant, criterion
validity, and responsiveness; and creation of the
final instrument.
RESULTS: Items with poor performance were elimi-
nated from the alpha-version instrument. In testing
the final instrument, test-retest reliability was ade-
quate at 0.702; internal consistency reliability was
also adequate at 0.785. Validity was confirmed us-
ing correlation and comparison analysis, and re-
sponse sensitivity was excellent.
CONCLUSIONS: The Nasal Obstruction Symptom
Evaluation Scale is a valid, reliable, and responsive
instrument that is brief and easy to complete and
has potential use for outcomes studies in adults with
nasal obstruction. (Otolaryngol Head Neck Surg
Surgical and medical treatments for nasal ob-
struction are a common part of otolaryngology
practice. Although symptoms of nasal obstruc-
tion can have several etiologies, such as muco-
sal congestion, turbinate hypertrophy, adenoid
hypertrophy, nasal mass, and others, deviation
of the nasal septum is a very common cause.
The de nitive treatment for a deviated nasal
septum is surgical correction, or septoplasty.
However, there is limited evidence to document
whether septoplasty is effective in improving
symptoms of nasal obstruction.
There are 2 potential ways to assess outcome
after nasal septoplasty: objective measures and
subjective measures. The objective assessment
of nasal obstruction is controversial, and there is
no agreement on an accepted measurement
There have been several prior studies of
patients subjective outcome after septo-
However, many of those studies were
retrospective, and the prospective studies have
used either nonvalidated survey instruments or
instruments that were not validated for the as-
sessment of nasal obstruction. Most studies us-
From the Bobby R. Alford Department of Otorhinolaryngol-
ogy and Communicative Sciences, Baylor College of Med-
icine (Dr Stewart), the Division of Otolaryngology—Head
and Neck Surgery, Department of Surgery, Duke Univer-
sity School of Medicine (Dr Witsell), the Department of
Otolaryngology and Communication Sciences, Medical
College of Wisconsin (Dr Smith), the Department of Oto-
laryngology—Head and Neck Surgery (EMW, BY), Univer-
sity of Washington School of Medicine (Drs Weaver and
Yueh), and the Surgery and Perioperative Care and Health
Services Research and Development Services (Drs Weaver
and Yueh), and VA Puget Sound Health Care System,
Seattle, WA, and the American Academy of Otolaryngolo-
gy—Head and Neck Surgery Foundation (Dr Hannley).
Supported by Career Development Award CD-98318 from
the Health Services Research and Development Service of
the Veterans Health Administration, Department of Veter-
ans Affairs (Dr Yueh).
Reprint requests: Michael G. Stewart, MD, MPH, Baylor
College of Medicine, The Bobby R. Alford Department of
Otolaryngology and Communicative Sciences, 6550 Fan-
nin St, Suite 1727, Houston, TX 77030; e-mail,
The views expressed in this article are those of the authors
and do not necessarily represent the views of the Depart-
ment of Veterans Affairs.
Copyright ' 2004 by the American Academy of Otolaryn-
gology—Head and Neck Surgery Foundation, Inc.

ing objective and subjective outcomes have sup-
ported the concept that septoplasty is effective
in achieving the outcomes measured, but there
has been no prospective study using an out-
comes instrument validated for nasal obstruc-
To perform a prospective assessment of subjec-
tive treatment outcomes, a validated outcome in-
strument is needed, which was the purpose of this
study. The development and validation of the out-
comes instrument, named the Nasal Obstruction
Symptom Evaluation (NOSE) Scale, was con-
ducted as part of a parallel prospective multicenter
observational clinical study. This multicenter
study was commissioned and funded by the Amer-
ican Academy of Otolaryngology—Head and Neck
Surgery Foundation and coordinated under the
auspices of its National Center for the Promotion
of Research in Otolaryngology.
We performed a multicenter prospective instru-
ment validation study in two phases. The study
was approved by the institutional review boards at
all participating institutions, and data collection
and follow-up were coordinated at the Duke Clin-
ical Research Institute. Data analysis was carried
out by one of the authors (M.G.S.) using SPSS
version 10.0 statistical software (SPSS Inc, Chi-
cago, IL).
Patients were enrolled consecutively into the
prospective protocol from the clinical practices of
the authors and their partners, using strict inclu-
sion and exclusion criteria. The enrollment period
was July 1, 2002, through January 31, 2003. All
septoplasty patients were eligible. Inclusion crite-
ria for the study were age of at least 18 years;
septal deviation consistent with presenting symp-
tom of chronic nasal obstruction; symptoms last-
ing at least 3 months; and persistent symptoms
after a 4-week trial of medical management, in-
cluding either topical nasal steroids, topical or oral
decongestants, or oral antihistamine—decongestant
combinations. Exclusion criteria were sinonasal
malignancy; radiation therapy to the head and
neck; septoplasty performed with concurrent sinus
surgery, rhinoplasty, or sleep apnea surgery; sep-
toplasty performed as access to other sites; prior
septoplasty, rhinoplasty, or turbinoplasty; history
or clinical evidence of chronic sinusitis (using the
Academy of Otolaryngology—Head and Neck Sur-
gery Rhinosinusitis Task Force de nition); septal
perforation; craniofacial syndrome; acute nasal
trauma or fracture in the past 3 months; nasal
valve collapse; adenoid hypertrophy; sarcoidosis;
Wegener s granulomatosis; uncontrolled asthma;
pregnancy; and illiteracy. Limited demographic
data were collected on patients who refused en-
rollment or were excluded.
An additional group of patients for between-
group discrimination was collected as a conve-
nience sample from the Baylor College of Medi-
cine and Duke University sites. Patients who were
seen in otolaryngology clinics for nonrhinologic
complaints, such as hearing loss or hoarseness,
were asked to complete the alpha-version of the
NOSE Scale.
Phase 1: Instrument Development
The authors, all of whom have had training and
experience in health services research, item selec-
tion, and instrument validation, reviewed all avail-
able disease-speci c instruments designed to as-
sess nasal symptoms, rhinitis, or sinusitis. Any
individual items pertaining to nasal obstruction
were extracted for further consideration. The au-
thors then developed additional items re ecting
other aspects of nasal obstruction as described by
patients. These obstruction-related items were
then grouped into a preliminary (alpha-version)
instrument, which contained 10 items (Fig 1). All
10 items were scored using a 5-point Likert scale
(not a problem, very mild problem, moderate
problem, fairly bad problem, and severe problem)
and were phrased, Over the past one month, how
much of a problem (was). . .? In the alpha-ver-
sion instrument, all aspects of nasal obstruction
that were covered in the existing sinonasal instru-
ments were included, and some new constructs
related to nasal obstruction were added, such as
panic and embarrassment.
Phase 2: Instrument Validation
A convenience sample of patients whose
planned surgery date was at least 1 week after the
enrollment visit were sent the alpha-version in-
Head and Neck Surgery
158 STEWART et al February 2004

strument again in 7 to 14 days, but before surgical
intervention—to assess test-retest reliability. Reli-
ability was assessed using the Goodman-Kruskal
� coef cient; a value of at least 0.70 was consid-
ered adequate test-retest reliability.
A table was constructed with individual items in
rows, and statistical attributes of the items, such as
mean, median, range, standard deviation, and
skewness, in columns. In addition, a bar graph of
response frequency for each item was created, to
Fig 1. The alpha-version of the NOSE instrument. (The NOSE Scale � 2003, the American Academy of Otolaryngology–Head
and Neck Surgery Foundation.)
Head and Neck Surgery
Volume 130 Number 2 STEWART et al 159

view the response distributions. Items with large
oor or ceiling effects, or skewed distributions of
responses (such a large proportion of patients re-
porting not a problem ), were marked for possi-
ble elimination.
Internal consistency reliability was assessed by
calculating the Cronbach � coef cient
noting item—total correlations. Internal consis-
tency reliability was considered adequate if at ��
. Items that did not contribute to the overall
internal consistency of the instrument were
marked for possible elimination.
Content validity was ensured during design of
the instrument in phase 1, as described earlier.
Construct validity was assessed using a combina-
tion of several techniques: principal components
factor analysis; item—item and item—total correla-
tions; and between-group discrimination. Principal
components factor analysis was performed on all
10 items using orthogonal varimax rotation of
All factors with an eigenvalue of greater
than 1.0 were retained for potential inclusion in
the nal rotated factor solution. Additional con-
struct validity was assessed by constructing a table
of item—item and item—total Spearman correlation
coef cients; a signi cant level of association was
set as a coef cient of �0.40. As different instru-
ment lengths were explored after individual
items were considered for elimination item—total
correlations were also assessed, again using Spear-
man coef cients and the same level of signi -
cance. Between-group discrimination (discrimi-
nant validity) was assessed by comparing total
scores between study patients and the sample of
patients seen for nonrhinologic complaints. The
Mann-Whitney U test was used to compare
groups; a signi cant difference was expected and
was de ned as P � 0.05.
Criterion validity was assessed by comparing
obstruction-speci c items with a summary visual
analog scale measuring dif culty in breathing
through your nose, which was included with the
alpha-version instrument. Statistical analysis was
performed using the Spearman correlation coef -
cient; a signi cant association was set at a coef -
cient of �0.40.
The results of individual item analysis were
combined with reliability and validity analysis to
construct a nal version of the instrument by elim-
inating items that were redundant, that showed
poor statistical distribution, and that did not con-
tribute to overall reliability and validity. This nal
version of the instrument was then evaluated for
response sensitivity.
Response sensitivity of the nal instrument
scoring only the nal included items was as-
sessed by calculating the standardized response
mean and the effect size
and comparing
those values with published standards. For both
measures, a value of approximately 0.2 represents
low sensitivity to change, 0.5 represents moderate
sensitivity, and 0.8 represents high sensitivity to
Data were used from the 3-month post-
operative data collection point. All patients had
undergone nasal open septoplasty or submucous
resection, after failure of medical therapy to re-
lieve obstruction.
A total of 32 patients were enrolled in the study,
from 4 sites of author af liation (Baylor College
of Medicine, Duke University, Medical College of
Wisconsin, and University of Washington) (25
men and 7 women; mean age, 47.0 years; age
range, 19 to 78 years). Twenty-one patients com-
pleted the test-retest portion of the validation pro-
cess. There is no sample size or power calculation
for psychometric validation, but a general rule of
thumb is that 25 to 50 patients make an adequate
sample for validation.
Initial item review of the alpha instrument dem-
onstrated that items 9 and 10 ( panic and em-
barrassment ) had poor statistical performance,
with a large oor effect, a low median, and a large
right (positive) skew, so those items were elimi-
nated. Performing principal components factor
analysis on the remaining 8 items revealed that
items loaded on 2 major components: items 1, 4, 5,
6, 7, and 8 loaded onto a primary factor, and items
2( sense of smell )and3( snoring ) loaded onto
a secondary factor. Analysis of internal consis-
tency of those 8 items demonstrated adequate in-
ternal consistency (�0.70) but also demonstrated
that items 2 and 3 each detracted from internal
consistency (ie, removal of each item resulted in a
higher value of Cronbach s �). Therefore items 2
and 3 were eliminated.
Head and Neck Surgery
160 STEWART et al February 2004

Next, interitem correlations were checked be-
tween the remaining 6 items. The correlation be-
tween items 5 ( trouble breathing through nose )
and7( having to breathe through mouth )was
very high at 0.698, demonstrating that the 2 items
were very close to measuring the same underlying
construct. Testing removal of each item demon-
strated that removal of item 7 resulted in a higher
internal consistency than removal of item 5, so
item 7 was removed. The instrument then con-
tained 5 items: items 1, 4, 5, 6, and 8. Internal
consistency reliability of this 5-item instrument
was high at ��0.785.
Test-retest reliability was assessed using the
total score on the 5-item instrument in a subgroup
of patients (n � 21). The mean time between
instrument administrations was 8.7 days. The re-
liability coef cient was adequate at ��0.702.
As part of the con rmation of construct validity,
the interitem and item—total correlations are
shown in Table 1. This analysis demonstrated
several expected associations. First, the items each
have a high correlation with the subscale total,
which is expected because items with poor item—
total correlation were eliminated. Next, the items
related to obstruction, congestion, and trouble
breathing had high levels of correlation with each
other but poor correlation with the item on sleep-
ing. The item on trouble breathing during exercise
had a high correlation with the item on trouble
breathing through the nose, but weak or absent
correlation with items on congestion and obstruc-
tion, and no correlation with the item on sleeping.
These ndings con rm that the items are measur-
ing related but distinct individual concepts that
nevertheless form a uni ed construct.
The group of nonrhinologic patients consisted
of 12 patients, and they had a mean raw score (on
the 5-item instrument) of 2.17, whereas the 32
study patients had a mean raw score on the same
instrument of 13.13 (Mann-Whitney U test, P �
0.001). Therefore the instrument demonstrated ex-
cellent between-group discrimination.
Criterion validity was assessed by measuring
the association of obstruction items with the single
summary visual analog scale. The values are
shown in Table 2, demonstrating that items assess-
ing obstruction correlate well with the visual an-
alog scale, and the item assessing sleeping dif -
culty did not correlate well. This analysis con rms
the criterion validity of the instrument.
To assess response sensitivity of the instrument,
we calculated the standardized response mean and
effect size for 21 patients who completed the in-
strument at the 3-month postoperative follow-up
period. The standardized response mean for the
5-item instrument was 1.66, which indicates very
high sensitivity to change. The effect size was
2.65, which also indicates very high sensitivity.
Table 1. Item–item and item–total correlations for the 5 item instrument
Congestion Obstruction
breathing Sleeping Exercise
Obstruction 0.665*
Trouble breathing 0.599* 0.58*
Sleeping 0.321 0.386 0.369
Exercise 0.327 0.447* 0.646* 0.398
Total 0.695* 0.753* 0.805* 0.738* 0.778*
Values shown are the Spearman correlation coef cient.
*Correlations greater than 0.40.
Table 2. Spearman correlation coefficients
between the visual analog summary scale and
individual items
Item Content Coefficient
1 Nasal congestion or stuf ness 0.64*
2 Nasal blockage or obstruction 0.55*
3 Trouble breathing through
4 Trouble sleeping 0.21
5 Unable to get air through nose
during exercise
*Correlations greater than 0.40.
Head and Neck Surgery
Volume 130 Number 2 STEWART et al 161

Because of the scoring algorithm for each item,
the range of raw scores on the nal instrument was
from 0 to 20. The instrument was then scaled to a
total score of 0 to 100 by multiplying the raw
score by 5. Because of item wording, a score of 0
means no problems with nasal obstruction and a
score of 100 means the worst possible problems
with nasal obstruction.
The nal version of the instrument is shown in
Figure 2.
Although global quality-of-life and health status
instruments are an important part of health status
assessment, for many conditions the changes in
health status are too subtle or disease speci ctobe
assessed using the content of a global instrument.
Therefore disease-speci c health status instru-
ments are needed.
This has been shown to be
true in many diseases, including visual loss from
and hearing loss.
We completed the validation of a disease-spe-
ci c instrument designed to assess nasal obstruc-
tion: the NOSE Scale. The instrument is brief and
easy to complete, with minimal respondent bur-
den. This is important if the instrument is going to
be given repeatedly in prospective trials. It is also
reliable, valid, and responsive to change in clinical
status, as demonstrated with the data presented
here. This means that scores on the instrument
remain consistent when the underlying patient s
status does not change (ie, there is little random or
spurious error in the assessment), that the instru-
ment is measuring what is supposed to measure,
and that the scores on the instrument do respond
when the patient s underlying status changes.
Like many similar instruments, the NOSE Scale
was validated for use in groups of patients, not
individual patients. Therefore it could be used for
Fig 2. The final version of the NOSE instrument. (The NOSE Scale � 2003, the American Academy of Otolaryngology–Head
and Neck Surgery Foundation.)
Head and Neck Surgery
162 STEWART et al February 2004

comparing disease-speci c health status between
groups of patients before and after treatment, or
used to compare the effects of different treatments,
for example, medical versus surgical therapy.
Similarly, it could be used to assess differences in
outcome when different surgical techniques are
used. It could also be used to compare symptom
severity between different groups of patients, for
example, those with and without nasal polyps.
However, it was not designed to be used with
individual patient data or to predict outcome in
The brevity of the instrument does not detract
from its sensitivity. In fact, studies have shown
that shorter instruments might be more sensitive to
change in clinical status than longer instru-
The NOSE Scale could also be used with a
global or generic quality of life instrument, to
assess the relative impact of the speci c disease on
different aspects of global quality of life. When
comparing the impact of different disease states,
those global data can be helpful. However, the
item content for many diseases is so unique that a
disease-speci c instrument is usually needed to be
able to assess the impact of disease treatment.
There are several other validated instruments
that are available for use in rhinology. These in-
clude the Chronic Sinusitis Survey (CSS),
Rhinoconjunctivitis Quality of Life Questionnaire
the Rhinosinusitis Disability Index
the Sino-Nasal Outcome Tool (SNOT-
and the Allergy Outcome Survey (AOS).
The CSS, RSDI, and SNOT-20 were all designed
to be used in patients with chronic sinusitis, and
not purely nasal obstruction. The RQLQ and AOS
were primarily designed for patients with allergic
rhinitis and conjunctivitis. All of these instruments
are valid and reliable and have been used success-
fully in prospective outcomes studies on chronic
infectious sinusitis and on allergic rhinitis.
We believe that the NOSE Scale is a valuable
and unique addition to this armamentarium of
valid and reliable instruments that can be used for
outcomes research in rhinology.
1. Katz S, Schmelzer B, Vidts G. Treatment of the obstruc-
tive nose by CO2-laser reduction of the inferior turbi-
nates: technique and results. Am J Rhinol 2000;14:51-5.
2. Pirila T, Tikanto J. Unilateral and bilateral effects of
nasal septum surgery demonstrated with acoustic rhi-
nometry, rhinomanometry, and subjective assessment.
Am J Rhinol 2001;15:127-33.
3. Passali D, Lauriello M, Anselmi M, et al. Treatment of
hypertrophy of the inferior turbinate: long-term results in
382 patients randomly assigned to therapy. Ann Otol
Rhinol Laryngol 1999;108:569-75.
4. Jessen M, Malm L. The importance of nasal airway
resistance and nasal symptoms in the selection of patients
for septoplasty. Rhinology 1984;22:157-64.
5. Siegel NS, Gliklich RE, Taghizadeh F, et al. Outcomes of
septoplasty. Otolaryngol Head Neck Surg 2000;122:228-
6. Stewart EJ, Robinson K, Wilson JA. Assessment of pa-
tient s bene t from rhinoplasty. Rhinology 1996;34:57-9.
7. Samad I, Stevens HE, Maloney A. The ef cacy of nasal
septal surgery. J Otolaryngol 1992;21:88-91.
8. Arunachalem PS, Kitcher E, Gray J, Wilson JA. Nasal
septal surgery: evaluation of symptomatic and general
health outcomes. Clin Otolaryngol 2001;26:367-70.
9. Dinis PB, Haider H. Septoplasty: long-term evaluation of
results. Am J Otolaryngol 2002;23:85-90.
10. DeVellis RF. Scale development: Theory and applica-
tions. Newbury Park: Sage Publications; 1991.
11. Aday LA. Designing and conducting health surveys: a
comprehensive guide, 2nd ed. San Francisco: Jossey-
Bass Publishers; 1996.
12. Lipsey MW. Design sensitivity: Statistical power for
experimental research. Newbury Park: Sage Publications;
13. Gliklich RE, Hilinski JM. Longitudinal sensitivity of
generic and speci c health measures in chronic sinusitis.
Qual Life Res 1995;4:27-32.
14. Liang MH, Fossel AH, Larson MG. Comparisons of ve
health status instruments for orthopedic evaluation. Med
Care 1990;28:632-42.
15. Patrick DL, Deyo RA. Generic and disease-speci c mea-
sures in assessing health status and quality of life. Med
Care 1989;27:S217-32.
16. Damiano AM, Steinberg EP, Cassard SD, et al. Compar-
ison of generic versus disease-speci c measures of func-
tional impairment in patients with cataract. Med Care
17. Stewart MG. Outcomes and patient-based hearing status
in conductive hearing loss. Laryngoscope 2001;111:11
18. Gliklich RE, Metson R. Techniques for outcomes re-
search in chronic sinusitis. Laryngoscope 1995;105:387-
19. Juniper EF, Guyatt GH. Development and testing of a
new measure of health status for clinical trials in rhino-
conjunctivitis. Clin Exp Allergy 1991;21:77-83.
20. Benninger MS, Senior BA. The development of the Rhi-
nosinusitis Disability Index. Arch Otolaryngol Head
Neck Surg 1997;123:1175-9.
21. Piccirillo JF, Merritt MG, Richards ML. Psychometric
and clinimetric validity of the 20-item Sino-Nasal Out-
come Test (SNOT-20). Otolaryngol Head Neck Surg
22. Kemker BJ, Corey JP, Branca J, et al. Development of
the Allergy Outcome Survey for allergic rhinitis. Otolar-
yngol Head Neck Surg 1999;121:603-5.
Head and Neck Surgery
Volume 130 Number 2 STEWART et al 163