Clinical Hub,UW Health Clinical Tool Search,UW Health Clinical Tool Search,Questionnaires,Related

A Multicenter Study Comparing Two Numerical Versions of the Edmonton Symptom Assessment System in Palliative Care Patients

A Multicenter Study Comparing Two Numerical Versions of the Edmonton Symptom Assessment System in Palliative Care Patients - Clinical Hub, UW Health Clinical Tool Search, UW Health Clinical Tool Search, Questionnaires, Related

Original Article
A Multicenter Study Comparing Two
Numerical Versions of the Edmonton
Symptom Assessment System in Palliative
Care Patients
interview. Intraclass correlation coefficients (ICCs) were calculated to assess rating
T6G 1Z2 Canada.
456 Journal of Pain and Symptom Management Vol. 41 No. 2 February 2011
� 2011 U.S. Cancer Pain Relief Committee
Published by Elsevier Inc. All rights reserved.
0885-3924/$ - see front matter
Results. In total, 1046 patients were screened. One hundred sixty were enrolled
and evaluable (female 51%, median age 61 [range 34e92], lung cancer 26%,
gastrointestinal cancer 22%). Mean ESAS scores ranged from 1.2 (nausea,
standard deviation [SD] 2.1) to 4.3 (appetite, SD 3.3). ICCs ranged from 0.65 to
0.83, with lowest scores (<0.8) for drowsiness, appetite, and well-being. Although
most patients rated both versions as very easy or easy to understand and complete,
the ESAS-r was significantly easier to understand than the ESAS (P¼ 0.008).
Significantly, more patients preferred the ESAS-r (39%) than the ESAS (14%,
P< 0.001) because of its definitions, clarity, and format.
Address correspondence to: Sharon M. Watanabe, MD,
Department of Symptom Control and Palliative
Care, Cross Cancer Institute, 11560 University
Avenue, Edmonton, Alberta,
E-mail: Sharon.Watanabe2@alb
Accepted for publication: April 20,
per site). Consenting patients completed the ESAS, ESAS-r, and a structured
palliative care sites in Canada and Switzerland,
Sharon M. Watanabe, MD, Cheryl Nekolaichuk, PhD, Crystal Beaumont, BSc,
Laureen Johnson, RN, Jeff Myers, MD, and Florian Strasser, MD
Division of Palliative Care Medicine (S.M.W., C.N.), Department of Oncology, University of Alberta,
and Department of Symptom Control and Palliative Care (S.M.W., C.B.), Cross Cancer Institute,
Edmonton, Alberta, Canada; Tom Baker Cancer Centre (L.J.), Calgary, Alberta, Canada; Faculty of
Medicine (J.M.), University of Toronto, Toronto, Ontario, Canada; and Oncological Palliative
Medicine (F.S.) and Division of Oncology/Hematology (F.S.), Department of Internal Medicine, and
Palliative Care Center (F.S.), Cantonal Hospital, St. Gallen, Switzerland
Context. The Edmonton Symptom Assessment System (ESAS) is a widely used,
self-report symptom intensity tool for assessingnine common symptoms inpalliative
care, with ratings ranging from 0 (none, best) to 10 (worst). Based on a ‘‘think-
aloud’’ study of 20 advanced cancer patients, the ESAS was revised (ESAS-r).
Objectives. To compare the consistency of patients’ symptom ratings and obtain
patient perspectives regarding ease of understanding and completion between the
ESAS and ESAS-r.
Methods. Cognitively intact patients (n¼ 160) were recruited from eight
using cross-sectional sampling (20

ts of th
ed by E
evelopment, intrarater reliability
mpling approach,
160 patients was
tive care sites in
monton (five sites,
Vol. 41 No. 2 February 2011 457Edmonton Symptom Assessment System Comparative Study
ing depression and anxiety, and understanding
the term well-being. Also, they often reversed
the scale for the symptom appetite (i.e.,
0¼ worst, 10¼ best as opposed to 0¼ best,
10¼ worst). Some patients suggested that the
Using a cross-sectional sa
a convenience sample of
recruited from eight pallia
Canada and Switzerland: Ed
described difficulties in distinguishing between
the symptoms of drowsiness and tiredness, rat-
Patients and Methods
and potential patient errors in symptom rat-
In a recent prospective ‘‘think-aloud’’
study of 20 advanced cancer patients,
ease of understanding
of the ESAS and ESAS-
about patient comprehension of the tool 2. To obtain patients’ opinions regarding
and completion
Conclusion. The ESAS-r retains core elemen
interpretation and clarity of symptom intensity
generation of ESAS development, with further
drowsiness, appetite, and well-being. J Pain Sym
� 2011 U.S. Cancer Pain Relief Committee. Publish
Key Words
Symptom assessment, advanced cancer, instrument d
The Edmonton Symptom Assessment System
is a self-report tool of symptom inten-
sity, initially developed for advanced cancer
patients. It includes nine common symptoms
of advanced cancer (pain, tiredness, nausea,
depression, anxiety, drowsiness, appetite, well-
being, shortness of breath), with the option of
adding a tenth patient-specific symptom. It is
designed to capture multidimensional symp-
tom profiles over time by obtaining repeated
quantitative measurements of symptom inten-
sity with minimal patient burden. The original
ESAS used visual analogue scales to rate symp-
tom intensity. In a subsequent version, they
have been replaced with 11-point numerical rat-
ing scales.
Since its original inception by Bruera and col-
leagues in 1991,
the ESAS has been adopted in
diverse palliative care programs and countries.
A recent bibliometric analysis of the ESAS
highlights a rapid and multinational uptake of
the tool over the past 15 years,
where it is
used for clinical, research, and administrative
A series of validation studies have
complemented the rapid clinical uptake of the
ESAS, providing further evidence for its psycho-
metric properties and clinical utility.
Although the ESAS was designed for self-
reporting, some concerns have been raised
time frame for rating the symptoms (‘‘now’’)
should be emphasized. Constipation was fre-
quently cited as an additional patient-specific
Based on concerns raised in the litera-
and the findings of the think-aloud
a revised version of the ESAS, the
ESAS-r, was created. The ESAS-r retains the
core elements of the ESAS (nine common
symptoms, option of adding a tenth symptom,
11-point numerical rating scales), with key revi-
sions focusing on symptom assessment time
frame, terminology, item order, and format
(see Methods for detailed description).
The hypothesis of the present study was that
the ESAS-r would be easier for patients to un-
derstand and complete than the current ver-
sion (ESAS). The two primary objectives for
this study were as follows:
1. To compare the consistency of patients’
symptom ratings between the two numer-
ical rating scale versions of the ESAS and
ESAS-r, using intrarater reliability esti-
mates; if patients’ interpretations of the
same symptom were different between
the two versions, then it was hypothesized
that intrarater reliability estimates would
be lower, particularly for problematic
items (e.g., drowsiness, tiredness, depres-
sion, anxiety, well-being, appetite).
e ESAS, with improved
ent. It represents the next
ion recommended for
Manage 2011;41:456e468.
lsevier Inc. All rights reserved.

458 Vol. 41 No. 2 February 2011Watanabe et al.
n¼ 100), Calgary (n¼ 20), Toronto (n¼ 20),
St. Gallen (n¼ 20). These sites were purpose-
fully selected, being limited to programs offer-
ing specialist palliative care services, including
tertiary palliative care units, inpatient hos-
pices, acute care hospital consultation services,
and outpatient consultation clinics.
Cognitively intact patients who had been
referred to the participating palliative care
programs were eligible. Approval was obtained
from the research ethics board for each site.
is a self-report symptom as-
sessment measure, consisting of nine symp-
toms, with the option of adding a tenth
patient-specific symptom. Patients rate the in-
tensity of each of these symptoms, using an
11-point numerical rating scale, ranging from
0 (symptom absent or best) to 10 (worst possi-
ble). Each symptom rating is interpreted inde-
pendently, although it is also possible to
calculate a total symptom distress score. How-
ever, previous research has demonstrated that
the analysis of individual symptom ratings
provides a more meaningful representation
of the symptom experience than a total
symptom distress score.
ESAS validation
studies have focused on gathering reliability
content validity evidence,
concurrent validity evidence,
tive validity evidence,
and sensitivity and/or
A copy of the tool appears
in Appendix A.
ESAS-r. The ESAS-r is amodified version of the
ESAS. The time frame for symptom ratings is
specified as ‘‘now.’’ Brief definitions have been
added for the following potentially confusing
symptoms: tiredness (lack of energy), drowsi-
ness (feeling sleepy), depression (feeling sad),
anxiety (feeling nervous), and well-being (how
you feel overall). ‘‘Appetite’’ has been changed
to ‘‘lack of appetite’’ to express the concept as
a symptom. The order of symptoms has
changed: related symptoms (e.g., tiredness
and drowsiness; nausea and appetite; depres-
sion and anxiety) are grouped together, and
‘‘well-being’’ is now the ninth symptom at the
end of the instrument. The example of ‘‘consti-
pation’’ has been added to the tenth item,
‘‘other problem.’’ The horizontal lines over
the numbers have been removed. Every second
scale is shaded in gray to improve readability. A
copy of the tool appears in Appendix B.
Three research team members (SMW, CN,
CB) developed a draft version of the ESAS-r,
which was subsequently reviewed by other
members of the research team. All team mem-
bers had extensive clinical and research expe-
rience in symptom assessment and in using
the ESAS in advanced cancer and palliative
populations. Definitions were inserted for
problematic items previously identified by par-
ticipants in the think-aloud study, using spe-
cific wording generated by patients from this
Feedback was also obtained from an
interdisciplinary group of palliative care clini-
cians and researchers (Edmonton), who rou-
tinely use the ESAS in their daily practice.
ESAS and ESAS-r (German). For the German-
speaking Swiss subsample of patients (n¼ 20),
a professional translator developed German
versions of the ESAS and ESAS-r, in close collab-
oration with the research team members in St.
Gallen. Using a standard translation protocol,
a forward and backward translation was used,
in which the tool was translated from English
to German, back to English, and then finalized
in German.
Palliative Performance Scale. The Palliative Per-
formance Scale (PPS)
is a measure of perfor-
mance status in palliative care patients, based
on ambulation, activity, evidence of disease,
self-care, intake, and level of consciousness.
Ratings are in 10% increments, ranging from
0% (death) to 100% (fully functional). The
PPS is used to routinely assess patients’ func-
tional status as a part of standard clinical
Folstein Mini-Mental State Examination.
Mini-Mental State Examination (MMSE) is
a screening tool for assessing five domains of
cognitive functioning: orientation, memory, at-
tention and calculation, recall and language.
Scores range from 0 (total impairment) to 30
(fully intact), with adjustments for age and
The MMSE is used to routinely as-
sess patients’ cognitive status as a part of stan-
dard clinical practice. The MMSE obtained
closest to the time of study participation was
recorded. In some instances, the MMSE score
was below the expected normal value, but the

intestinal (22%). Sixty-two percent of patients
Vol. 41 No. 2 February 2011 459Edmonton Symptom Assessment System Comparative Study
site investigator deemed that cognitive func-
tion had normalized clinically.
Edmonton Classification System for Cancer Pain.
The Edmonton Classification System for Can-
cer Pain (ECS-CP) classifies cancer pain ac-
cording to five features: pain mechanism,
incident pain, psychological distress, addictive
behavior, and cognitive function. The ECS-CP
was included in this study to describe the
pain features of this sample.
Structured Interview. The structured interview
consisted of seven questions focusing on pa-
tients’ familiarity with the ESAS, as well as
ease of understanding, ease of completion, as-
sessment of differences and preferences be-
tween the two versions. Patient opinions were
elicited using quantitative ratings (5-point
Likert scales) and qualitative comments. The
interview questions were developed by the re-
search team, using parallel questions to those
used in the previous think-aloud validation
A research nurse or assistant screened cur-
rent and new admissions to each palliative
care service for study eligibility. Patients’ cogni-
tive status was assessed, based on the most re-
cently documented MMSE score on the
patient’s chart and the clinical opinion of the
attending physician, which mirrors clinical
Before data collection, a research nurse or
assistant informed eligible patients about the
study and obtained written consent. All con-
senting patients independently completed
the ESAS and then the ESAS-r, in the presence
of the research nurse or assistant. The instru-
ments were not completed in random order,
as patients were expected to have been already
exposed to the ESAS during the course of care
by the palliative care service, thereby diminish-
ing the value of randomization. Thereafter, the
research nurse or assistant administered the
structured questionnaire to elicit patients’
opinions regarding the two versions.
The following information was obtained
from the patients’ medical records: age, gen-
der, primary cancer diagnosis, current cancer
treatment, education level, PPS, MMSE, and
ECS-CP scores.
had previously completed the ESAS. Average
scores for the ESAS and ESAS-r are summa-
rized in Table 2. Mean scores ranged from
1.2 for nausea (ESAS, standard deviation
[SD] 2.1) to 4.3 for appetite (ESAS, SD 3.3).
Most patients ratedboth versions as very easy or
easy to understand (ESAS, 78%; ESAS-r, 83%) and
complete (ESAS, 87%; ESAS-r, 89%). However, as
shown in Table 3, the ESAS-r was significantly
easier to understand than the ESAS (P¼
0.008). There were no significant differences
in ease of completion between the two forms.
Sixty-two patients (39%) preferred the ESAS-r,
which was significantly more than those who
preferred the ESAS (n¼ 22, 14%, P< 0.001).
Less than half (n¼ 76, 47%) reported no pref-
erence for either form. Based on qualitative
comments, the most frequent reason for
ESAS-r preference was the inclusion of
Data Analysis
Data from the eight sites were pooled and
analyzed aggregately because of the small sam-
ple size at each site. Quantitative data were an-
alyzed using descriptive statistics. Independent
and paired samples t-tests, as well as the Pear-
son Chi-squared test, were used for group com-
parisons. Significance levels were set at 0.05.
Intraclass correlation coefficients (ICCs) were
calculated to obtain intrarater reliability esti-
mates. For the purposes of this study, a reliabil-
ity estimate of 0.8 or higher was considered
a reasonable estimate.
Using conservative as-
sumptions, if the ICC is about 0.80, a sample
size of 160 would yield a 90% confidence inter-
val of about 0.75e0.85. Qualitative comments
were analyzed using content analysis to iden-
tify common themes.
Participant Description
In total, 1046 patients were screened be-
tween June 12, 2008 and March 31, 2009.
Three hundred twenty eligible patients were
approached regarding their willingness to par-
ticipate in the study. One hundred sixty pa-
tients consented and completed the study.
Participant characteristics are described in
Table 1. Most were inpatients (84%), with a me-
dian age of 61 years and an equal distribution
of men and women. The two most common
cancer diagnoses were lung (26%) and gastro-

responses as either a 4 (n¼ 1) or a 5 (n¼ 1).
Fig. 1 illustrates a similar pattern of distribu-
460 Vol. 41 No. 2 February 2011Watanabe et al.
Table 1
Patient Characteristics (n¼ 160)
Characteristics Frequency, n (%)
Female 82 (51)
Primary cancer diagnosis
Lung 41 (26)
Gastrointestinal 35 (22)
Genitourinary 29 (18)
Breast 21 (13)
Other cancer 31 (19)
Noncancer 5 (3)
Current cancer treatment
definitions (n¼ 30).Other reasons, in descend-
ing order, included clarity (n¼ 19), format
(n¼ 16), and inclusion of an example for the
tenth patient-specific symptom (n¼ 7). There
were no significant demographic differences
in age, gender, education level, or PPS between
patients who preferred the ESAS-r and those
who either preferred the ESAS orhadnoprefer-
ence. The groups also did not differ signifi-
cantly in terms of prior ESAS use.
When asked how different their responses were
between the ESAS and ESAS-r, about one in four
None 106 (66)
Systemic therapy 31 (20)
Radiotherapy 13 (8)
Systemic and radiotherapy 2 (1)
Unknown 8 (5)
Education level (years)
0e4 4 (3)
5e8 10 (6)
9e12 85 (53)
Greater than 12 59 (37)
Unknown 2 (1)
Pain classification
Neuropathic pain 44 (28)
Incident pain 69 (43)
Psychological distress 36 (23)
Addictive behavior 14 (9)
Cognitive status impaired 6 (4)
Previously filled out ESAS
Yes 99 (62)
No 58 (36)
Unknown 3 (2)
Patient setting
Inpatient 134 (84)
Outpatient 26 (16)
Median (range)
Age (years) 61 (34e92)
Folstein MMSE (0e30)
29 (0e30)
Palliative performance scale
50 (20e90)
Percentages do not total 100 because of one participant having
more than one primary cancer diagnosis.
Scores on admission and/or most recent chart documentation;
for some patients, cognitive status had significantly improved by
the time of recruitment.
tion, when responses were grouped according
to participants’ expressed form preference
(i.e., ESAS, ESAS-r, no preference).
When asked how important the differences were
between the two forms, 41% of participants re-
ported that the differences were not important
(rating¼ 1). The remaining participants
(59%) reported varying levels of importance,
with 26 patients (16%) rating the differences
between the two forms as important to very im-
portant. Fig. 2 illustrates this similar pattern of
distribution, when responses were grouped ac-
cording to participants’ expressed form prefer-
ence (i.e., ESAS, ESAS-r, no preference).
ICCs ranged from 0.65 (well-being) to 0.83
(depression, tiredness, pain), with scores of
less than 0.8 for drowsiness (ICC¼ 0.79), appe-
tite (ICC¼ 0.74), and well-being (ICC¼ 0.65)
(Table 2). Figs. 3e5 illustrate the variability in
responses for these three symptoms, using box
plots. Box plots provide a visual representation
of distributions by summarizing the median,
quartiles (box), 95% interval (whiskers), and
extreme values (highest and lowest). For each
symptom, box plots were created by comparing
participants’ ESAS-r scores (X-axis) with their
corresponding ESAS scores (Y-axis). If there
were a perfect correlation between ESAS and
ESAS-r scores, then it would be expected to
see a single point for each numerical rating
from 0 to 10, which would align in a straight
line. As shown in these figures, there was consid-
erable variability in scores, and in somecases, re-
versal of scores between the two versions. For
example, some participants who rated their
appetite and well-being as a 10 on the ESAS
reversed their score to 0 on the ESAS-r (Figs. 4
and 5).
In this prospective multicenter study con-
ducted in a variety of palliative care settings, pa-
tients’ perceived ease of understanding and
completion of both forms of the symptom
participants (28%) reported no difference in
their responses, with most ratings (99%) rang-
ing from 1 to 3, on a 5-point Likert scale
(1¼ no difference, 5¼ very different). Only
two participants rated the difference in their

Table 2
ICCs for Symptom Ratings Using ESAS and ESAS-r (n¼ 160) and Average Symptom Ratings for ESAS
and ESAS-r (n¼ 160)
Symptom ICC Confidence Interval (95%)
Mean (SD)
Depression 0.83 0.78e0.88 1.8 (2.2) 2.1 (2.3)
Tiredness 0.83 0.78e0.87 4.1 (2.7) 3.9 (2.8)
Pain 0.83 0.77e0.87 2.9 (2.5) 3.0 (2.7)
Shortness of breath 0.82 0.76e0.87 2.2 (2.7) 2.2 (2.7)
Anxiety 0.81 0.75e0.86 2.2 (2.4) 2.2 (2.4)
Nausea 0.80 0.74e0.85 1.2 (2.1) 1.3 (2.4)
Drowsiness 0.79 0.72e0.84 3.3 (2.7) 3.6 (2.9)
Appetite 0.74 0.66e0.81 4.3 (3.3) 3.6 (3.3)
Well-being 0.65 0.55e0.73 4.0 (2.6) 3.6 (2.8)
Vol. 41 No. 2 February 2011 461Edmonton Symptom Assessment System Comparative Study
intensity tool were high. However, the ESAS-r
was significantly easier to understand (P¼
0.008), with significantly more patients prefer-
ring the ESAS-r to the ESAS (P< 0.001). Al-
though most patients did not perceive their
responses to be different between the two ver-
sions, themoderate ICCs, particularly for drows-
iness, appetite, and well-being, suggest that
there was considerable variability in responses
between forms. These findings suggest that
the ESAS-r can replace the ESAS. However, on-
going prospective validation evidence for
drowsiness, appetite, and well-being, for exam-
ple, with comparable quantitative measures,
are needed.
Approximately 40% of patients preferred
the revised version of the ESAS to the original
version, with the most common reason being
the definitions. In some clinical settings, the
ESAS is completed by patients with guidance
from a health care professional. In this situa-
tion, the inclusion of definitions may not be
as critical, as the health care professional is
available to provide clarification. However,
these definitions can be helpful for training
new staff in administering the ESAS, as well
Table 3
Comparison of ESAS and ESAS-r in Terms of Ease of U
Preference (n¼
Survey Question
Mean (SD)
Q1. How easy was it to understand each form?
1.81 (1.2)
Q2. How easy was it to fill out each form?
1.48 (1.0)
n (%)
Q3. Overall, which form do you prefer? 22 (14)
Scale¼ 1 (very easy) to 5 (very hard).
Paired sample t-test for Q1 and Q2, one sample Chi-squared test for Q3.
as for ensuring reasonable consistency across
clinicians in terms of their explanations of
symptoms, particularly for abstract concepts
such as well-being. Furthermore, the use of
electronic kiosks or the Internet for self-
reporting ESAS scores for the purpose of
screening for symptom distress has been de-
As patients do not have immediate
access to a health care professional while com-
pleting the ESAS using these modalities, the
inclusion of definitions would be helpful to as-
sist with interpretation of potentially confusing
Substantive differences in symptom intensity
ratings between the ESAS and ESAS-r were ob-
served for three of the defined symptoms,
drowsiness, appetite, and well-being, with ICCs
below 0.8 for these three symptoms. Although
the ICCs were above 0.8 for the remaining symp-
toms, the highest ICC was 0.83 with moderately
wide confidence intervals, suggesting consider-
able variability in responses between the two
forms. These response differences may have
been because of differences in interpretation
of the terminology, potentially suggesting an
improvement in scores when definitions were
nderstanding, Ease of Completion, and Patient
No Preference P-value
Mean (SD)
1.66 (1.0) d 0.008
1.41 (0.9) d 0.149
n (%) n (%)
62 (39) 76 (47) <0.001

rating of difference

No preference
Fig. 1. Overall, how different were your responses between Form A (ESAS) and Form B (ESAS-r)? (n¼ 160).
462 Vol. 41 No. 2 February 2011Watanabe et al.
used for clarification. This is supported, in part,
by the finding that the ESAS-r was significantly
easier to understand than the ESAS and partic-
ipants’ qualitative comments regarding the use-
fulness of definitions. Noguera et al.
demonstrated that using different words to
express anxiety and depression in the Spanish
version of the ESAS affected the screening per-
formance of the scales, when compared with
the Hospital Anxiety Depression Scale as
a gold standard.
The symptom of drowsiness has previously
been identified as being confusing, particularly
32tnatropmi ton-1
rating of importan

Fig. 2. Overall, how important is the difference between
when compared with tiredness.
The inclusion
of definitions for both these symptoms, in the
ESAS-r, provides greater clarity regarding their
differences. By grouping them together, pa-
tients may compare their responses more read-
ily and make adjustments to their ratings if
necessary, if they have difficulty distinguishing
between the two symptoms. The presence of
a health care professional during tool adminis-
tration, particularly for patients who are not
familiar with the ESAS, could further help pa-
tients clarify these two symptoms to ensure
consistency in interpretation of symptoms.
tnatropmi yrev-54
No preference
Form A (ESAS) and Form B (ESAS-r)? (n¼ 158).

Fig. 3. Distribution of drowsiness scores (ESAS vs.
Vol. 41 No. 2 February 2011 463Edmonton Symptom Assessment System Comparative Study
One item that had been previously identi-
fied as problematic was ‘‘appetite,’’ the ratings
for which were often reversed.
In the
ESAS-r, the term was changed to ‘‘lack of appe-
tite’’ to convert it to a symptom, in keeping
with the other items in the tool. Qualitative
comments from patients suggested that the
use of a double negative (‘‘no lack of appe-
tite’’) was confusing. The moderate ICC for
appetite (r¼ 0.74) further reinforces this in-
consistency in responses between the two
forms, which may have been related to difficul-
ties in item interpretation.
Fig. 4. Distribution of appetite scores (ESAS vs.
The symptom of well-being can be difficult
for patients to comprehend, as suggested by
previous research.
The inclusion of the defi-
nition in the ESAS-r, ‘‘how you feel overall,’’
encourages patients to consider this symptom
as an overall assessment of their health, in
view of their other symptoms. The placement
of this symptom at the end of the instrument
further reinforces its use as an overall summary
of their symptoms.
The inclusion of the example of ‘‘constipa-
tion’’ at thebottomof the ‘‘other problem’’ scale
in the ESAS-r may have introduced bias by
prompting patients to report constipation as
opposed to other problems. However, while re-
porting of constipation increased with the
ESAS-r compared with the ESAS (39 patients
vs. 12 patients, respectively), reporting of prob-
lems other than constipation was not substan-
tially decreased (34 vs. 37 patients, respectively).
In the previous think-aloud study, patients
suggested clarifying the time frame for rating
In the ESAS-r, the time frame
was specified as ‘‘now’’ because this was how
the tool was originally designed. However,
this is not explicitly written on the original
ESAS, although it is described in the guide-
lines for administration.
This may have
contributed to some of the variability in re-
sponses between the two forms. Some authors
have reported using a time frame of 24 hours
for the ESAS.
In a study of 1147 cancer out-
patients, Shi et al.
have reported that ratings
for worst pain in the past week correlate more
closely with pain interference than ratings for
current pain.
If the ESAS is completed in the presence of
a health care professional, then the range of
symptom intensity over time can be explored,
in which clinicians can compare the patient’s
current symptom ratings with past symptom
experiences (e.g., over the past week, worst,
best, average). However, if the tool is being
completed without an opportunity to elabo-
rate on the scores, then it would be important
to choose the most relevant time frame. This
may differ based on the palliative care setting,
patient needs, fluctuation of symptoms over
time, and frequency of symptom assessments.
There are several limitations of this study.
For patients who were naı¨ve to the ESAS, the
ESAS-r was consistently presented after the
ESAS, which could have affected perception

464 Vol. 41 No. 2 February 2011Watanabe et al.
of the revised instrument. However, this was
a relatively small proportion of the entire sam-
ple (n¼ 58, 36%). In addition, there were no
Fig. 5. Distribution of wel
significant differences, in terms of preference
for the ESAS-r vs. the ESAS, between partici-
pants who had previously completed the
ESAS and those who were not familiar with
it. The presence of a research nurse or assis-
tant may have influenced patient responses,
as opposed to having patients complete the
ESAS without someone present. It is also un-
certain if the German versions of the ESAS
and ESAS-r were interpreted in the same way
as the English versions; gathering validity evi-
dence for the German versions of the ESAS
was beyond the scope of this study, a focus
for future research. However, results for the
subgroup of patients in St. Gallen did not dif-
fer significantly from results for the entire
study group. A fourth limitation relates to reli-
ance on clinical impression of recovery of cog-
nitive function in patients who had an
abnormal MMSE before study entry. However,
this is in keeping with clinical practice. Also,
patients with mild cognitive impairments are
able to complete the ESAS, sometimes with
the assistance of a health care provider. A fur-
ther limitation is that most participants were
inpatients, reflecting referral patterns of the
participating programs, and therefore the re-
sults may not be generalizable to outpatients.
scores (ESAS vs. ESAS-r).
Future research priorities include gathering
further validity evidence for the ESAS-r, focus-
ing on optimal definitions and time frames
for symptom assessments. In particular, further
research is needed to better understand the var-
iability in responses for the symptoms drowsi-
ness, appetite, and well-being. It is possible
that this variability represents an improvement
in ratings with the ESAS-r in comparison with
theESAS. This needs to be explored inmore de-
tail. Based on participants’ feedback, the word-
ing for the definition for appetite continues to
be potentially confusing and requires further
validation. Most patients in this study were fa-
miliar with theESAS, which provided important
perspectives of peoplewhowere knowledgeable
about this tool. Additional testing with patients
who arenot familiar with theESAS, alone and in
the presence of a health care provider, is war-
ranted. As an extension of the qualitative
think-aloud ESAS study,
further qualitative re-
search focusing on patients’ and health care
professionals’ interpretations of item scales,
particularly problematic items, would be useful.
Finally, gathering further validity evidence

1. Bruera E, Kuehn N, Miller MJ, Selmser P,
ton Symptom Assessment Scale (ESAS) as an audit
tool. J Palliat Care 1999;15:14e19.
Vol. 41 No. 2 February 2011 465Edmonton Symptom Assessment System Comparative Study
7. Porzio G, Ricevuto E, Aielli F, et al. The Sup-
portive Care Task Force at the University of L’Aq-
uila: 2-years experience. Support Care Cancer
Macmillan K. The Edmonton Symptom Assessment
System (ESAS): a simple method for the assessment
of palliative care patients. J Palliat Care 1991;7:6e9.
2. Regional Palliative Care Program in Edmonton
Alberta. Guidelines for using the Edmonton Symp-
tom Assessment System. Available from http://
www.palliative.org. Accessed September 1, 2010.
3. Cummings G, Biondo P, Hagen N, Fainsinger R,
Stiles C. Bibilometric review: Edmonton Symptom
Assessment Scale (ESAS). Palliat Med 2008;22:
4. Heedman PA, Strang P. Symptom assessment in
advanced palliative home care for cancer patients
using the ESAS: clinical aspects. Anticancer Res
5. Chow E, Davis L, Holden L, Tsao M, Danjoux C.
Prospective assessment of patient-rated symptoms
following whole brain radiotherapy for brain metas-
tases. J Pain Symptom Manage 2005;30:18e23.
6. Dudgeon DJ, Harlos M, Clinch JJ. The Edmon-
for using the ESAS in non-English-speaking
countries and languages other than English is
Disclosures and Acknowledgments
This work was supported by a Canadian
Institutes for Health Research New Emerging
Team grant. The authors declare no conflicts
of interest.
The authors wish to thank the patients, their
families, and the following collaborators:
Christina Dunn, RN, Carla Stiles, RN, Pablo
Amigo, MD, Sarah Burton-Macleod, MD, In-
grid deKock, MD, Joan Faily, MD, Robin Fain-
singer, MD, Mehrnoush Mirhosseini, MD,
Doreen Oneschuk, MD, Yoko Tarumi, MD,
Vincent Thai, MD, Gary Wolch, MD, David
Blum, MD, Hanspeter Ha¨ne, MD-student,
Susanne Linder, RN, Rolf Oberholzer, MD,
Susanne Ott-Jaworski, MD-student, and the
palliative care program staff in Edmonton,
Calgary, Toronto and St. Gallen.
8. Nekolaichuk C, Watanabe S, Beaumont C. The
Edmonton Symptom Assessment System: a 15-year
retrospective review of validation studies (1991-
2006). Palliat Med 2008;22:111e122.
9. Richardson LA, Jones GW. A review of the reli-
ability and validity of the Edmonton Symptom As-
sessment System. Curr Oncol 2009;16:53e64.
10. Watanabe S, McKinnon S, Macmillan K,
Hanson J. Palliative care nurses’ perception of the
Edmonton Symptom Assessment Scale: a pilot sur-
vey. Int J Palliat Nurs 2006;12:111e114.
11. Garyali A, Palmer JL, Yennurajalingam S, et al.
Errors in symptom intensity self-assessment by pa-
tients receiving outpatient palliative care. J Palliat
Med 2006;9:1059e1065.
12. Watanabe S, Nekolaichuk C, Beaumont C,
Mawani A. The Edmonton Symptom Assessment
System: what do patients think? Support Care Can-
cer 2009;17:675e683.
13. Nekolaichuk CL, Maguire TO, Suarez-
Almazor M, Rogers WT, Bruera E. Assessing the
reliability of patient, nurse, and family caregiver
symptom ratings in hospitalized advanced cancer
patients. J Clin Oncol 1999;17:3621e3630.
14. Bruera E, MacDonald S. Audit methods: the
Edmonton Symptom Assessment System. In:
Higginson I, ed. Clinical audit in palliative care. Ox-
ford, UK: Radcliffe Medical Press, 1993: 61e77.
15. Nekolaichuk CL, Bruera E, Spachynski K, et al.
A comparison of patient and proxy symptom assess-
ments in advanced cancer patients. Palliat Med
16. Pautex S, Berger A, Chatelain C, Herrmann F,
Zulian GB. Symptom assessment in elderly cancer
patients receiving palliative care. Crit Rev Oncol
Hematol 2003;47:281e286.
17. Moro C, Brunelli C, Miccinesi G, et al. Edmon-
ton Symptom Assessment Scale: Italian validation in
two palliative care settings. Support Care Cancer
18. Chang VT, Hwang SS, Feuerman M. Validation
of the Edmonton Symptom Assessment Scale. Can-
cer 2000;88:2164e2171.
19. Davison SN, Jhangri GS, Johnson JA. Cross-sec-
tional validity of a modified Edmonton Symptom
Assessment System in dialysis patients: a simple as-
sessment of symptom burden. Kidney Int 2006;69:
20. Stromgren AS, Groenvold M, Pedersen L,
Olsen AK, Sjogren P. Symptomatology of cancer pa-
tients in palliative care: content validation of self-
assessment questionnaires against medical records.
Eur J Cancer 2002;38:788e794.
21. Philip J, Smith WB, Craft P, Lickiss N. Concur-
rent validity of the modified Edmonton Symptom
Assessment System with the Rotterdam Symptom
Checklist and the Brief Pain Inventory. Support
Care Cancer 1998;6:539e541.

22. Davison SN, Jhangri GS, Johnson JA. Longitudi-
nal validation of a modified Edmonton Symptom
Assessment System (ESAS) in haemodialysis pa-
tients. Nephrol Dial Transplant 2006;21:3189e3195.
23. Vignaroli E, Pace EA, Willey J, et al. The Ed-
monton Symptom Assessment System as a screening
tool for depression and anxiety. J Palliat Med 2006;
24. Anderson F, Downing GM, Hill J, Casorso L,
Lerch N. Palliative Performance Scale (PPS):
a new tool. J Palliat Care 1996;12:5e11.
25. Folstein MF, Folstein S, McHugh PR. ‘‘Mini-
mental state’’: a practical method for grading the
cognitive state of patients for the clinician. J Psy-
chiatr Res 1975;12:189e198.
26. Crum RM, Anthony JC, Bassett SS, Folstein MF.
Population-based norms for the Mini-Mental State
Examination by age and educational level. JAMA
27. Fainsinger RL, Nekolaichuk CL. A ‘‘TNM’’ clas-
sification system for cancer pain: the Edmonton
Classification System for Cancer Pain (ECS-CP).
Support Care Cancer 2008;16:547e555.
28. Gall MD, Gall JP, Borg WR. Educational re-
search: An introduction, 7th ed. Boston, MA: Pear-
son Education, 2003.
29. Cancer Care Ontario. Cancer system quality in-
dex: Symptom assessment. Available from http://
05&pageId¼68032. Accessed September 1, 2010.
30. Noguera A, Centeno C, Carvajal A, et al. ‘‘Are
you discouraged? Are you anxious, nervous or un-
easy?’’: in Spanish some words could be better.
J Palliat Med 2009;12:707e712.
31. Shi Q, Wang XS, Mendoza TR, Pandya KJ,
Cleeland CS. Assessing persistent cancer pain:
a comparison of current pain ratings and pain re-
called from the past week. J Pain Symptom Manage
466 Vol. 41 No. 2 February 2011Watanabe et al.

Appendix A
Edmonton Symptom Assessment System (current version)
Vol. 41 No. 2 February 2011 467Edmonton Symptom Assessment System Comparative Study

Appendix B
468 Vol. 41 No. 2 February 2011Watanabe et al.
Edmonton Symptom Assessment System (revised version) (ESAS-r)