bs_bs_banner
Australasian Journal of Dermatology (2015) 56, 100–102
doi: 10.1111/ajd.12280
ORIGINAL RESEARCH
Inter-observer reliability of the PASI in a clinical setting Samantha Cabrera,1* Niranthari Chinniah,2* Nannette Lock,2 Geoffrey D. Cains1,2 and Jane Woods2 1
Faculty of Medicine, University of New South Wales, and 2Department of Dermatology, Liverpool Hospital, Sydney, New South Wales, Australia
INTRODUCTION ABSTRACT Background: With the evolving emphasis on evidence-based practice, the use of reliable clinical scales forms an important foundation for clinical assessment. The psoriasis area and severity index (PASI) is the most widely used tool for the measurement of psoriasis severity; however, there has been some debate over the potential reproducibility of PASI scoring. Objectives: To determine the inter-observer reliability of the PASI at a large tertiary hospital with a psoriasis treatment centre. Methods: In total, 34 patients who were due for their 3-monthly follow up at a psoriasis treatment centre were independently evaluated by five clinical staff (observers) from the Department of Dermatology. Each observer independently determined the PASI score of each patient and the inter-observer reliability coefficient was determined by employing intra-class correlation coefficients (ICC). Results: There was a significant degree of concordance among the PASI scoring of observers (ICC = 0.804; CI 95%: 0.706–0.883). Conclusions: Our cross-sectional study suggests that the PASI provides a reproducible method of assessing psoriasis severity among patients seen in a busy dermatology clinic at a large tertiary hospital. Key words: ICC, inter-observer reliability, interrater reliability, PASI, PASI reliability, psoriasis, psoriasis area and severity index, psoriasis assessment, psoriasis severity, reliability.
Correspondence: Dr Niranthari Chinniah, Department of Dermatology, Liverpool Hospital, 45–47 Goulburn Street, Liverpool, NSW 2170, Australia. Email: [email protected] Samantha Cabrera, BMedSc(Hons). Niranthari Chinniah, MBBS(Hons). Nannette Lock,MNurs. Geoffrey D. Cains, FACD. Jane Woods, FACD. *Co-first authors. Conflict of interest: none Submitted 19 August 2014; accepted 20 October 2014. © 2014 The Australasian College of Dermatologists
The psoriasis area and severity index (PASI) is the most widely used tool for the measurement of psoriasis severity in clinical trials.1–3 Developed in 1978 by Fredriksson and Pettersson to evaluate the treatment efficacy of a new retinoid drug in a single clinical trial, the PASI permits the independent assessment of the various characteristics of psoriatic plaque appearance.4 These include the degree of erythema, thickness and scaling, weighted to the individual assessment of the degree of body surface area (BSA) affected, in the various regions of the body (head, upper limbs, trunk and lower limbs). These scores are combined to ultimately generate an overall PASI score that can be used as a basis for the therapeutic decision of the clinician. In clinical trials, changes in the PASI score of a patient are crucial to determine whether the chosen drug therapy is effective. In moderate and severe psoriasis, clinicians typically consider a 75% improvement of the patient’s PASI score (i.e. a 75% reduction of the baseline PASI), termed PASI 75, as the primary end-point to determine clinically meaningful success with the chosen drug therapy.1 However, PASI 90 and PASI 50 have also been regarded as clinically significant.5,6 Although the PASI is the gold standard for the assessment of moderate and severe psoriasis, it has its limitations. Overall, the PASI score does not discriminate whether the score is primarily affected by the severity of plaque characteristics, the degree of involvement, or both. Erythema, thickness and scaling are scored with equal weighting in each of the four body regions. Thus, a reduction in scaling with a concomitant increase in skin erythema could be recorded with the same PASI score. Since it was designed to assess chronic plaque-type psoriasis, the PASI score of other forms of psoriasis may be misleading and can undermine the determination of disease severity, such as in a case of the more serious and difficult to treat erythrodermic psoriasis.7 Like most psoriasis assessment tools, the PASI is Abbreviations: BSA ICC LS-PGA PASI PGA QOL
body surface area intra-class correlation coefficient Lattice system physician’s global assessment psoriasis area and severity index physician’s global assessment quality of life
Inter-observer reliability of the PASI
101
Figure 1 Scatter plot displaying the psoriasis area and severity index (PASI) scoring of 34 patients by five observers.
a qualitative assessment and therefore it is difficult to exclude a subjective bias in its application. When minor changes are noticed in a small area, the PASI may not be sensitive in its discrimination, primarily due to the weighting factor of affected BSA.1 Moreover, the PASI does not account for effects of the disease on quality of life (QOL). The reliability of the PASI is still much debated due to limited information on the reproducibility of PASI scores in clinical practice. A recent study has demonstrated a significant decrease in inter-observer agreement in patients with PASI < 20 which, in turn, resulted in unacceptable therapeutic decision agreement.8 However, when comparing the PASI to two other psoriasis assessment tools – the Lattice system physician’s global assessment (LS-PGA), and the physician’s global assessment (PGA) the PASI was rated with the highest interrater reliability.9 This study aims to determine the inter-observer reliability of the PASI assessment tool at a large tertiary hospital with a psoriasis treatment centre.
Statistical analysis The PASI scores assessment was subsequently tabulated. The inter-observer reliability coefficient was determined by employing intra-class correlation coefficients (ICC).
RESULTS Patients’ characteristics and psoriasis type There was relative sex equality in the cohort of 34 patients, namely 18 (53%) men and 16 (47%) women. A wide age range was noted (22–68 years of age, median, 45. The patients were from various cultural backgrounds with six patients noted to be of mixed ethnicity. Of the patients, 30 (88%) had chronic plaque-type psoriasis, three (9%) had mixed guttate/plaque-type psoriasis and one (3%) had pustular-type psoriasis. In total, 22 (65%) were considered to have generalised psoriasis while 12 (35%) had localised psoriasis.
Intra-class correlation METHODS A total of 34 patients were recruited from the Liverpool Hospital Dermatology Clinic to participate in this study after their informed consent was obtained. The observers were five clinical staff and consisted of a nurse, a resident medical officer, a dermatology consultant and two accredited dermatology registrars. Each observer independently evaluated the PASI score of each patient by utilising a standardised PASI form; with each observer using one form per patient. The study was conducted over 2 separate days with 17 patients assessed each day. To control variability that might influence the interobserver assessment of psoriasis severity, each patient remained in one room, while the observers rotated to perform their assessment. This helped ensure that all observers were exposed to the same external environment conditions (lighting, temperature and room layout). All patients wore identical examination gowns. Each PASI form was immediately collected, de-identified and stored.
An ICC of 0.804 (CI 95%: 0.706–0.883) showed there was a significant degree of concordance among the observers. Figure 1 shows the distribution of PASI scores among observers for each patient, while Figure 2 shows the distance of the PASI scoring from the mean PASI score of each patient. These figures illustrate the PASI inter-observer reliability. In patients with PASI scores greater than 10, there was an observable reduction in concordance.
DISCUSSION In this study, we assessed the inter-observer reliability of the PASI in 34 patients, by individually assessing them with five qualified observers from a tertiary dermatology clinic. Significantly, a high degree of concordance (ICC = 0.804) of the PASI scoring was seen among the observers, despite their varied experience of PASI assessment. It was also noted that there was an observable reduction in concordance in PASI scores greater than 10. Since the weighting © 2014 The Australasian College of Dermatologists
102
S Cabrera et al.
Figure 2 Scatter plot displaying the distance of the psoriasis area and severity index (PASI) scores from the mean PASI score of each of 34 patients.
factor of the overall PASI score is the degree of affected BSA, we speculate that with larger affected BSA there is greater variation in its evaluation. All our observers had received training in the use of the PASI at the dermatology clinic, which could be one of the contributing factors to a high level of inter-observer correlation. To prevent reductions in inter-observer correlation, such as in more widespread psoriasis affecting greater BSA, repeat PASI training may be necessary for clinical staff to promote consistency in the evaluation across the various aspects of the PASI. Our study subsequently highlights the reproducibility of PASI scoring in a clinical setting. The inter-observer reliability of the PASI was also demonstrated in a similar study by Berth-Jones and colleagues (2006); in which the PASI was found to have an interobserver ICC of 0.90 when comparing the PASI scoring of 14 dermatologists of 16 patients with chronic plaque-type psoriasis. The study by Berth-Jones and colleagues also found the PASI produced both the highest inter-observer and intra-observer reliability compared with two other popular psoriasis assessment tools: the PGA and the LS-PGA. In a separate study exploring the most important objective and subjective measurements of severity in psoriasis in various psoriasis assessment tools, the PASI was determined to be the most adequate assessment tool available to evaluate severity in plaque-type psoriasis.10 Conversely, in a study by Langley and Ellis (2004) comparing the PASI scoring of 17 physicians (53% experienced and 47% inexperienced) in 35 patients, the PASI was demonstrated to have a higher inter-observer variation, compared to the PGA and LS-PGA.11 The statistical methods used to determine inter-observer agreement, however, has been shown to lead to varying results, with ICC determined to be the most appropriate measurement of inter-observer agreement of PASI scoring when compared to two other common types of statistical analyses.8 Although the PASI is the best validated objective measurement of psoriasis severity, it does not account for the diseases’ effects on QOL, associated symptoms (e.g. arthritis and pruritus), or degree of disability and therefore is limited in its ability to represent the true burden of the disease. The PASI must therefore be used in conjunction with QOL questionnaires in order to holistically reflect psoriasis severity. © 2014 The Australasian College of Dermatologists
CONCLUSION Our cross-sectional study suggests that the PASI provides a reproducible method of assessing psoriasis severity and progress in patients seen in a busy dermatology clinic at a large tertiary hospital.
REFERENCES 1. 2.
3.
4. 5.
6.
7.
8.
9.
10.
11.
Feldman S, Krueger G. Psoriasis assessment tools in clinical trials. Ann. Rheum. Dis. 2005; 64 (Suppl. 2): ii65–ii68. Louden BA, Pearce DJ, Lang W et al. A simplified psoriasis area severity index (SPASI) for rating psoriasis severity in clinic patients. Dermatol. Online J. 2004; 10: 7. Naldi L, Svensson A, Diepgen T et al. Randomized clinical trials for psoriasis 1977–2000: the EDEN survey. J. Invest. Dermatol. 2003; 120: 738–41. Fredriksson T, Pettersson U. Severe psoriasis – oral therapy with a new retinoid. Dermatologica 1978; 157: 238–44. Al-Suwaidan SN, Feldman SR. Clearance is not a realistic expectation of psoriasis treatment. J. Am. Acad. Dermatol. 2000; 42: 796–802. Carlin CS, Feldman SR, Krueger JG et al. A 50% reduction in the psoriasis area and severity index (PASI 50) is a clinically significant endpoint in the assessment of psoriasis. J. Am. Acad. Dermatol. 2004; 50: 859–66. Ashcroft D, Wan Po A, Williams H et al. Clinical measures of disease severity and outcome in psoriasis: a critical appraisal of their quality. Br. J. Dermatol. 1999; 141: 185–91. Gourraud PA, Le Gall C, Puzenat E et al. Why statistics matter: limited inter-rater agreement prevents using the psoriasis area and severity index as a unique determinant of therapeutic decision in psoriasis. J. Invest. Dermatol. 2012; 132: 2171–5. Berth-Jones J, Grotzinger K, Rainville C et al. A study examining inter- and intrarater reliability of three scales for measuring severity of psoriasis: psoriasis area and severity index, physician’s global assessment and lattice system physician’s global assessment. Br. J. Dermatol. 2006; 155: 707–13. Schmitt J, Wozel G. The psoriasis area and severity index is the adequate criterion to define severity in chronic plaque-type psoriasis. Dermatology 2005; 210: 194–9. Langley RG, Ellis CN. Evaluating psoriasis with psoriasis area and severity index, psoriasis global assessment, and lattice system physician’s global assessment. J. Am. Acad. Dermatol. 2004; 51: 563–9.
This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material.
Australasian Journal of Dermatology (2015) 56, 100–102
doi: 10.1111/ajd.12280
ORIGINAL RESEARCH
Inter-observer reliability of the PASI in a clinical setting Samantha Cabrera,1* Niranthari Chinniah,2* Nannette Lock,2 Geoffrey D. Cains1,2 and Jane Woods2 1
Faculty of Medicine, University of New South Wales, and 2Department of Dermatology, Liverpool Hospital, Sydney, New South Wales, Australia
INTRODUCTION ABSTRACT Background: With the evolving emphasis on evidence-based practice, the use of reliable clinical scales forms an important foundation for clinical assessment. The psoriasis area and severity index (PASI) is the most widely used tool for the measurement of psoriasis severity; however, there has been some debate over the potential reproducibility of PASI scoring. Objectives: To determine the inter-observer reliability of the PASI at a large tertiary hospital with a psoriasis treatment centre. Methods: In total, 34 patients who were due for their 3-monthly follow up at a psoriasis treatment centre were independently evaluated by five clinical staff (observers) from the Department of Dermatology. Each observer independently determined the PASI score of each patient and the inter-observer reliability coefficient was determined by employing intra-class correlation coefficients (ICC). Results: There was a significant degree of concordance among the PASI scoring of observers (ICC = 0.804; CI 95%: 0.706–0.883). Conclusions: Our cross-sectional study suggests that the PASI provides a reproducible method of assessing psoriasis severity among patients seen in a busy dermatology clinic at a large tertiary hospital. Key words: ICC, inter-observer reliability, interrater reliability, PASI, PASI reliability, psoriasis, psoriasis area and severity index, psoriasis assessment, psoriasis severity, reliability.
Correspondence: Dr Niranthari Chinniah, Department of Dermatology, Liverpool Hospital, 45–47 Goulburn Street, Liverpool, NSW 2170, Australia. Email: [email protected] Samantha Cabrera, BMedSc(Hons). Niranthari Chinniah, MBBS(Hons). Nannette Lock,MNurs. Geoffrey D. Cains, FACD. Jane Woods, FACD. *Co-first authors. Conflict of interest: none Submitted 19 August 2014; accepted 20 October 2014. © 2014 The Australasian College of Dermatologists
The psoriasis area and severity index (PASI) is the most widely used tool for the measurement of psoriasis severity in clinical trials.1–3 Developed in 1978 by Fredriksson and Pettersson to evaluate the treatment efficacy of a new retinoid drug in a single clinical trial, the PASI permits the independent assessment of the various characteristics of psoriatic plaque appearance.4 These include the degree of erythema, thickness and scaling, weighted to the individual assessment of the degree of body surface area (BSA) affected, in the various regions of the body (head, upper limbs, trunk and lower limbs). These scores are combined to ultimately generate an overall PASI score that can be used as a basis for the therapeutic decision of the clinician. In clinical trials, changes in the PASI score of a patient are crucial to determine whether the chosen drug therapy is effective. In moderate and severe psoriasis, clinicians typically consider a 75% improvement of the patient’s PASI score (i.e. a 75% reduction of the baseline PASI), termed PASI 75, as the primary end-point to determine clinically meaningful success with the chosen drug therapy.1 However, PASI 90 and PASI 50 have also been regarded as clinically significant.5,6 Although the PASI is the gold standard for the assessment of moderate and severe psoriasis, it has its limitations. Overall, the PASI score does not discriminate whether the score is primarily affected by the severity of plaque characteristics, the degree of involvement, or both. Erythema, thickness and scaling are scored with equal weighting in each of the four body regions. Thus, a reduction in scaling with a concomitant increase in skin erythema could be recorded with the same PASI score. Since it was designed to assess chronic plaque-type psoriasis, the PASI score of other forms of psoriasis may be misleading and can undermine the determination of disease severity, such as in a case of the more serious and difficult to treat erythrodermic psoriasis.7 Like most psoriasis assessment tools, the PASI is Abbreviations: BSA ICC LS-PGA PASI PGA QOL
body surface area intra-class correlation coefficient Lattice system physician’s global assessment psoriasis area and severity index physician’s global assessment quality of life
Inter-observer reliability of the PASI
101
Figure 1 Scatter plot displaying the psoriasis area and severity index (PASI) scoring of 34 patients by five observers.
a qualitative assessment and therefore it is difficult to exclude a subjective bias in its application. When minor changes are noticed in a small area, the PASI may not be sensitive in its discrimination, primarily due to the weighting factor of affected BSA.1 Moreover, the PASI does not account for effects of the disease on quality of life (QOL). The reliability of the PASI is still much debated due to limited information on the reproducibility of PASI scores in clinical practice. A recent study has demonstrated a significant decrease in inter-observer agreement in patients with PASI < 20 which, in turn, resulted in unacceptable therapeutic decision agreement.8 However, when comparing the PASI to two other psoriasis assessment tools – the Lattice system physician’s global assessment (LS-PGA), and the physician’s global assessment (PGA) the PASI was rated with the highest interrater reliability.9 This study aims to determine the inter-observer reliability of the PASI assessment tool at a large tertiary hospital with a psoriasis treatment centre.
Statistical analysis The PASI scores assessment was subsequently tabulated. The inter-observer reliability coefficient was determined by employing intra-class correlation coefficients (ICC).
RESULTS Patients’ characteristics and psoriasis type There was relative sex equality in the cohort of 34 patients, namely 18 (53%) men and 16 (47%) women. A wide age range was noted (22–68 years of age, median, 45. The patients were from various cultural backgrounds with six patients noted to be of mixed ethnicity. Of the patients, 30 (88%) had chronic plaque-type psoriasis, three (9%) had mixed guttate/plaque-type psoriasis and one (3%) had pustular-type psoriasis. In total, 22 (65%) were considered to have generalised psoriasis while 12 (35%) had localised psoriasis.
Intra-class correlation METHODS A total of 34 patients were recruited from the Liverpool Hospital Dermatology Clinic to participate in this study after their informed consent was obtained. The observers were five clinical staff and consisted of a nurse, a resident medical officer, a dermatology consultant and two accredited dermatology registrars. Each observer independently evaluated the PASI score of each patient by utilising a standardised PASI form; with each observer using one form per patient. The study was conducted over 2 separate days with 17 patients assessed each day. To control variability that might influence the interobserver assessment of psoriasis severity, each patient remained in one room, while the observers rotated to perform their assessment. This helped ensure that all observers were exposed to the same external environment conditions (lighting, temperature and room layout). All patients wore identical examination gowns. Each PASI form was immediately collected, de-identified and stored.
An ICC of 0.804 (CI 95%: 0.706–0.883) showed there was a significant degree of concordance among the observers. Figure 1 shows the distribution of PASI scores among observers for each patient, while Figure 2 shows the distance of the PASI scoring from the mean PASI score of each patient. These figures illustrate the PASI inter-observer reliability. In patients with PASI scores greater than 10, there was an observable reduction in concordance.
DISCUSSION In this study, we assessed the inter-observer reliability of the PASI in 34 patients, by individually assessing them with five qualified observers from a tertiary dermatology clinic. Significantly, a high degree of concordance (ICC = 0.804) of the PASI scoring was seen among the observers, despite their varied experience of PASI assessment. It was also noted that there was an observable reduction in concordance in PASI scores greater than 10. Since the weighting © 2014 The Australasian College of Dermatologists
102
S Cabrera et al.
Figure 2 Scatter plot displaying the distance of the psoriasis area and severity index (PASI) scores from the mean PASI score of each of 34 patients.
factor of the overall PASI score is the degree of affected BSA, we speculate that with larger affected BSA there is greater variation in its evaluation. All our observers had received training in the use of the PASI at the dermatology clinic, which could be one of the contributing factors to a high level of inter-observer correlation. To prevent reductions in inter-observer correlation, such as in more widespread psoriasis affecting greater BSA, repeat PASI training may be necessary for clinical staff to promote consistency in the evaluation across the various aspects of the PASI. Our study subsequently highlights the reproducibility of PASI scoring in a clinical setting. The inter-observer reliability of the PASI was also demonstrated in a similar study by Berth-Jones and colleagues (2006); in which the PASI was found to have an interobserver ICC of 0.90 when comparing the PASI scoring of 14 dermatologists of 16 patients with chronic plaque-type psoriasis. The study by Berth-Jones and colleagues also found the PASI produced both the highest inter-observer and intra-observer reliability compared with two other popular psoriasis assessment tools: the PGA and the LS-PGA. In a separate study exploring the most important objective and subjective measurements of severity in psoriasis in various psoriasis assessment tools, the PASI was determined to be the most adequate assessment tool available to evaluate severity in plaque-type psoriasis.10 Conversely, in a study by Langley and Ellis (2004) comparing the PASI scoring of 17 physicians (53% experienced and 47% inexperienced) in 35 patients, the PASI was demonstrated to have a higher inter-observer variation, compared to the PGA and LS-PGA.11 The statistical methods used to determine inter-observer agreement, however, has been shown to lead to varying results, with ICC determined to be the most appropriate measurement of inter-observer agreement of PASI scoring when compared to two other common types of statistical analyses.8 Although the PASI is the best validated objective measurement of psoriasis severity, it does not account for the diseases’ effects on QOL, associated symptoms (e.g. arthritis and pruritus), or degree of disability and therefore is limited in its ability to represent the true burden of the disease. The PASI must therefore be used in conjunction with QOL questionnaires in order to holistically reflect psoriasis severity. © 2014 The Australasian College of Dermatologists
CONCLUSION Our cross-sectional study suggests that the PASI provides a reproducible method of assessing psoriasis severity and progress in patients seen in a busy dermatology clinic at a large tertiary hospital.
REFERENCES 1. 2.
3.
4. 5.
6.
7.
8.
9.
10.
11.
Feldman S, Krueger G. Psoriasis assessment tools in clinical trials. Ann. Rheum. Dis. 2005; 64 (Suppl. 2): ii65–ii68. Louden BA, Pearce DJ, Lang W et al. A simplified psoriasis area severity index (SPASI) for rating psoriasis severity in clinic patients. Dermatol. Online J. 2004; 10: 7. Naldi L, Svensson A, Diepgen T et al. Randomized clinical trials for psoriasis 1977–2000: the EDEN survey. J. Invest. Dermatol. 2003; 120: 738–41. Fredriksson T, Pettersson U. Severe psoriasis – oral therapy with a new retinoid. Dermatologica 1978; 157: 238–44. Al-Suwaidan SN, Feldman SR. Clearance is not a realistic expectation of psoriasis treatment. J. Am. Acad. Dermatol. 2000; 42: 796–802. Carlin CS, Feldman SR, Krueger JG et al. A 50% reduction in the psoriasis area and severity index (PASI 50) is a clinically significant endpoint in the assessment of psoriasis. J. Am. Acad. Dermatol. 2004; 50: 859–66. Ashcroft D, Wan Po A, Williams H et al. Clinical measures of disease severity and outcome in psoriasis: a critical appraisal of their quality. Br. J. Dermatol. 1999; 141: 185–91. Gourraud PA, Le Gall C, Puzenat E et al. Why statistics matter: limited inter-rater agreement prevents using the psoriasis area and severity index as a unique determinant of therapeutic decision in psoriasis. J. Invest. Dermatol. 2012; 132: 2171–5. Berth-Jones J, Grotzinger K, Rainville C et al. A study examining inter- and intrarater reliability of three scales for measuring severity of psoriasis: psoriasis area and severity index, physician’s global assessment and lattice system physician’s global assessment. Br. J. Dermatol. 2006; 155: 707–13. Schmitt J, Wozel G. The psoriasis area and severity index is the adequate criterion to define severity in chronic plaque-type psoriasis. Dermatology 2005; 210: 194–9. Langley RG, Ellis CN. Evaluating psoriasis with psoriasis area and severity index, psoriasis global assessment, and lattice system physician’s global assessment. J. Am. Acad. Dermatol. 2004; 51: 563–9.
This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material.
