Anaesthesia, 1991, Volume 46, pages 207-2 12 APPARATUS

Potential errors in pulse oximetry 11. Effects of changes in saturation and signal quality”

R. K. WEBB, A. C . RALSTON

AND

W. B. RUNCIMAN

Summary

The published studies of pulse oximeter performance under conditions of normal, high and low saturation, exercise, poor signal quality and cardiac arrhythmia are reviewed. Most pulse oximeters have an absolute mean error of less than 2% at normal saturation andperfusion; two-thirds have a standard deviation ( S D ) of less than 2%, and the remainder an SD of less than 3%. Some pulse oxirneters tend to read lOOO/d with fractional saturations of 97-98%. Pulse oximeters may be suitable hyperoxic alarmsfor neonates if the alarm limit chosen is directly validated for each device. Pulse oximeters are poorly calibrated at low saturations and are generally less accurate and less precise than at normal saturations; nearly 30% of 244 values reviewed were in error by more than 5% at saturations of less than 80%. Ear, nose and forehead probes respond more rapidly to rapid desaturation than finger probes, but are generally less accurate and less precise. Ear oximetry may be inaccurate during exercise. Low signal quality can result in failure to present a saturation reading, but data given with low signal quality warning messages are generally no less accurate than those without. Cardiac arrhythmias do not decrease accuracy of pulse oximeters so long as saturation readings are steady. Key words

Equipment; pulse oximeter. Measurement .

The use of pulse oximeters is now widespread, but they cannot, for practical purposes, be calibrated by the user, so it is important that performance be evaluated ‘in the field’ under both normal (good perfusion, saturation within a normal range and no interfering substances or extraneous factors) and under adverse conditions. This paper reviews the published studies on the accuracy of pulse oximeters for patients and volunteers under conditions of normal, low and high saturation, exercise, poor signal quality and cardiac arrhythmia. The notation used is that described earlier:’ functional saturation (Sao,); fractional saturation (%HbO,); saturation value indicated by a pulse oximeter (Spo,). The large volume of clinical trial data that is published allows manufacturers to re-assess and improve their software frequently. Many of the pulse oximeters used to collect the data referred to in this paper have had software revisions since the original studies were published. Thus the statements of accuracy of any particular unit should not be taken as a definitive measure of its current capability;

rather the data presented should be used as a guide to the potential magnitude of differences in the performance of different pulse oximeters under various adverse conditions. The majority of the data were available only as scattergrams or calculated linear regression lines and correlation coefficients. Much of the information presented in this paper was derived from these scattergrams and may not be as accurate as the authors’ original data. Errors are for this reason grouped into 5% intervals and no attempt is made to determine accuracy or precision from these data. Normal saturation

‘Normal saturation’ is defined for the purpose of this review as a fractional saturation of 90 to 97.5%; this corresponds to an arterial oxygen partial pressure of 13.3 to 13.7 kPa, if no other haemoglobin species are present, apart from oxy- and reduced haemoglobin. This may be considered by some to be a rather generous range of ‘normal values’ but is necessary in part to take into account

R.K. Webb, MB, BS, FFARACS, Senior Staff Specialist, A.C. Ralston, BSc,BAppSc, Medical Physicist, W.B. Runciman, Bsc(Med), MB, BCh, FFARACS, PhD, Professor and Head of Department of Anaesthesia and Intensive Care, Royal Adelaide Hospital, North Terrace, Adelaide, South Australia, Australia 5000. Correspondence should be addressed to Professor W.B. Runciman please. * Combined study with Australian Patient Safety Foundation, GPO Box 400, Adelaide, South Australia, Australia 5001. Accepted 6 August 1990.

0003-2409/91/030207

+06 $03.00/0

@ 1991 The Association of Anaesthetists of Gt Britain and Ireland

207

R.K. Webb, A.C. Ralston and W.B. Runciman the usual levels of carboxy- and methaemoglobin present in a normal population and in part to allow the inclusion of data from other studies which use saturations of greater than 90% as one of their data groups. Morris et al.’ evaluated 15 pulse oximeters on normal healthy volunteers. The absolute mean errors ranged from 0 to 2.3%; the standard deviation (precision) was less than 2% in 10 units, between 2 and 3% in four units and greater than 3% in one unit. Nickerson et al.’ evaluated four pulse oximeters and found that when the saturation was greater than 90% the absolute mean error ranged from 1 to 2.2%; the standard deviation was less than 2% in three units, and between 2 and 3% in the fourth unit. Choe et aL4 evaluated six pulse oximeters in patients with a P a q of greater than 60 mmHg and found the absolute mean error ranged from 0 to 2.7%. The standard deviation was less than 2% for five of the six units. Re-analysis of data from our previously reported study of pulse oximeter accuracy in intensive care patients’ was performed when results were limited to those patients with good perfusion and saturations in the 90 to 97.5% range. The majority of the pulse oximeters had acceptable performance in this range; 10 of the 13 units had an absolute mean error of less than 1.0%; the standard deviation was less than 2% in eight, and between 2 and 3% in the remaining five. High saturation

‘High saturation’ is defined for the purpose of this review as a functional saturation of greater than 97%. Pulse oximeters are normally limited by their software so as to not give a saturation reading of greater than 100%; this limits the potential for positive errors and make bias and precision calculations difficult to interpret in this high range. Differences between pulse oximeters are also minimised if the patient population upon which the pulse oximeters are evaluated has a consistently high saturation. We re-analysed our data in order to determine the tendency of pulse oximeters to read 100% when the saturation is high but less than 100%. All the pulse oximeter finger probe data points with a cooximeter fractional saturation of between 97 and 98% were examined and the number of times that a pulse oximeter simultaneously displayed a reading of 100% was determined. This varied from 13 out of 15 (87%) for one device down to 0 out of 17 (OYO)for another, with three of the 20 pulse oximeters reading 100% more than 80% of the time and a further five doing so more than 50% of the time. Five of the pulse oximeters did not read 100% at all when cooximeter saturation was between 97 and 98%. When a similar analysis was performed on the Emergency Care Research Institute (ECRI) data it was found that the same brands of pulse oximeter tended to read 100% in this context (Table 1). The oxygen dissociation curve flattens out at high saturation levels ( > 900/) so that a large increase in Pao, results in a very small increase in saturation. This is of no consequence for most adult patients, but is very important for neonates at risk of retinopathy caused by hyperoxaemia. One clinical trial used the Ohmeda 3700 and concluded that it was highly accurate and suitable as a detector for hyperoxaemia when the alarm limit was set to 95%.’ Another study compared the Ohmeda 3700 and the Nellcor

Table 1. Number of pulse oximeter readings of 100% when cooximeter reading was 97-98%.

Oximeter

Study l 6

Criticare CSI 503 Engstrom EOS Spectramed Pulsat Criticare CSI 504 Biochem Microspan 3040 Radiometer Oximeter Simed S-100 Invivo 4500 Datex Satlite Datascope Accusat Physio-Control 1600 Nonin 8604D Sensormedics Oxyshutle Novametrix 505 Pulsemate Colin BX-5 Minolta Pulsox 7 Ohmeda Biox 3700 Ohmeda Biox 3740 Nellcor N-200 Kontron 7840

0/17 (0%) 0/15 (0%) 0/15 (0%) 0/14 (0%) 0/10 (0%) 1/17 (6%) 1/15 (7%) 2/15 (13%) 3/22 (14%) 3/14 (21%) 4/16 (25%) 4/16 (25%) 6/16 (38%) 11/22 (50%) 10/16 (63%) 11/17 (65%) 11/15 (73%) 13/16 (81%) 15/18 (83%) 13/15 (87%)

Study 25 -

0/17 (0%) -

1/11 (9%) 1/17 (6%) 2/9 (22%) 119 (11%) 4/9 (44%) 2/17 (12%) 319 (33%) 2/7 (29%) 3/17 (18%) -

4/9 (44%) 5/16 (31%) 3/17 (18%) -

N- 100 on infants with frequent hyperoxaemic incidents, as determined by transcutaneous oxygen saturation measurements.’ It showed that with the alarm limit set to 95% the Nellcor identified all of the 26 hyperoxaemic incidents and gave 25 false alarms; whereas, with the same alarm limit, the Ohmeda identified 13 of 35 hyperoxaemic incidents and gave 29 false alarms. The suitability of the alarm setting for each device should be validated directly.

Low saturation ‘Low saturation’ is defined for the purposes of this review as a fractional saturation of less than 80%. Pulse oximeters have a high potential for error at low saturations, since ethically manufacturers cannot induce severe hypoxia repeatedly in volunteers for calibration purposes. Patient data

The accuracy of 13 pulse oximeters at low saturations was determined in intensive care patient^.^ All units were less accurate and nine out of 13 were less precise than when saturations were greater than 80% (Table 2);’ eight out of 13 units tended to underread by substantial amounts at low saturations. Taylor and Whitwam’ assessed the performance of five pulse oximeters on patients with saturations down to 63%. There were 16 data points with a saturation of 80% or less. On 10 occasions the pulse oximeters read lower than the cooximeter (- 1 to - 12%, mean = -6.5%), and on six they read higher (+ I to 18%, mean = 9.8%); five of the six overread data points were with one pulse oxirneter. Thus, overall, there was a tendency for pulse oxirneters to underread. West eta1.I’ recorded Spo, measured by the Ohmeda Biox I11 and the Nellcor N-100, and used the Hewlett Packard multiwavelength transmittance oximeter as a reference, for 149 rapid desaturations in patients who had intermittent sleep apnoea. They found that the pulse oximeters tended to underestimate the minimum reached.

+

Potential errors in pulse oximetry

209

Table 2. Accuracy of 13 pulse oximeters using finger probes on patients in the Intensive Care Unit.5 Values are expressed as mean (SD).

Brand

Saturation > 80% Bias (precision)

Saturation < 80% Bias (precision)

-0.3% (1.9) +O.O% (2.0) -0.3% (1.8) +0.8% (1.7) +1.4% (1.8) +0.7% (1.9) - 1.0% (2.5) -0.1% (2.8) +O.O% (1.9) -1.5% (1.8) -0.3% (1.8) +0.1% (2.2) +0.7% (1.6)

- 7 . 1 % (3.2) +1.4% (1.5) -0.6% (4.9) -5.5% (3.5) +8.8% (4.8) -8.1% (4.3) -5.3% (6.2) -5.5% (1.9) -6.0% (6.9) -6.7% (3.2) -0.4% (1.8) +1.8% (1.6) -3.4% (3.2)

Datascope Accusat Datex Satlite Invivo 4500 Nellcor N-200 Nonin 8604D Novametrix 505 Ohmeda 3700 Ohmeda 3740 Physio-Control 1600 Radiometer Oximeter Sensormedics Oxyshuttle Simed S- 100 Spectramed Pulsat

However, these results are presented in the form of regression analysis and are not readily comparable to other studies. Ridley" assessed the accuracy of the Nellcor NlOOE and the Ohmeda Biox 3700 in 25 paediatric surgical patients with cyanotic congenital heart disease. There were 49 data points with a saturation of 80% or less in the evaluation of the Nellcor N100E. The Nellcor overread on 20 occasions (on eight by 5% or less, on seven by 5-10%, and on five by 10-20% (maximum positive error IT%)), and underread on 27 occasions (on 21 by 5% or less, five by 5-10% and on one by 10-20% (maximum negative error - 16%)). Similar results were obtained for the 51 low saturation data points with the Ohmeda Biox 3700. The Ohmeda Biox 3700 overread (maximum error 20%) on 18 occasions, and underread (maximum error - 11 YO)on 30 occasions. The underread errors were again on average less than the overread errors (overread errors four by < 5 % , three by 5-10%, five by 1&15%, six by 15-20%; underread errors 20 by < -5Y0,nine by -5-- lo%, one by -lo%-- 1%0). Some of the patients had poor peripheral perfusion, which may have contributed to the pulse oximeter reading errors. The accuracy of low saturation readings by Nellcor pulse oximeters has been assessed by several groups,'2-16all of whom found good correlation (0.91 to 0.999) between the pulse oximeter readings and an in vitro oximeter. The combined results show the unit underread on 44 occasions and overread on 52 occasions. Almost 40% of these errors were greater than 5%. This confirms that good correlation may not be an indicator of high accuracy if a significant bias and poor precision also exists. The correlation coefficient for the whole range of saturations in the paper by Boxer14 was good at 0.95 but the 95% limits of agreement for those saturations below 80% was a poor -9.8 to +7.7?'0. Mihm and Halperin" monitored 18 patients in respiratory distress with a Nellcor pulse oximeter, and obtained a correlation coefficient of 0.99 between Spo, and %HbO, measured by an IL 282 cooximeter during three lifethreatening arterial desaturation events when the Sao, decreased to below 60%. The linear regression line for all 131 data points was y = 0.97xf 1.51 with a r value of 0.96. There were nine data points with a saturation of < 80%. In four cases the pulse oximeter underread and in five it overread. All errors were less than 5%.

Volunteer and animal data

Some studies have used transiently desaturated volunteers to assess the accuracy of pulse oximeters; these are more relevant to circumstances such as the sudden delivery of hypoxic gas mixtures than to the chronic hypoxaemia of some critically ill patients. Three groups of researchers used a stepwise reduction in inspired 0, concentrations in volunteers to assess accuracy of pulse oximeters at each hypoxic plateau. Nickerson3 used four pulse oximeters and found biases (SD) which ranged from -4.9 (2.9)% to +3.0 (2.0)0/, for saturations of 65-80%. Unfortunately they compared the pulse oximeters with the sum of the oxy-, carboxy- and methaemoglobin concentrations measured by the cooximeter and therefore these findings cannot be compared with the majority of other studies which use %HbO, or Sao, for comparison. Choe4 used six pulse oximeters, and when compared with %HbO, had biases (SD) ranging from -0.1 (1.6)% to 1.7 (l.8)Yo for Pao, 2 8.0 kPa, and from - 1.4 (l.6)% to +2.2 (2.3)% for Pao,