Physica Medica 32 (2016) 272–276

Contents lists available at ScienceDirect

Physica Medica j o u r n a l h o m e p a g e : h t t p : / / w w w. p h y s i c a m e d i c a . c o m

Technical Notes

Objective criteria for acceptability and constancy tests of digital subtraction angiography Hugo de las Heras a,*, Ricardo Torres b, José Miguel Fernández-Soto a,c, Eliseo Vañó a,c a b c

Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), c. Profesor Martín Lagos, S/N, 28040 Madrid, Spain Servicio de Radiofísica y Protección Radiológica. Hospital Universitario Río Hortega, c/ Dulzaina, 2, 47012 Valladolid, Spain Departamento de Radiología, Facultad de Medicina, Universidad Complutense, Avda. Complutense s/n, 28040 Madrid, Spain

A R T I C L E

I N F O

Article history: Received 28 August 2015 Received in revised form 13 October 2015 Accepted 14 October 2015 Available online 28 October 2015 Keywords: Quality control Digital subtraction angiography Radiologic phantom Radiation dose

A B S T R A C T

Purpose: Demonstrate an objective procedure to quantify image quality in digital subtraction angiography (DSA) and suggest thresholds for acceptability and constancy tests. Methods: Series of images were obtained in a DSA system simulating a small (paediatric) and a large patient using the dynamic phantom described in the IEC and DIN standards for acceptance tests of DSA equipment. Image quality was quantified using measurements of contrast-to-noise ratio (CNR). Overall scores combining the CNR of 10–100 mg/ml Iodine at a vascular diameter of 1–4 mm in a homogeneous background were defined. Phantom entrance surface air kerma (Ka,e) was measured with an ionisation chamber. Results: The visibility of a low-contrast vessel in DSA images has been identified with a CNR value of 0.50 ± 0.03. Despite using 14 times more Ka,e (8.85 vs 0.63 mGy/image), the protocol for large patients showed a decrease in the overall score CNRsum of 67% (4.21 ± 0.06 vs 2.10 ± 0.05). The uncertainty in the results of the objective method was below 5%. Conclusion: Objective evaluation of DSA images using CNR is feasible with dedicated phantom measurements. An objective methodology has been suggested for acceptance tests compliant with the IEC/DIN standards. The defined overall scores can serve to fix a reproducible baseline for constancy tests, as well as to study the device stability within one acquisition series and compare different imaging protocols. This work provides aspects that have not been included in the recent European guidelines on Criteria for Acceptability of Medical Radiological Equipment. © 2015 Associazione Italiana di Fisica Medica. Published by Elsevier Ltd. All rights reserved.

Introduction After the commercial introduction of digital subtraction angiography (DSA) in the eighties [1], the technique has been included in most general interventional imaging systems. Although the subtraction process increases image noise, the real-time visualisation of low-contrast vessels is improved due to the use of contrast media and the removal of distracting background tissue [2]. DSA poses a real challenge to radiation protection because it is the modality producing one of the largest patient instantaneous dose rates among all x-ray examinations. Incident air kerma values of 4–8 mGy per frame at the patient entrance may result from abdominal DSA images obtained with radiation pulses of 40–80 ms. These values are equivalent to instantaneous dose rates of around 100 mGy/s or 6000 mGy/min, about 100 times more than typical

* Corresponding author. Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), c. Profesor Martín Lagos, S/N, 28040 Madrid, Spain. Tel.: +49 89 45 24 7050; fax: +49 8106 249 119. E-mail address: [email protected] (H. de las Heras)

fluoroscopy acquisitions. One reason for these large rates is the fact that DSA is often configured to use a higher dose per frame to compensate in part or in full for the additional noise in the final subtracted image [3]. Despite its widespread use and the large dose rates employed, publications about imaging acquisition optimisation and quality control tests for DSA [4,5] are scarce. In particular, the latest European guidelines on “Criteria for Acceptability of Medical Radiological Equipment used in Diagnostic Radiology, Nuclear Medicine and Radiotherapy” [6] do not even mention the subject of DSA. The AAPM report 15 [7] describes a phantom for quality control of DSA, which includes vessels of different sizes. The IEC standard 61223-3-3 describes a similar phantom with some improvements learnt over time [8], such as a minimum thickness of the vessels (to avoid detectability being significantly influenced by spatial resolution). The German guidelines for quality control [9] make use of this phantom, which is also described in the German standard DIN EN 612233-3 [10] (which substituted 6868-54) and DIN 6868-4 [11] (former 6868-8), to establish minimum image quality requirements for DSA images at a reference level of the image detector dose. The Spanish protocol [12] and the British IPEM report 91 [13] (former report 77)

http://dx.doi.org/10.1016/j.ejmp.2015.10.089 1120-1797/© 2015 Associazione Italiana di Fisica Medica. Published by Elsevier Ltd. All rights reserved.

H. de las Heras et al./Physica Medica 32 (2016) 272–276

273

make use of different test objects [14,15] for each one of the aspects involved in DSA. The features described in the documents mentioned above for DSA quality control are mask creation and logarithmic subtraction, dynamic range, contrast sensitivity and artefacts [8]. Except for the contrast sensitivity, these features are very stable in time (experience shows that the results of the tests are almost always the same as in the acceptance test), so they only need to be checked during acceptance tests and after major changes in the imaging system (hardware and software). Regarding the evaluation of contrast sensitivity, all the mentioned documents make use of subjective methods like the visual detection of low-contrast signals. However, an objective and sensitive evaluation of contrast sensitivity and dose per image is necessary to reproducibly detect small changes in the device performance. This evaluation should be part of the acceptance tests, but it is particularly important for constancy tests and for any attempts to optimise the relation between dose and image quality in DSA protocols. The purpose of this work is to describe and test a practical procedure to objectively evaluate image quality in DSA series. The defined scores shall serve to establish criteria for acceptance and constancy tests. The method is exemplified using the phantom described in the international standard IEC 61223-3-3 for acceptance tests in DSA systems. Materials and methods The phantom: IEC design type B The phantom (QUART GmbH, Zorneding, Germany) is described in detail in the IEC standard 61223-3-3 annex B [8] (Fig. 1). It is also compliant with the German standard DIN EN 61223-3-3 and 6868-4 (former 6868-8). The phantom simulates the injection of a contrast medium into vessels under continuous x-ray irradiation. The body of the phantom is a 57 mm-height PMMA block with a copper (99.5% purity) wedge featuring seven steps between 0.2 and 1.4 mm (equivalent to PMMA thicknesses between 6 and 45 mm). Four aluminium strips (0.05, 0.1, 0.2 and 0.4 mm thickness and 99.5% purity) on a slider simulate blood vessels that correlate with a density of 5–10 mg Iodine per ml at a vascular diameter of 1–4 mm. The injection process is simulated by inserting the aluminium strips (on the slider surface) into the field of view. This process takes less than 0.3 seconds [16]. The 7-step copper wedge on top of the slider serves to assess the ability of the x-ray system to image the aluminium strips within a background of varying attenuation. Due to the slider movement, the phantom shifts slightly during the simulation of the contrast injection. This shift simulates a slight movement of the patient during acquisition of the post-contrast images. Algorithms for mask pixel shift [2] can be tested by observing the resulting thin grey stripes between the steps (see Fig. 2). Additionally, negative contrast (a signal that is darker than the background) can also be tested by inverting the position of the slider with the aluminium strips. However, mask pixel shift and negative contrast are not considered in this work. Experimental set-up and data acquisition Digital subtraction images of the IEC type B phantom (placed at the isocentre of the x-ray system) were taken with a Philips Allura biplane FD 20/10 (Philips Healthcare, Eindhoven, The Netherlands). To simulate an adult patient, extra series were taken with additional 20 cm of polymethylmethacrylate (PMMA) and lowering the patient couch to keep the phantom centred at the isocentre (see Fig. 3) [16]. The exposure parameters (determined by the automatic exposure control) are indicated in Table 1. The slider of the

Figure 1. The IEC8 design type B (reproduced with permission). All distances are expressed in mm.

phantom was activated from the control room using a 5 m-cable and a remote control provided by the phantom manufacturer. Phantom entrance surface air kerma (Ka,e) [17] (including backscatter) was measured using an ionisation chamber (20X6-60 E; S/N: 32263, Ref. 22195, electrometer 22195, Radcal Corp., Monrovia, CA). The chamber was fixed with tape to the patient couch. Data analysis The images were analysed following the mentioned IEC standard, i.e. visually evaluating the image acquired after mask creation and the image acquired after simulation of the contrast injection. The standard states that “DSA contrast sensitivity is assessed by counting the number of steps on which each simulated vascular structure is visible”, which is a highly subjective task (See Fig. 2).

274

H. de las Heras et al./Physica Medica 32 (2016) 272–276

Figure 2. Post-contrast (subtracted) images of the phantom without additional attenuation (left) and with additional 20 cm of PMMA (right). The dashed ROI indicates the region used to obtain the background signal xb and the standard deviation σb. The solid ROIs indicate two examples of the regions used to obtain the signal averages xs. The arrows indicate the fourth step of the copper wedge.

To obtain an objective evaluation of contrast sensitivity, the contrastto-noise ratio (CNR) corresponding to each step was measured. For this purpose, a reference region of interest (ROI) including 120 pixels was selected to measure the mean grey level of the background xb and the corresponding standard deviation σb (dashed ROI in Fig. 2).

This reference ROI was chosen to be the region between strips with the lowest noise level. Same-sized ROIs were selected to measure the mean grey level xs within the aluminium strips (one example is shown by the solid ROI in Fig. 2). The values of CNR were obtained as

CNR = ( x s − x b ) σb

Figure 3. (Left) The fluoroscopy device and the phantom on the patient couch. (Right) Set-up including additional 20 cm of PMMA.

Table 1 Exposure parameters (kVp and mAs are taken from the DICOM header). Additional PMMA

Filtration (mm Al)

Tube voltage (kVp)

Tube current–time product (mAs)

Image rate (pulse/s)

Ka,e (mGy/ image)

none 20 cm

3.9 3.9

80 90

5 87

3 3

0.63 8.85

(1)

For practical purposes one needs to focus on a few measurements rather than measuring the CNR in all steps. However, due to the noise (mainly quantum noise) of x-ray images, to obtain reproducible results (reduce uncertainties) it is necessary to evaluate different ROIs as opposed to one single ROI, as well as different images belonging to the same series. This trade-off was solved by focusing on the four vessels at the fourth step of the wedge (indicated with arrows in Fig. 2) because it is the middle step, and calculating the average of the measurements obtained in the last five images of each series. Two types of scores were defined to quantify overall image quality. The first one, CNRdif, is the difference between the maximum and the minimum CNR values measured in this fourth step. This quantity tests the ability of the x-ray system to differentiate between tissues of similar absorption properties, such as heart tissue and blood vessel tissue. The second score, CNRsum, is the sum of the four CNR values measured in the fourth step. This quantity combines the degree of visibility of the four vessels into one single value. Results Figure 2 shows the DSA images of the phantom for the exposure corresponding to a small patient (left) and a large patient (right).

Figure 4. Grey level profiles of step 4 from the post-contrast (subtracted) images shown in Fig. 3. Each shown profile is the average of 10 pixel rows within the step. The CNR corresponding to each simulated vessel is displayed above its profile. The profile of the thinnest vessel has been framed for clarity.

H. de las Heras et al./Physica Medica 32 (2016) 272–276

275

Table 2 Values of CNR in step 4, averaged over the last five images of each series, and corresponding overall scores. The standard deviation is indicated in brackets. Step

Phantom alone Ka,e: 0.63 mGy/image

Phantom plus 20 cm PMMA Ka,e: 8.85 mGy/image

Vessel

4

Overall scores

Vessel

Overall scores

1

2

3

4

CNRsum

CNRdif

1

2

3

4

CNRsum

CNRdif

0.47 (0.05)

0.59 (0.03)

1.08 (0.05)

2.07 (0.06)

4.21 (0.14)

1.60 (0.02)

0.32 (0.02)

0.37 (0.03)

0.58 (0.03)

0.82 (0.05)

2.10 (0.11)

0.50 (0.04)

Figure 4 shows the corresponding profiles of the four vessels at the 4th step of the copper wedge. The profiles show that the thinnest vessel is almost indistinguishable from the background noise, especially for the large patient, although from the DSA image some observers may say that this vessel is “visible” because they know it is there. On the contrary, calculating the CNR provides an objective measure of the degree of visibility of this vessel against the quantum noise of the background. The CNR corresponding to the profiles are also shown in Fig. 4 for comparison. Table 2 shows that, as an example, the term “visible” could be substituted by “a CNR above 0.50” because this value of the CNR corresponds to a vessel portion that is hardly recognisable in Fig. 2 and can be interpreted as a vessel in the profile of Fig. 4. Using this substitution, the visual evaluation of contrast sensitivity mentioned in the IEC standard for acceptance tests is transformed into an objective evaluation. The suggested scores describing overall image quality (CNRsum and CNRdif) have been calculated for the 4th step of the wedge and averaged over the last five images of each series. The results are shown in Table 2 including the corresponding standard deviations in brackets. Using these scores it is easy to see that despite using 14 times more Ka,e (8.85 vs 0.63 mGy/image), the protocol for large patients showed a decrease in CNRsum of 67%. This result reflects the larger amount of backscatter and image noise in the case of the adult patient. The method and the scores suggested here can be used to similarly evaluate other imaging protocols. In addition, the CNR measurements for one series (keeping exactly the same size and position of the ROI) are helpful to study the stability of the system within that acquisition series. In our example, the coefficient of variation among the last five images in the adult series for the first step (CNR = 0.34, 0.29, 0.35, 0.32, 0.32, average 0.32) was 6%, and 7% on average for all steps. In comparison, the coefficient of variation among five measurements from one single image (changing the size of the ROI) was 5% on average for all steps. Therefore, to conveniently study different acquisition protocols, one should calculate the average score for each ROI in at least a few images within the DSA series. The variability in these images (represented by the standard deviations in Table 2) is due to the unavoidable fluctuations within the imaging system (mainly due to the thermionic emission within the x-ray tube and to the Swank noise within the flat panel). We have shown that using five images and the indicated overall scores, the uncertainty in the results (given by the standard error of the mean) is below 5% in all cases. Discussion Visual evaluations of image quality are very subjective because different observers, and also the same observer under different circumstances, can have different opinions about the visibility of a certain signal. Not only observer performance, but also the performance and the set-up of the monitor and the ambient light affect the visual evaluations. Therefore the usual subjective procedure to analyse the images during quality control should be improved, being this imaging mode a high-dose modality. The evaluation of CNR solves this problem by assigning an objective score to the degree of visibility of each signal, which in this

case is a simulated low-contrast vessel. This evaluation requires a DICOM viewer, the measurement of average and standard deviation in several ROIs and a simple calculation using formula [1]. Despite this extra effort in comparison to the subjective method, the result of this calculation provides an objective assessment of image quality that can be used to reproducibly assess contrast sensitivity during acceptance tests, set action or suspension levels and establish baseline values for constancy tests. The quantification provided by the scores CNRsum and CNRdif (averaged over several images within a series) can also serve to make comparisons between different DSA imaging protocols or, more importantly, to define exposure modes and optimise DSA imaging protocols. The described scores may not be generalised and directly compared to results from other DSA systems. Systems might have different processing algorithms and these might change with any software upgrades. As an example, Fig. 2 shows that the thinnest aluminium strip at step four is hardly visible in the right image of Fig. 2, but it can still be followed against a homogeneous background. Therefore, the scores corresponding to the right image (CNR sum ≈ 2.10 and CNRdif ≈ 0.50) could be used to pre-define a low-dose exposure mode. This mode could be used by the radiologist when high image quality is not required (for example to follow a guide wire within a big vessel). On the other hand the scores corresponding to the left image (CNRsum > 4.21 and CNRdif > 1.60) could be used to set exposure modes for clinical tasks that require better image quality, such as certain procedures in neuroangiography. The scores can also contribute to objective procedures applied to other modalities [18], such as paediatric cardiology, where the amount of contrast agent and dose is especially critical. Similarly, the CNR measured for different vessel portions (or in other positions within other dynamic phantoms) could be used to propose objective acceptance criteria in national and international standards. For example a certain overall score could be obtained using the method suggested here in a future wide survey by agreement with all stakeholders. The resulting values could be considered as intervention or suspension level in DSA protocols. These levels have not been included in the recent European guidelines on Criteria for Acceptability of Medical Radiological Equipment. The described objective evaluation method using five images has shown an uncertainty below 8% (for CNR dif) and below 4% (for CNRsum). The largest sources of uncertainty are the unavoidable fluctuations within a DSA loop together with the position and size of the chosen ROI. In comparison, the difference between the images shown in Fig. 2, which visually provide the same result (“acceptable”), is 67% (4.21 vs 2.10 as evaluated by CNRsum). If a larger sensitivity is required to detect even smaller differences in performance, the number of images included in the analysis can be accordingly increased. This is the main advantage of mathematical evaluations as opposed to visual evaluations. The CNR measurements were performed on subtracted images, which include processing algorithms that may be different for each x-ray manufacturer. Since both, subtraction and processing, affect the actual images that are provided to the radiologist, the two of them were evaluated together. This way, the results provided by CNR measurements could also help to study the correspondence between

276

H. de las Heras et al./Physica Medica 32 (2016) 272–276

measurements of phantom (or “technical”) image quality and clinical image quality. Objective measures like CNR can study this correspondence with a higher reliability than traditional methods based on visual evaluations. As a recommendation to phantom manufacturers, we think that it would be helpful to include a specific slot for dosimeter detectors in these phantoms, so that the dose measurement is made always in the same position. In addition, phantoms for DSA assessment should be easy to interpret, provide practical information for physicists and technicians and enable remote data acquisition to reduce irradiation risk of the user. For ease of interpretation, manufacturers could design dedicated software to evaluate the objective parameters available within their phantoms. Finally, it is crucial that phantom manufacturers stick to the measures indicated in the standard documents (like the IEC 61223-3-3) to guarantee reproducibility within different phantoms. As long as the IEC phantom is used, the metrics presented here are valid. Using other dynamic phantoms to apply the presented method is also possible, but it requires the definition of other ROIs for the calculation of the CNR metrics and the definition of corresponding acceptance or intervention levels.

Conclusion Numerical evaluation of DSA images using CNR is feasible with dedicated phantom measurements. This quantification of image quality has been used to find an objective evaluation method for acceptance tests in accordance to the IEC standard. The defined overall scores can serve to fix a reproducible baseline for constancy tests, as well as to study the device stability within one acquisition series and compare different imaging protocols.

Acknowledgements The authors thank the International Electrotechnical Commission (IEC) for permission to reproduce information from its International Standard IEC 61223-3-3 ed.1.0 (1996). All such extracts are copyright of IEC, Geneva, Switzerland. All rights reserved. Further information on the IEC is available from www.iec.ch. IEC has no responsibility for the placement and context in which the extracts and contents are reproduced by the authors, nor is IEC in any way responsible for the other content or accuracy therein. The authors also thank Santiago Cano Palomo and Manuel María Moreu Gamazo for their help during image acquisitions, as well as Yvonne Schöfer and the reviewers of this paper for their help to improve its readability and completeness. The first author works part-time as consultant for QUART, a company that manufactures the phantom we used in this study. The other authors, including the senior author, have no conflict of interest and have always shared access to all data related to this work.

Appendix Supplementary material Supplementary data to this article can be found online at doi:10.1016/j.ejmp.2015.10.089. References [1] Balter S, Ergun D, Tscholl E, Buchmann F, Verhoeven L. Digital subtraction angiography: fundamental noise characteristics. Radiology 1984;152:195–8. [2] Pooley RA, McKinney JM, Miller DA. The AAPM/RSNA physics tutorial for residents digital fluoroscopy. Radiographics 2011;21:521–34. [3] Jones K, Balter S, Rauch P, Wagner LK. Medical imaging using ionizing radiation: optimization of dose and image quality in fluoroscopy. Med Phys 2014;41(1):014301. [4] Kotre CJ, Marshall NW. A review of image quality and dose issues in digital fluorography and digital subtraction angiography. Radiat Prot Dosimetry 2001;94(1–2):73–6. [5] Doi K. Design criteria for and evaluation of phantoms employed for digital angiography including contrast media studies. Radiat Prot Dosimetry 1993;49(1–3):249–55. [6] European Commission. Criteria for acceptability of medical radiological equipment used in diagnostic radiology, Nuclear medicine and radiotherapy. Radiation protection N° 162. Luxembourg: Publications Office of the European Union; 2012. [7] AAPM Report 15. Performance evaluation and quality assurance in digital subtraction angiography: a report of the digital radiography/fluorography task group diagnostic x-ray imaging committee. AAPM Report 15. New York: American Institute of Physics, Inc.; 1985. [8] IEC 61223-3-3. Evaluation and routine testing in medical imaging departments – Part 3-3: acceptance tests – Imaging performance of X-ray equipment for digital subtraction angiography (DSA). Geneva: International Electrotechnical Commission; 1996. [9] Qualitätssicherungs-Richtlinie (QS-RL). Guideline to perform quality assurance in x-ray facilities for examination or treatment of humans after paragraphs §§ 16 and 17 of the x-ray ordinance – 2003, Berlin; updated in 2011. [10] DIN EN 61223-3-3:2000-12. Evaluation and routine testing in medical imaging departments – part 3-3: acceptance tests; imaging performance of X-ray equipment for digital subtraction angiography (IEC 61223-3-3:1996). German version EN 61223-3-3:1996. Berlin: Beuth Verlag; 2000. [11] DIN 6868-4. Image quality assurance in diagnostic X-ray departments – Part 4: constancy testing of medical X-ray equipment for fluoroscopy. Berlin: Beuth Verlag; 2007. [12] SEFM, SEPR, SERAM. Protocolo español de control de calidad en radiodiagnóstico (aspectos técnicos). Madrid: Sociedad Española de Física Médica, Sociedad Española de Protección Radiológica, Sociedad Española de Radiología Médica; 2011. [13] IPEM. Recommended standards for the routine performance testing of diagnostic X-ray imaging systems, report No 91. York: Institute of Physics and Engineering in Medicine; 2005. [14] Cowen AR, Haywood JM, Workman A, Clarke OF. A set of X-ray test objects for image quality control in digital subtraction fluorography. I: design considerations. Br J Radiol 1987;60:1001–9. [15] Thijssen MA, Thijssen HO, Merx JL, van Woensel MP. Quality analysis of DSA equipment. Neuroradiology 1988;30:561–8. [16] See the actual frame acquisitions (AVI video files) in the following links: Small patient, ; Large patient, . [17] ICRU. Patient dosimetry for X rays used in medical imaging. ICRU report 74. J ICRU 2005;5:1–113. [18] Lubis LE, Bayuadi I, Pawiro SA, Ng KH, Bosmans H, Soejoko DS. Optimization of dose and image quality of paediatric cardiac catheterization procedure. Phys Med 2015;doi:10.1016/j.ejmp.2015.05.011. pii: S1120-1797(15)00121-0, Epub ahead of print.