Integrated Environmental Assessment and Management — Volume 10, Number 3—pp. 325–326 © 2014 SETAC
325
Authors' reply Kirk T O'Reilly,*y Jaana Pietari,z and Paul D Boehmz yExponent, Bellevue, Washington, USA zExponent, Maynard, Massachusetts, USA
(Submitted 25 April 2014; Accepted 28 April 2014)
* To whom correspondence may be addressed: [email protected] Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/ieam.1548
represent RTS source profiles, CMB suggested a high source contribution to most urban sediment samples. When we used the test‐plot samples (Mahler et al. 2005) as RTS profiles, the median RTS contribution to the 120 sediment samples dropped from 49 to zero percent. Van Metre, Mahler, and Crane also failed to include any source profile associated with manufactured gas plant operations, even though such PAH‐rich wastes are known to be common sources of PAHs in urban water systems. Crane is correct that the use of a negative control is not discussed in the CMB manual (Coulter 2004). This is because the manual describes how to use the model to evaluate the contribution of known sources, not as described by Van Metre and Mahler (2010), to “test the hypothesis that CT sealcoat is a major source of PAHs to urban lakes.” The CMB outputs with RTS source profiles provide no support for that hypothesis, unless they are compared with model result without RTS as a source input. We commend Crane (2014), who, unlike Van Metre and Mahler (2010), did include the results of an RTS‐ free negative control and found a significant correlation between measured and modeled sediment concentration regardless of whether an RTS source profile was included. We obtained a similar result when we re‐ran Van Metre and Mahler’s CMB approach but excluded an RTS source. We disagree, however, with the statement in Crane’s comment that model performance was significantly better when “RTS sources” (urban parking lot dust) were included as model inputs. Whereas Crane (2014) found slight but significant differences in secondary model fitting parameters, R2 and x2, this is expected when comparing a model run with 5 source inputs, 2 RTS and 3 non‐RTS, with a model run with just 3 source inputs. More importantly, it does not overcome the fact that the ability of the model to fit sediment concentration in the absence of an RTS source profile indicates that CMB supports the null hypothesis. Neither Van Metre and Mahler’s (2010) nor Crane’s (2014) use of CMB demonstrates whether RTS is actually a source in any urban sediment. A large part of Crane’s comments focus on factors that influence the uncertainty of 3 multivariate methods, CMB, PCA, and UNMIX. This is consistent with the main point of our paper. We do not dispute that issues with each method must be considered. That is why we recommend that multiple methods be used and the results compared. Crane’s discussion of PCA and sample size suggests a misunderstanding of both the method and the references cited. Although PCA has a range of applications that may differ in data requirements, we use it as an exploratory tool to compare environmental samples with proposed sources (Johnson et al. 2004). The results presented in O’Reilly et al. (2014a) are valuable for indicating chemical differences between sediments and the RTS materials claimed as their source of PAHs. When PCA is applied to samples that are affected by a mixture of sources, samples representing source chemistry should plot as end members, and the mixtures should plot within the area bounded by these samples (Johnson et al. 2004). When we applied PCA with the data presented in
Letters to the Editor
DEAR SIR: We appreciate the opportunity to respond to Crane’s comments on our article, “Parsing Pyrogenic Polycyclic Aromatic Hydrocarbons: Forensic Chemistry, Receptor Models, and Source Control Policy” (O’Reilly et al. 2014a). Dr. Crane raises a number of interesting points, but these serve to support, not distract from, the conclusions of our paper. In our paper, we describe the challenges of using multivariate approaches, such as EPA’s Chemical Mass Balance (CMB) model and principal component analysis (PCA), to identify sources of pyrogenic polycyclic aromatic hydrocarbons (PAH) in urban sediments. We also discuss why it is critical that, when these methods are used to promote source control policy, technical specialists accurately describe the uncertainty inherent in the results. Our paper included a case study focused on the application of CMB discussed in Van Metre and Mahler (2010), and because Crane (2014) used almost the same methods, we discuss both papers in this response. In their evaluations of the hypothesis that refined tar‐based pavement sealers (RTS) are a source of PAHs in urban sediments, Van Metre, Mahler, and Crane fail to recognize that CMB output has little value unless the actual sources to a system are known. This statement is based on the simple mathematical concept that one can calculate an unknown only when there are a sufficient number of knowns. While numerous authors, such as Li et al. (2003), have used receptor models to evaluate contributions to sediments, the outputs are ultimately only as meaningful as the inputs. It has not been proven that these models can parse out the multiple contributors of PAHs to urban sediments. Without independent verification that the source inputs used are appropriate and sufficient, CMB output cannot be used to verify the contribution of a given source. In both Van Metre and Mahler (2010) and Crane (2014), most of the source profiles used as model inputs were mere mathematical constructs that have not been shown to represent real sources in any of the watersheds investigated. Generation of these artificial source profiles using manipulated averages of published means and geometric means is described in Li et al. (2003). Samples used to generate these profiles include those taken from within the smokestacks of industrial facilities in Taiwan (Yang et al. 2002) and the atmosphere of the Russian Arctic (Halsall et al. 1997). Li et al. (2003) admit that the coefficient of variation or relative standard deviations often exceed 100%, which indicates the final profiles have little similarity with the underlying data. The only source profiles based directly on measurements of potential source materials were urban parking‐lot dust samples and samples collected from RTS test plots. When Van Metre and Mahler (2010) or Crane (2014) used parking‐lot dust to
326
Van Metre and Mahler (2010) or Crane (2014), we found very few sediment samples within the area bounded by their proposed sources. These findings support our contention that neither Van Metre and Mahler (2010) nor Crane (2014) used appropriate source profiles when running CMB. On the issue of source collinearity, Crane states that we did not identify sources that have a PAH profile similar to RTS. We point her to Table 1 in O’Reilly et al. (2012), and Van Metre and Mahler’s (2010) Table S4, which demonstrate high correlation between potential source profiles. The CMB output files contain 2 performance factors—source estimability and Tstat—that indicate the influence of collinearity. Although the model will estimate a source contribution even if a source is determined to be inestimable, it flags each source as either estimable or inestimable (Coulter 2004). Inestimable sources are caused by excessive similarity or collinearity among the source profiles. When we ran CMB described as Model A in Van Metre and Mahler (2010), RTS was not identified as an estimable source for any of the 120 sediment samples, whereas fewer than 50% passed the Tstat criteria. Crane (2014) did not provide estimability results, so her claim that no collinearity was observed when she ran CMB is incomplete. Moving on to policy, our goal was not to conduct a survey of PAH regulations but to focus on the use of receptor models to support source‐control decisions. Van Metre and Mahler of the US Geological Survey and Crane of the Minnesota Pollution Control Agency clearly believe that RTS should be banned and have worked hard to advocate that position. Mills (2000) noted that an inherent conflict of interest exists when agency staff, who should be the suppliers of policy‐neutral science, advocate for a preferred policy position. Although there may not always be a solid line between policy neutrality and advocacy, scientists must be “brutally honest” with decision makers and the public about both the results and inherent uncertainties of a science‐ based analysis (Lackey 2007). Problems arise when personal preference influences how research findings are presented. Since the first paper on this topic (Mahler et al. 2005), when the results of a local study were spun to suggest a nationwide issue, Van Metre, Mahler, and Crane have interpreted data in ways that support their policy preferences. Given the layperson’s unfamiliarity with receptor models, it is easy to overstate the importance of their results by ignoring problems with the selected source profiles and minimizing the uncertainty in the output. An example of the latter is quoting calculated source contributions to a tenth of a percent, even though the inputs have uncertainty of 20% to 40%.
Integr Environ Assess Manag 10, 2014—KT O'Reilly et al.
In conclusion, we agree with Crane that the forensic methods and receptor models discussed have limitations. But, unlike Van Metre, Mahler, and Crane, our goal was not to support a hypothesis by presenting limited results from application of a single method. Rather, the purpose of our manuscript was to discuss the strengths and weaknesses of the models, and given their limitations, to recommend how they should be applied to assist in setting source‐control policy. Acknowledgment—We have received support from the Pavement Coatings Technology Council. The opinions are those of the authors.
REFERENCES Coulter CT. 2004. EPA‐CMB 8.2 users' manual. EPA 452/R‐04‐011. Washington DC: US Environmental Protection Agency. Crane JL. 2014. Source apportionment and distribution of polycyclic aromatic hydrocarbons, risk considerations, and management implications for urban stormwater pond sediments in Minnesota, USA. Arch Environ Contam Toxicol 66:176–200. Halsall CJ, Barrie LA, Fellin P, Muir DCG, Billeck BN, Lockhart L, Rovinsky FY, Kononov EY, Pastukhov B. 1997. Spatial and temporal variation of polycyclic aromatic hydrocarbons in the Arctic atmosphere. Environ Sci Technol 31: 3593–3599. Johnson GW, Ehrlich R, Full W. 2004. Principal components analysis and receptor models in environmental forensics. In: Murphy BL, Morrison RD, editors. Introduction to environmental forensics. Burlington (MA): Elsevier Academic Press. 461 p. Lackey RT. 2007. Science, scientists and policy advocacy. Conserv Biol 21: 12–17. Li A, Jang J‐K, Scheff PA. 2003. Application of EPA CMB8.2 model for source apportionment of sediment PAHs in Lake Calumet, Chicago. Environ Sci Technol 37:2958–2965. Mahler BJ, Van Metre PC, Bashara TJ, Wilson JT, Johns DA. 2005. Parking lot sealcoat: An unrecognized source of urban polycyclic aromatic hydrocarbons. Environ Sci Technol 39:5560–5566. Mills TJ. 2000. Policy advocacy by scientists risks science credibility and may be unethical. Northwest Sci 74:165–168. O'Reilly K, Pietari J, Boehm P. 2012. Forensic assessment of refined tar‐based sealers as a source of polycyclic aromatic hydrocarbons (PAHs) in urban sediments. Environ Forensics 13:185–196. O'Reilly KT, Pietari J, Boehm PD. 2014a. Parsing pyrogenic polycyclic aromatic hydrocarbons: Forensic chemistry, receptor models, and source control policy. Integr Environ Assess Manag 10:279–285. Van Metre PC, Mahler BJ. 2010. Contribution of PAHs from coal–tar pavement sealcoat and other sources to 40 U.S. lakes. Sci Tot Environ 409: 334–344. Yang HH, Lai SO, Hsieh LT, Hsueh HJ, Chi T. 2002. Profiles of PAH emission from steel and iron industries. Chemosphere 48:1061–1074.
325
Authors' reply Kirk T O'Reilly,*y Jaana Pietari,z and Paul D Boehmz yExponent, Bellevue, Washington, USA zExponent, Maynard, Massachusetts, USA
(Submitted 25 April 2014; Accepted 28 April 2014)
* To whom correspondence may be addressed: [email protected] Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/ieam.1548
represent RTS source profiles, CMB suggested a high source contribution to most urban sediment samples. When we used the test‐plot samples (Mahler et al. 2005) as RTS profiles, the median RTS contribution to the 120 sediment samples dropped from 49 to zero percent. Van Metre, Mahler, and Crane also failed to include any source profile associated with manufactured gas plant operations, even though such PAH‐rich wastes are known to be common sources of PAHs in urban water systems. Crane is correct that the use of a negative control is not discussed in the CMB manual (Coulter 2004). This is because the manual describes how to use the model to evaluate the contribution of known sources, not as described by Van Metre and Mahler (2010), to “test the hypothesis that CT sealcoat is a major source of PAHs to urban lakes.” The CMB outputs with RTS source profiles provide no support for that hypothesis, unless they are compared with model result without RTS as a source input. We commend Crane (2014), who, unlike Van Metre and Mahler (2010), did include the results of an RTS‐ free negative control and found a significant correlation between measured and modeled sediment concentration regardless of whether an RTS source profile was included. We obtained a similar result when we re‐ran Van Metre and Mahler’s CMB approach but excluded an RTS source. We disagree, however, with the statement in Crane’s comment that model performance was significantly better when “RTS sources” (urban parking lot dust) were included as model inputs. Whereas Crane (2014) found slight but significant differences in secondary model fitting parameters, R2 and x2, this is expected when comparing a model run with 5 source inputs, 2 RTS and 3 non‐RTS, with a model run with just 3 source inputs. More importantly, it does not overcome the fact that the ability of the model to fit sediment concentration in the absence of an RTS source profile indicates that CMB supports the null hypothesis. Neither Van Metre and Mahler’s (2010) nor Crane’s (2014) use of CMB demonstrates whether RTS is actually a source in any urban sediment. A large part of Crane’s comments focus on factors that influence the uncertainty of 3 multivariate methods, CMB, PCA, and UNMIX. This is consistent with the main point of our paper. We do not dispute that issues with each method must be considered. That is why we recommend that multiple methods be used and the results compared. Crane’s discussion of PCA and sample size suggests a misunderstanding of both the method and the references cited. Although PCA has a range of applications that may differ in data requirements, we use it as an exploratory tool to compare environmental samples with proposed sources (Johnson et al. 2004). The results presented in O’Reilly et al. (2014a) are valuable for indicating chemical differences between sediments and the RTS materials claimed as their source of PAHs. When PCA is applied to samples that are affected by a mixture of sources, samples representing source chemistry should plot as end members, and the mixtures should plot within the area bounded by these samples (Johnson et al. 2004). When we applied PCA with the data presented in
Letters to the Editor
DEAR SIR: We appreciate the opportunity to respond to Crane’s comments on our article, “Parsing Pyrogenic Polycyclic Aromatic Hydrocarbons: Forensic Chemistry, Receptor Models, and Source Control Policy” (O’Reilly et al. 2014a). Dr. Crane raises a number of interesting points, but these serve to support, not distract from, the conclusions of our paper. In our paper, we describe the challenges of using multivariate approaches, such as EPA’s Chemical Mass Balance (CMB) model and principal component analysis (PCA), to identify sources of pyrogenic polycyclic aromatic hydrocarbons (PAH) in urban sediments. We also discuss why it is critical that, when these methods are used to promote source control policy, technical specialists accurately describe the uncertainty inherent in the results. Our paper included a case study focused on the application of CMB discussed in Van Metre and Mahler (2010), and because Crane (2014) used almost the same methods, we discuss both papers in this response. In their evaluations of the hypothesis that refined tar‐based pavement sealers (RTS) are a source of PAHs in urban sediments, Van Metre, Mahler, and Crane fail to recognize that CMB output has little value unless the actual sources to a system are known. This statement is based on the simple mathematical concept that one can calculate an unknown only when there are a sufficient number of knowns. While numerous authors, such as Li et al. (2003), have used receptor models to evaluate contributions to sediments, the outputs are ultimately only as meaningful as the inputs. It has not been proven that these models can parse out the multiple contributors of PAHs to urban sediments. Without independent verification that the source inputs used are appropriate and sufficient, CMB output cannot be used to verify the contribution of a given source. In both Van Metre and Mahler (2010) and Crane (2014), most of the source profiles used as model inputs were mere mathematical constructs that have not been shown to represent real sources in any of the watersheds investigated. Generation of these artificial source profiles using manipulated averages of published means and geometric means is described in Li et al. (2003). Samples used to generate these profiles include those taken from within the smokestacks of industrial facilities in Taiwan (Yang et al. 2002) and the atmosphere of the Russian Arctic (Halsall et al. 1997). Li et al. (2003) admit that the coefficient of variation or relative standard deviations often exceed 100%, which indicates the final profiles have little similarity with the underlying data. The only source profiles based directly on measurements of potential source materials were urban parking‐lot dust samples and samples collected from RTS test plots. When Van Metre and Mahler (2010) or Crane (2014) used parking‐lot dust to
326
Van Metre and Mahler (2010) or Crane (2014), we found very few sediment samples within the area bounded by their proposed sources. These findings support our contention that neither Van Metre and Mahler (2010) nor Crane (2014) used appropriate source profiles when running CMB. On the issue of source collinearity, Crane states that we did not identify sources that have a PAH profile similar to RTS. We point her to Table 1 in O’Reilly et al. (2012), and Van Metre and Mahler’s (2010) Table S4, which demonstrate high correlation between potential source profiles. The CMB output files contain 2 performance factors—source estimability and Tstat—that indicate the influence of collinearity. Although the model will estimate a source contribution even if a source is determined to be inestimable, it flags each source as either estimable or inestimable (Coulter 2004). Inestimable sources are caused by excessive similarity or collinearity among the source profiles. When we ran CMB described as Model A in Van Metre and Mahler (2010), RTS was not identified as an estimable source for any of the 120 sediment samples, whereas fewer than 50% passed the Tstat criteria. Crane (2014) did not provide estimability results, so her claim that no collinearity was observed when she ran CMB is incomplete. Moving on to policy, our goal was not to conduct a survey of PAH regulations but to focus on the use of receptor models to support source‐control decisions. Van Metre and Mahler of the US Geological Survey and Crane of the Minnesota Pollution Control Agency clearly believe that RTS should be banned and have worked hard to advocate that position. Mills (2000) noted that an inherent conflict of interest exists when agency staff, who should be the suppliers of policy‐neutral science, advocate for a preferred policy position. Although there may not always be a solid line between policy neutrality and advocacy, scientists must be “brutally honest” with decision makers and the public about both the results and inherent uncertainties of a science‐ based analysis (Lackey 2007). Problems arise when personal preference influences how research findings are presented. Since the first paper on this topic (Mahler et al. 2005), when the results of a local study were spun to suggest a nationwide issue, Van Metre, Mahler, and Crane have interpreted data in ways that support their policy preferences. Given the layperson’s unfamiliarity with receptor models, it is easy to overstate the importance of their results by ignoring problems with the selected source profiles and minimizing the uncertainty in the output. An example of the latter is quoting calculated source contributions to a tenth of a percent, even though the inputs have uncertainty of 20% to 40%.
Integr Environ Assess Manag 10, 2014—KT O'Reilly et al.
In conclusion, we agree with Crane that the forensic methods and receptor models discussed have limitations. But, unlike Van Metre, Mahler, and Crane, our goal was not to support a hypothesis by presenting limited results from application of a single method. Rather, the purpose of our manuscript was to discuss the strengths and weaknesses of the models, and given their limitations, to recommend how they should be applied to assist in setting source‐control policy. Acknowledgment—We have received support from the Pavement Coatings Technology Council. The opinions are those of the authors.
REFERENCES Coulter CT. 2004. EPA‐CMB 8.2 users' manual. EPA 452/R‐04‐011. Washington DC: US Environmental Protection Agency. Crane JL. 2014. Source apportionment and distribution of polycyclic aromatic hydrocarbons, risk considerations, and management implications for urban stormwater pond sediments in Minnesota, USA. Arch Environ Contam Toxicol 66:176–200. Halsall CJ, Barrie LA, Fellin P, Muir DCG, Billeck BN, Lockhart L, Rovinsky FY, Kononov EY, Pastukhov B. 1997. Spatial and temporal variation of polycyclic aromatic hydrocarbons in the Arctic atmosphere. Environ Sci Technol 31: 3593–3599. Johnson GW, Ehrlich R, Full W. 2004. Principal components analysis and receptor models in environmental forensics. In: Murphy BL, Morrison RD, editors. Introduction to environmental forensics. Burlington (MA): Elsevier Academic Press. 461 p. Lackey RT. 2007. Science, scientists and policy advocacy. Conserv Biol 21: 12–17. Li A, Jang J‐K, Scheff PA. 2003. Application of EPA CMB8.2 model for source apportionment of sediment PAHs in Lake Calumet, Chicago. Environ Sci Technol 37:2958–2965. Mahler BJ, Van Metre PC, Bashara TJ, Wilson JT, Johns DA. 2005. Parking lot sealcoat: An unrecognized source of urban polycyclic aromatic hydrocarbons. Environ Sci Technol 39:5560–5566. Mills TJ. 2000. Policy advocacy by scientists risks science credibility and may be unethical. Northwest Sci 74:165–168. O'Reilly K, Pietari J, Boehm P. 2012. Forensic assessment of refined tar‐based sealers as a source of polycyclic aromatic hydrocarbons (PAHs) in urban sediments. Environ Forensics 13:185–196. O'Reilly KT, Pietari J, Boehm PD. 2014a. Parsing pyrogenic polycyclic aromatic hydrocarbons: Forensic chemistry, receptor models, and source control policy. Integr Environ Assess Manag 10:279–285. Van Metre PC, Mahler BJ. 2010. Contribution of PAHs from coal–tar pavement sealcoat and other sources to 40 U.S. lakes. Sci Tot Environ 409: 334–344. Yang HH, Lai SO, Hsieh LT, Hsueh HJ, Chi T. 2002. Profiles of PAH emission from steel and iron industries. Chemosphere 48:1061–1074.
