Environ Monit Assess (2014) 186:2599–2608 DOI 10.1007/s10661-013-3563-y
Selecting the optimum plot size for a California design-based stream and wetland mapping program Leila G. Lackey & Eric D. Stein
Received: 17 July 2013 / Accepted: 19 November 2013 / Published online: 1 December 2013 # Springer Science+Business Media Dordrecht 2013
Abstract Accurate estimates of the extent and distribution of wetlands and streams are the foundation of wetland monitoring, management, restoration, and regulatory programs. Traditionally, these estimates have relied on comprehensive mapping. However, this approach is prohibitively resource-intensive over large areas, making it both impractical and statistically unreliable. Probabilistic (design-based) approaches to evaluating status and trends provide a more cost-effective alternative because, compared with comprehensive mapping, overall extent is inferred from mapping a statistically representative, randomly selected subset of the target area. In this type of design, the size of sample plots has a significant impact on program costs and on statistical precision and accuracy; however, no consensus exists on the appropriate plot size for remote monitoring of stream and wetland extent. This study utilized simulated sampling to assess the performance of four plot sizes (1, 4, 9, and 16 km2) for three geographic regions of California. Simulation results showed smaller plot sizes (1 and 4 km2) were most efficient for achieving desired levels of statistical accuracy and precision. However, larger plot sizes were more likely to contain rare and spatially limited wetland subtypes. Balancing L. G. Lackey : E. D. Stein (*) Southern California Coastal Water Research Project, 3535 Harbor Blvd., Suite 110, Costa Mesa, CA 92626, USA e-mail: [email protected] L. G. Lackey Environmental Science and Engineering, University of California, Los Angeles, Los Angeles, CA, USA
these considerations led to selection of 4 km2 for the California status and trends program. Keywords Generalized random tessellation stratified sampling . Simulated sampling . Aerial imagery cost . Wetland status and trends assessment
Introduction Wetland and stream mapping is the foundation for many regulatory, restoration, and management programs, including those that support state and federal no-net-loss policies (Mitsch and Gosselink 2000; Nusser and Goebel 1997) and inform decisions on compensatory mitigation (Baron et al. 2002; Clare, Krogman, Foote and Lemphers 2011). Accurate estimates of wetland and stream extent and distribution are also necessary to evaluate the effectiveness of programs and policies and to serve as base layers for ambient condition surveys. The predominant approach for evaluating stream and wetland extent (here referred to jointly as aquatic resources) is comprehensive inventory and mapping of all aquatic features—for example, this approach is used by the U.S. Fish and Wildlife Service in the National Wetland Inventory (NWI). However, comprehensive mapping is prohibitively expensive for large or complex areas. In contrast, design-based mapping uses a probabilistic approach to produce extent and trend estimates more frequently and at significantly lower costs by only mapping a statistically selected sample of the target area (Olsen and Peck
2600
2008). Under a design-based approach, a grid is laid out over the entire area of interest, and plots are selected at random and mapped. Then, the fraction of the target area covered by the aquatic resource of interest is estimated from the density of aquatic resource in the sampled plots (Albert et al. 2010; Gregoire 1999; Lackey and Stein 2013). This approach can be independent of the spatial distribution of aquatic resources and does not require a pre-existing map of aquatic resources. By mapping probabilistically, observations can be completed at a single point in time and repeated at regular intervals, enhancing ability to estimate status and trends (S&T). Existing examples of this approach include the National Wetland Inventory Status and Trends (NWI-S&T) monitoring program operated by the US Fish and Wildlife Service (Dahl 2011), a state-level program in Minnesota (MN-S&T) (Kloiber and Norris 2013), and the National Inventory of Landscapes in Sweden (NILS) (Ståhl et al. 2010). One design factor that significantly affects the cost and statistical accuracy of probabilistic surveys is the size of mapped plots. Smaller plots require less time and effort to map but are expected to have a higher sample variance than larger plots (Rossi 2004). However, while more smaller plots are required to achieve the same level of statistical precision, lower per-plot costs mean that sampling programs based on small plots may be more cost-effective overall. In contrast, individual larger plots have a higher probability of including aquatic resource types that are unevenly distributed across the landscape. As a result, while a sample based on small plots may be the most efficient mechanism to achieve target levels of statistical precision, this estimate may over-represent more common and evenly distributed stream and wetland types. Only one of the existing S&T programs, the MNS&T program, systematically evaluated plot sizes. The NWI-S&T and NILS programs use a 10.36 km2 (4 mi2) and a 1 km2 plot size, respectively, but these designs were not selected based on a careful examination of the areas in question (Dahl 2011; Ståhl et al. 2010). The MN-S&T program determined the appropriate plot size by artificially nesting smaller plot sizes inside the 10.36 km2 (4 mi2) NWI-S&T plots located within Minnesota (Deegan and Aunan 2006). Then, program designers estimated the sample variance and minimum required sample size as a function of plot size. This analysis found that 2.59 km2 (1 mi2) plots provided the best balance of statistical precision and cost-
Environ Monit Assess (2014) 186:2599–2608
efficiency; program cost estimates based on larger plot sizes were more expensive due to higher costs per plot, and estimates based on smaller plot sizes were more expensive due to larger required sample sizes. The example of the MN-S&T program highlights the potential benefits of plot size optimization for S&T programs. By determining that a 2.59 km2 plot was preferred for this program, per-plot mapping costs were one quarter of the per-plot costs for the NWI-S&T while still maximizing statistical precision. However, the ability to generalize these findings to other areas, such as California, is uncertain for a number of reasons. First, Minnesota has a higher mean wetland density, particularly for palustrine and lacustrine wetland types (Kloiber and Norris 2013). Second, wetlands in arid and semi-arid landscapes, such as California, are more unevenly distributed across the landscape. Finally, the goal of the California S&T program is to include both streams and wetlands, unlike the Minnesota program, which includes only wetlands. The different spatial distribution of streams and wetlands adds an additional factor to the consideration of optimum plot size. The objective of this study was to identify the plot size for monitoring stream and wetland extent in California that would balance statistical precision, program costs, and ability to monitor both evenly distributed streams and rare or unevenly distributed wetlands. This study used simulated sampling to evaluate different plot sizes for estimating aquatic resource extent. Results compared sample precision against estimated survey costs to select an appropriate plot size for a California S&T program. Specifically, we aim to answer the question of which plot size costeffectively meets program objectives for sample accuracy and precision? Although the study focused on California, the approach and results, combined with previous results from the MN analysis, will be helpful to programs in other heterogeneous landscapes attempting to make decisions about mapped plot size for design-based efforts.
Materials and methods General approach We utilized simulated sampling to evaluate the performance of various plot sizes for estimating stream and
Environ Monit Assess (2014) 186:2599–2608
wetland extent and distribution. Simulated sampling repeatedly samples from an existing dataset to model results based on different design options, allowing evaluation of the statistical accuracy and precision of each plot size option for estimating mean aquatic resource density. The cost of each option was then compared with its statistical performance in order to recommend an optimum plot size for the California S&T program.
2601
Bay Area wetland maps are viewable online through the California Wetlands Portal,1 and geodatabases were obtained directly from the map producer. Central Coast wetland maps are viewable and available for download as part of the NWI.2 We obtained South Coast maps directly from the producer, but these maps will be made publicly available through the NWI at a future date. Simulated sampling
Study areas We simulated sampling in three representative areas of California, the San Francisco Bay Area, the Central Coast Region, and the South Coast Region (Fig. 1). These areas were selected based on the availability of pre-existing, high-quality, contemporary maps of aquatic resources. Multiple areas were utilized to better represent the diversity of landscapes and aquatic resource types present in California. All wetland maps were produced using 1-m 2005 National Agriculture Imagery Program (NAIP) imagery as the primary source, and all were mapped using equivalent protocols. Unfortunately, the only statewide data source, the NWI, was not considered suitable for this study. The NWI covers approximately two thirds of the state and was produced between the 1970s and the 2000s. Comparability of maps produced at different points in time is limited by evolution in mapping techniques and protocols and by increased availability of highresolution, color, and infrared imagery.
Four continuous grids were produced for each study area with grid cell (plot) sizes of 1, 4, 9, and 16 km2. These sizes cover the range of plot sizes used by existing S&T programs. The grids were clipped to the boundaries of the respective study areas and then intersected with the aquatic resource maps. Finally, we determined area density of aquatic resources (streams and wetlands) for each grid cell. For each grid size, we simulated ten different plot sample sizes (40 to 400, by 40, for the 1 km2 grids; 30 to 300, by 30, for the 4 km2 grids; 20 to 200, by 20, for the 9 km2 grids; and 10 to 100, by 10, for the 16 km2 grids). We simulated sampling in R using the spsurvey package (Kincaid and Olsen 2011; R Development Core Team 2011). We utilized generalized random tessellation stratified (GRTS) sampling (Stevens and Olsen 2003, 2004). Previous work has established unstratified GRTS sampling is more reliable than stratified or simple random sampling designs for monitoring wetland and stream extent (Lackey and Stein 2013). Simulations were repeated 5,000 times to produce an empirical distribution of four sample parameters calculated by the spsurvey package and the grts function: the sample mean, standard error of the sample mean, and upper and lower 95 % confidence interval bounds of the sample mean. Based on the work of Stevens and Olsen (2003), the grts function utilizes an alternate estimate of sample variance with a narrower confidence interval. Sampling performance We used three metrics to assess simulated sampling performance. First, we determined the fraction of sample plots that contained any aquatic resources and the fraction that contained each of four aquatic resource subtypes: estuarine and marine wetlands, lacustrine 1
Fig. 1 Extent of three study areas
2
www.californiawetlands.net/tracker/ba/map http://www.fws.gov/wetlands/Data/Data-Download.html
2602
wetlands, palustrine wetlands, and riverine wetlands. This metric allowed us to determine which sampling design provides the greatest coverage for rarer or spatially limited wetland subtypes. By area, approximately three-fourths of mapped riverine wetlands would be considered buffered streamlines while approximately one-fourth are in or near-channel riverine wetland. This approximation is based on differences between wetland maps produced with linear streamlines and maps produced with two-dimensional streambeds. Estuarine, marine, and lacustrine wetlands are relatively rare and are constrained to the costal boundary of each study area while lacustrine wetlands are more evenly distributed. Palustrine and riverine wetlands are significantly more common, but palustrine wetlands are less evenly distributed than riverine. Second, for each simulated sampling repetition, statistical accuracy was assessed by recording the 95 % confidence interval of the sample mean and determining whether it contained the true mean density for the study area (defined by dividing the total aquatic resource area by the total area of the study region). After 5,000 repetitions, the overall accuracy of the sample confidence intervals was determined by dividing the number of intervals that contained the true mean by 5,000. The result indicates the likelihood that a given plot size can produce a statistically reliable estimate of aquatic resource density. Expected accuracy for the 95 % confidence interval was 0.95. Finally, statistical precision was assessed for each repetition as half the width of the 95 % confidence interval of the mean, divided by the sample mean. After 5,000 repetitions, the average sample error was then calculated. This metric indicated the statistical precision of sample estimates, important for detecting changes in aquatic resource density over time. Estimated survey costs Estimated program costs considered imagery acquisition and map production. We developed cost estimates through expert elicitation. Estimates were further supported by cost estimates obtained by the California Natural Resources Agency while developing the State S&T program. For imagery costs, we considered both no-cost imagery from the NAIP and contracted imagery acquisition. NAIP imagery is available from the U.S. Department of Agriculture and is currently available free of charge. Contracted imagery costs were based
Environ Monit Assess (2014) 186:2599–2608
on estimates provided by a local aerial photography company that has experience producing imagery for wetland mapping. Map production costs were estimated by two groups that routinely produce wetland maps for state and federal resource and regulatory programs.
Results and discussion Estimated costs: imagery and mapping Estimated costs for S&T programs were primarily driven by two components: imagery and map production. Imagery costs could include no-cost imagery from publicly available sources, such as NAIP, or newly acquired imagery. NAIP imagery is commonly used for aquatic resource map production and meets all of the technical requirements including 1-m resolution and four-band, color plus IR (Dahl 2011). However, use of NAIP or other freely available imagery sources limits the season of acquisition. For example, NAIP imagery is typically acquired between June and August. In Southern California, the limited rainy season puts the ideal timeframe for imagery acquisition for wetland mapping in June or even earlier. In addition, while NAIP has recently been acquired every 1 to 3 years, future acquisition frequency and image details would be outside the control of the S&T program managers. Estimates of costs to acquire new imagery were provided by a private company with substantial experience in aerial image acquisition for scientific surveys. Estimates were also consistent with those obtained by the California Natural Resources Agency while developing budget estimates for the State program. Based on these consultations, we estimated that 1-m or better resolution imagery for the plot sizes considered in this study could be acquired through single-pass acquisition at a rate of $150 to $450 per plot, depending on remoteness. Plot size did not affect cost estimates as all sizes considered could be captured with a single image and aerial pass. Map production costs were developed by two groups with many years of experience mapping aquatic resources for academic, government, and non-profit groups. The two groups reviewed their labor and contract records to produce estimates of time and salary costs and independently arrived at a mapping cost of $25 per km2. Both estimates considered all segments of map production including source imagery and data
Environ Monit Assess (2014) 186:2599–2608
management, streamline modeling and polygon production, classification of aquatic resources, and quality assurance and quality control. Mappers indicated that mapping costs would scale linearly with plot size. In addition, neither group believed that cost would be influenced by the total number of plots, that is, there would be no additional costs associated with stopping and starting mapping to produce aquatic resource maps for a greater number of discrete locations. We acknowledge that the above cost estimates are based on a limited survey of image acquisition and wetland mapping professionals. However, we believe that the results are still relevant for evaluating the statistical efficiency of different plot sizes. The three image acquisition costs levels—$0, $150, and $450—were (as presented below) sufficiently divergent to produce separation in results between cost levels. In addition, map production costs are a function of area mapped and therefore reflect the relative difference in area more than the cost of mapping each plot. Changing map production costs would not have an impact on the relative cost to produce sample plot maps, i.e., a 4 km2 plot will always be four times as costly to map as a 1 km2 plot. Finally, additional support for the estimates has come from the California Natural Resources Agency, as part of budget estimation for a California S&T program.
Study regions: density and wetland subtypes The three study regions had different density and diversity of wetland resources, allowing us to evaluate the statistical accuracy and precision and cost effectiveness of different plot sizes under a range of conditions. The Bay Area region had the highest wetland density overall, 0.17 km2 wetland per 1 km2 of area, due to the significant estuarine and marine wetland area contributed by the San Francisco Bay—75 % of the total wetland area or a total of 1,500 km2. The remaining wetland area in this region was 67 % palustrine, 16 % riverine, and 13 % lacustrine. The Central and South Coast regions had significantly lower overall densities, 0.035 and 0.053 km2 km−2, respectively, dominated in almost equal parts by palustrine and riverine wetlands (42 % and 37 % in the Central Coast and 38 % and 36 % in the South Coast). Lacustrine wetlands accounted for 14 % of wetland area in both of these regions, and estuarine and marine wetlands made up the remaining 7 % and 10 % in the Central and South Coast, respectively.
2603
Wetland density in the Central Coast study region was the closest match to other regions of California based on estimates from NWI maps prepared within the last decade. Recent NWI maps suggest that wetland density in California is 0.032 km2 km−2, closest to the density of the Central Coast region, and comprised of 38 % palustrine, 34 % riverine, 21 % lacustrine, and 8 % estuarine and marine, similar to the Central and South Coast regions. In contrast, for maps produced more than a decade ago, statewide density estimates are 0.053 km2 km−2, closest to the South Coast region, comprised of 44 % palustrine, 41 % lacustrine, 10 % estuarine and marine, and 5 % riverine. The significant difference in the portion of wetlands mapped as riverine is likely due to a change in conventional wetland mapping protocols. Prior to the most recent decade, mapped streamlines were reserved as one-dimensional features. In contrast, current mapping protocols convert linear streamlines into two-dimensional approximations of streambed width.
Plot size: resource subtypes and inclusion frequency The difference in wetland subtype densities between the three regions also allowed us to examine the relationship between plot size and the likelihood sampled plots would contain wetland types of interest. Larger plots were more likely to contain each resource subtype, and the fraction of plots that contained certain subtypes decreased significantly between a 4- and 1-km2 plot size (Fig. 2), suggesting a potential minimum desirable plot size of 4 km2. In each study area, close to 100 % of plots contained wetlands, even for the smallest plot size considered. Mapped streamlines and riverine wetlands, which tend to be uniformly distributed, were present in more than 80 % of plots, regardless of plot size. However, for palustrine wetlands, which are less uniformly distributed, the fraction of plots drops from 90 % to 100 % for the largest plot size, 16 km2, to approximately 70 % for 1 km2 plots in the Central and South Coast, and less than 50 % for the Bay Area. The largest decrease in palustrine frequency occurred between the 4 and 1 km2 plot size—dropping from 88 % to 91 % to 70 % in the Central and South Coast and from 74 % to 47 % in the Bay Area. For the less-frequent wetland types, prevalence decreased by approximately half between each plot size.
2604
Environ Monit Assess (2014) 186:2599–2608
Plot size: statistical accuracy and precision
Fig. 2 Fraction of plots with aquatic resources, estuarine or marine, lacustrine, palustrine, and riverine subtypes for 1, 4, 9, and 16 km2 plot sizes in the a Bay Area, b Central Coast, and c South Coast study areas
For all three study regions, the 1 km2 plot size had the highest statistical accuracy for a given estimated cost when no-cost imagery was assumed (Fig. 3). For example, assuming no-cost imagery and a total cost of $10,000 in the Central Coast, accuracy for a 1 km2 plot size was 0.90 versus 0.69 for a 16 km2 plot size (Table 1). In addition, a 1 km2 plot size was also generally the most efficient mechanism for achieving a statistical accuracy of at least 0.90, although differences were less pronounced for the Bay Area (Table 2). If we assume that contract imagery must be purchased, differences between plot sizes for the efficiency versus statistical accuracy relationship were much smaller (Tables 1 and 2). Under these circumstances, the 4 km2 plot size often had the best statistical accuracy and/or was the most cost-efficient. Smaller plots were also more efficient for achieving a narrower calculated confidence interval and lower mean error rate (Fig. 4). The difference in statistical precision between the 1 and 16 km2 plot sizes was largest for the Bay Area. For example, assuming no-cost imagery and a $10,000 total cost, a 1 km2 plot size was associated with an 11 % error rate while a 16 km2 plot size was associated with a 70 % error rate (Table 1). Similar assumptions in the Central and South Coast had error rates of 16–17 % for a 1 km2 plot size and 40–41 % for a 16 km2 plot size. Differences in unit error rates between plot sizes decreased when contract imagery was assumed (Table 1). Under the contracted imagery scenarios, a 4 km2 plot size became the most efficient choice for achieving a 25 % error rate (Table 2). Of note related to sample accuracy was the “underperformance” of sample results related to expected accuracy in the Central and South Coast study areas (Fig. 2). For these two regions, the accuracy of the 95 % confidence interval did not reach 0.95 for any of the plot sizes or sample sizes tested. Instead, accuracy plateaued at approximately 0.90, indicating that only 90 % of the 95 % confidence intervals contained the true mean. In contrast, in the Bay Area, which also had a significantly higher aquatic resource density than the Central or South Coast (0.17 km2 km−2 vs. 0.04 and 0.05 in the Central and South Coast, respectively), accuracy was greater than 0.95 for all but a few of the smallest simulated sample sizes. We considered a number of factors in an effort to explain the lower than expected accuracy for the Central
Environ Monit Assess (2014) 186:2599–2608
2605
with values bounded by zero and one—did not reduce this discrepancy (results not shown). Second, the variance estimator used under GRTS sampling produces a narrower confidence interval than the more commonly used estimator of sample variance (Stevens and Olsen 2003). However, because reduced accuracy was only observed in two of the three regions, we do not believe that the GRTS variance estimator is driving the discrepancy. Therefore, we believe that the discrepancy may be related to the low mean aquatic resource density in those regions (0.03 and 0.05 km2 km−2, respectively) compared with the Bay Area (0.17 km2 km−2). Others have identified the inadequacy of the sample variance estimator when small sample sizes are utilized (Kupper and Hafner 1989; Liu 2009). Statewide generalization
Fig. 3 Accuracy versus estimated costs assuming no-cost imagery for 1 (circle), 4 (triangle), 9 (plus), and 16 (cross)km2 plot sizes in the a Bay Area, b Central Coast, and c South Coast study areas
and South Coast. First, transformation of the population using the lognormal and the arcsine transformations— often used for right-tailed populations and populations
Small plot sizes (1 or 4 km2) were more efficient than large plot sizes (9 or 16 km2) for measuring aquatic resource extent in California. Differences between plot sizes were smaller when contract imagery was considered than when no-cost imagery was assumed (Figs. 3 and 4). Independent of plot size, regions with low average wetland density (0.03–0.05 km2 km−2) were associated with lower statistical accuracy and higher error rates than the region with high average wetland density (0.17 km2 km−2). In addition, cost differences between plot sizes were smaller for low-density regions compared with high-density regions when statistical precision was considered. When statistical accuracy was considered, cost differences were larger for lower-density regions. Finally, 4 km2 plots where preferred over 1 km2 plots when contract imagery was assumed, due to the interaction between plot size and sample variance. In general, we do not consider cost efficiency to be independent of statistical accuracy and precision in the design of a long-term monitoring program. Our objective was to determine the plot size that most efficiently satisfied requirements for statistical accuracy and precision. Removing cost from the considerations would be the same as implicitly assuming production costs were equal for all plot sizes. In our opinion, this assumption does not produce an accurate assessment of program efficiency. In all study areas, larger plots were more accurate and less variable than smaller plots but were significantly less efficient to map (Figs. 3 and 4). Fewer plots were necessary to achieve the same level of accuracy and precision, but because larger plots were also
2606 Table 1 Accuracy and error rates for a given survey cost
Environ Monit Assess (2014) 186:2599–2608
Plot size
No-cost imagery ($10,000)
Low-cost imagery ($50,000)
High-cost imagery ($75,000)
Accuracy
Error
Accuracy
Error
Accuracy
Error
1 km2
90 %
17 %
89 %
21 %
87 %
27 %
4 km2
85 %
29 %
89 %
19 %
88 %
23 %
9 km2
78 %
36 %
89 %
21 %
88 %
23 %
16 km2
69 %
41 %
85 %
25 %
85 %
25 %
1 km2
90 %
16 %
90 %
20 %
89 %
26 %
4 km2
86 %
28 %
91 %
19 %
90 %
22 %
9 km2
81 %
36 %
90 %
21 %
90 %
22 %
16 km2
76 %
40 %
89 %
22 %
89 %
22 %
1 km2
94 %
11 %
96 %
14 %
95 %
20 %
4 km2
96 %
28 %
96 %
16 %
96 %
20 %
9 km2
93 %
47 %
96 %
20 %
95 %
23 %
16 km2
91 %
70 %
97 %
27 %
97 %
27 %
Central Coast
South Coast
Bay Area
difference in mapping costs significantly influenced estimation of total costs, and the 1 km2 plot size remained the most cost-effective. In contrast, as-
more expensive to map, smaller plots were the most efficient mechanism for optimizing statistical accuracy and precision. When no-cost imagery was assumed, the
Table 2 Costs to achieve 90 % accuracy (95 % in the Bay Area) or a 25 % error rate
Plot size
No-cost imagery
Low-cost imagery
High-cost imagery
Accuracy
Error
Accuracy
Accuracy
1 km2
$8,000
$5,000
$56,000
$35,000
$152,000
$95,000
4 km2
$24,000
$12,000
$60,000
$30,000
$132,000
$66,000
$40,500
$22,500
$67,500
$37,500
$121,500
$67,500
>$40,000
$36,000
>$55,000
$49,500
>$85,000
$76,500
1 km2
$5,000
$4,000
$35,000
$28,000
$85,000
$76,000
4 km2
$15,000
$12,000
$37,500
$30,000
$82,500
$66,000
$27,000
$22,500
$45,000
$37,500
$81,000
$67,500
>$40,000
$28,000
>$55,000
$38,500
>$85,000
$59,500
1 km2
$5,000
$3,000
$35,000
$21,000
$95,000
$57,000
4 km2
$6,000
$12,000
$15,000
$30,000
$33,000
$66,000
$13,500
$22,500
$22,500
$37,500
$40,500
$67,500
$20,000
$40,000
$27,500
$55,000
$42,500
$85,000
Error
Error
Central Coast
9 km2 2
16 km
South Coast
9 km2 2
16 km
Bay Area
9 km2 2
16 km
Environ Monit Assess (2014) 186:2599–2608
Fig. 4 Error versus estimated costs assuming no-cost imagery for 1 (circle), 4 (triangle), 9 (plus), and 16 (cross)km2 plot sizes in the a Bay Area, b Central Coast, and c South Coast study areas
2607
sumption of contract imagery decreased the effect of plot size on survey costs and, in the case of 1 and 4 km2 plot sizes, meant that the 4 km2 plot became more cost-effective. The 4-km2 plots were also preferred over 1-km2 plots with regard to the diversity of aquatic resource types included in each sample plot (Fig. 2). Primary design considerations for the California S&T program are the cost considerations associated with achieving statistical accuracy and precision. Secondary design considerations for a large-scale S&T program include use of maps as a sample frame for ambient environmental field assessments. Less than 10 % of California has aquatic resource maps produced within the past decade; therefore, for large areas of the state, the S&T plot maps will represent the best-available aquatic resource maps. Therefore, larger plot sizes may provide better landscape context and elucidation of wetland complexes or specific resource subtypes. In addition, in the event that S&T plots show a gain or loss in aquatic resource area, proximal landscape elements, such as agricultural land use or urban or suburban development, within the mapped area can support hypothesis generation to help explain the drivers of gains or losses. Larger plots will also be better equipped to capture these interactions. Based on the balance of considerations—cost efficiency for achieving statistical accuracy and precision, capturing a diversity of aquatic resource and landscape types, and supporting hypothesis generation related to area gains and losses—a 4 km2 plot size optimized the objectives for development of the California S&T program. The work presented here is the first simulation study on the tradeoffs of appropriate plot size for monitoring the spatial extent of wetlands in a design-based program. Existing S&T programs utilize plot sizes between 1 km2, for the NILS, and 10.36 km2 (4 mi2), for the NWI-S&T (Dahl 2011; Ståhl et al. 2010). Neither program reported a cost–benefit analysis supporting their choice of plot size. The MN-S&T program (2.59 km2, 1 mi2) did consider the sample variance and estimated cost for different plot sizes but confined their analysis to a limited subset of the state (Deegan and Aunan 2006). In addition, the Minnesota analysis was limited to standard sample size calculations, focused on wetland density as opposed to streams and wetlands, and did not consider the empirical variability of sample estimates through simulation.
2608
Conclusions Overall, small plot sizes (1 or 4 km2) were more efficient for statistical accuracy and precision than larger plot sizes (9 or 16 km2) for the study areas considered; however, larger plot sizes contained a greater diversity of rare or geographically constrained wetland types such as estuarine, lacustrine, and marine wetlands. Balancing these considerations for the California S&T program led to selection of the 4 km2 plot size. The 4 km2 plot size also allows for more robust optimization, regardless of whether imagery is free or must be purchased. These results and tradeoffs should transfer well to other regional, tribal, or state programs using design-based methods to assess the extent and distribution of aquatic resources in heterogeneous landscapes. The approach presented here can also be easily replicated for programs that may want to conduct their own location-specific optimization. Acknowledgments The funding for this study was provided by the US Environmental Protection Agency (Grant CD-00 T22301) through the California Natural Resources Agency. Information vital to the cost analysis was graciously provided by Shawna Dark and Patricia Pendleton of California State University, Northridge; Kristen Cayce of the San Francisco Estuary Institute; and Nick Arentz of Skyview Aerial Photo, Inc., Murrieta, CA. Guiding input on analysis and interpretation was provided by the Technical Advisory Committee to the project, particularly Kerry Ritter of the Southern California Coastal Water Research Project, Steve Kloiber of the Minnesota Department of Natural Resources, and Paul Jones of the US Environmental Protection Agency; and by the dissertation committee to Dr. Lackey at the University of California, Los Angeles, particularly Richard Ambrose and Mark Handcock.
References Albert, C. H., Yoccoz, N. G., Edwards, T. C., Graham, C. H., Zimmermann, N. E., & Thuiller, W. (2010). Sampling in ecology and evolution—Bridging the gap between theory and practice. Ecography, 33(6), 1028–1037. doi:10.1111/j. 1600-0587.2010.06421.x. Baron, J. S., Poff, N. L., Angermeier, P. L., Dahm, C. N., Gleick, P. H., Hairston, N. G., et al. (2002). Meeting ecological and societal needs for freshwater. Ecological Applications, 12(5), 1247–1260. doi:10.1890/1051-0761(2002)012[1247: MEASNF]2.0.CO;2. Clare, S., Krogman, N., Foote, L., & Lemphers, N. (2011). Where is the avoidance in the implementation of wetland law and policy? Wetlands Ecology and Management, 19(2), 165–182. doi:10.1007/s11273-011-9209-3.
Environ Monit Assess (2014) 186:2599–2608 Dahl, T. E. (2011). Status and trends of wetlands in the conterminous United States 2004 to 2009. US Department of the Interior, Fish and Wildlife Service. Deegan, G., & Aunan, T. (2006). Survey design factors for wetland monitoring. Minnesota Department of Natural Resources, Forestry/Resource Assessment. Gregoire, T. G. (1999). Design-based and model-based inference in survey sampling: Appreciating the difference. Canadian Journal of Forest Research, 28, 1429–1447. Kincaid, T. M., & Olsen, A. R. (2011). spsurvey: Spatial survey design and analysis; Vienna, Austria: R Foundation for Statistical Computing Kloiber, S. M., & Norris, D. J. (2013). Status and trends of wetlands in Minnesota: Wetland quantity trends from 2006 to 2011. St. Paul, MN; Minnesota Department of Natural Resources. Kupper, L. L., & Hafner, K. B. (1989). How appropriate are popular sample size formulas? The American Statistician, 43(2), 101–105. doi:10.1080/00031305. 1989.10475628. Lackey, L. G., & Stein, E. D. (2013). Design-based sampling for aquatic resource extent monitoring in California. Wetlands. Liu, X. S. (2009). Sample size and the width of the confidence interval for mean difference. British Journal of Mathematical and Statistical Psychology, 62(Pt 2), 201–215. doi:10.1348/ 000711008X276774. Mitsch, W. J., & Gosselink, J. G. (2000). The value of wetlands: Importance of scale and landscape setting. Ecological Economics, 35(1), 25–33. doi:10.1016/S0921-8009(00) 00165-8. Nusser, S., & Goebel, J. (1997). The national resources inventory: A long-term multi-resource monitoring programme. Environmental and Ecological Statistics, 4(3), 181–204. Olsen, A. R., & Peck, D. V. (2008). Survey design and extent estimates for the Wadeable Streams Assessment. Journal of the North American Benthological Society, 27(4), 822–836. doi:10.1899/08-050.1. R Development Core Team. (2011). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.Rproject.org. Rossi, J. (2004). The effect of sampling unit size on the perception of the spatial pattern of earthworm (Lumbricus terrestris L.) middens. Applied Soil Ecology, 27(2), 189–196. doi:10.1016/ j.apsoil.2004.03.001. Ståhl, G., Allard, A., Esseen, P.-A., Glimskär, A., Ringvall, A., Svensson, J., et al. (2010). National Inventory of Landscapes in Sweden (NILS)—Scope, design, and experiences from establishing a multiscale biodiversity monitoring system. Environmental Monitoring and Assessment, 173(1–4), 579– 595. doi:10.1007/s10661-010-1406-7. Stevens, D. L., & Olsen, A. R. (2003). Variance estimation for spatially balanced samples of environmental resources. Environmetrics, 14(6), 593–610. doi:10.1002/env.606. Stevens, D. L., & Olsen, A. R. (2004). Spatially balanced sampling of natural resources. Journal of the American Statistical Association, 99(465), 262–278. doi:10.1198/0162145 04000000250.
Selecting the optimum plot size for a California design-based stream and wetland mapping program Leila G. Lackey & Eric D. Stein
Received: 17 July 2013 / Accepted: 19 November 2013 / Published online: 1 December 2013 # Springer Science+Business Media Dordrecht 2013
Abstract Accurate estimates of the extent and distribution of wetlands and streams are the foundation of wetland monitoring, management, restoration, and regulatory programs. Traditionally, these estimates have relied on comprehensive mapping. However, this approach is prohibitively resource-intensive over large areas, making it both impractical and statistically unreliable. Probabilistic (design-based) approaches to evaluating status and trends provide a more cost-effective alternative because, compared with comprehensive mapping, overall extent is inferred from mapping a statistically representative, randomly selected subset of the target area. In this type of design, the size of sample plots has a significant impact on program costs and on statistical precision and accuracy; however, no consensus exists on the appropriate plot size for remote monitoring of stream and wetland extent. This study utilized simulated sampling to assess the performance of four plot sizes (1, 4, 9, and 16 km2) for three geographic regions of California. Simulation results showed smaller plot sizes (1 and 4 km2) were most efficient for achieving desired levels of statistical accuracy and precision. However, larger plot sizes were more likely to contain rare and spatially limited wetland subtypes. Balancing L. G. Lackey : E. D. Stein (*) Southern California Coastal Water Research Project, 3535 Harbor Blvd., Suite 110, Costa Mesa, CA 92626, USA e-mail: [email protected] L. G. Lackey Environmental Science and Engineering, University of California, Los Angeles, Los Angeles, CA, USA
these considerations led to selection of 4 km2 for the California status and trends program. Keywords Generalized random tessellation stratified sampling . Simulated sampling . Aerial imagery cost . Wetland status and trends assessment
Introduction Wetland and stream mapping is the foundation for many regulatory, restoration, and management programs, including those that support state and federal no-net-loss policies (Mitsch and Gosselink 2000; Nusser and Goebel 1997) and inform decisions on compensatory mitigation (Baron et al. 2002; Clare, Krogman, Foote and Lemphers 2011). Accurate estimates of wetland and stream extent and distribution are also necessary to evaluate the effectiveness of programs and policies and to serve as base layers for ambient condition surveys. The predominant approach for evaluating stream and wetland extent (here referred to jointly as aquatic resources) is comprehensive inventory and mapping of all aquatic features—for example, this approach is used by the U.S. Fish and Wildlife Service in the National Wetland Inventory (NWI). However, comprehensive mapping is prohibitively expensive for large or complex areas. In contrast, design-based mapping uses a probabilistic approach to produce extent and trend estimates more frequently and at significantly lower costs by only mapping a statistically selected sample of the target area (Olsen and Peck
2600
2008). Under a design-based approach, a grid is laid out over the entire area of interest, and plots are selected at random and mapped. Then, the fraction of the target area covered by the aquatic resource of interest is estimated from the density of aquatic resource in the sampled plots (Albert et al. 2010; Gregoire 1999; Lackey and Stein 2013). This approach can be independent of the spatial distribution of aquatic resources and does not require a pre-existing map of aquatic resources. By mapping probabilistically, observations can be completed at a single point in time and repeated at regular intervals, enhancing ability to estimate status and trends (S&T). Existing examples of this approach include the National Wetland Inventory Status and Trends (NWI-S&T) monitoring program operated by the US Fish and Wildlife Service (Dahl 2011), a state-level program in Minnesota (MN-S&T) (Kloiber and Norris 2013), and the National Inventory of Landscapes in Sweden (NILS) (Ståhl et al. 2010). One design factor that significantly affects the cost and statistical accuracy of probabilistic surveys is the size of mapped plots. Smaller plots require less time and effort to map but are expected to have a higher sample variance than larger plots (Rossi 2004). However, while more smaller plots are required to achieve the same level of statistical precision, lower per-plot costs mean that sampling programs based on small plots may be more cost-effective overall. In contrast, individual larger plots have a higher probability of including aquatic resource types that are unevenly distributed across the landscape. As a result, while a sample based on small plots may be the most efficient mechanism to achieve target levels of statistical precision, this estimate may over-represent more common and evenly distributed stream and wetland types. Only one of the existing S&T programs, the MNS&T program, systematically evaluated plot sizes. The NWI-S&T and NILS programs use a 10.36 km2 (4 mi2) and a 1 km2 plot size, respectively, but these designs were not selected based on a careful examination of the areas in question (Dahl 2011; Ståhl et al. 2010). The MN-S&T program determined the appropriate plot size by artificially nesting smaller plot sizes inside the 10.36 km2 (4 mi2) NWI-S&T plots located within Minnesota (Deegan and Aunan 2006). Then, program designers estimated the sample variance and minimum required sample size as a function of plot size. This analysis found that 2.59 km2 (1 mi2) plots provided the best balance of statistical precision and cost-
Environ Monit Assess (2014) 186:2599–2608
efficiency; program cost estimates based on larger plot sizes were more expensive due to higher costs per plot, and estimates based on smaller plot sizes were more expensive due to larger required sample sizes. The example of the MN-S&T program highlights the potential benefits of plot size optimization for S&T programs. By determining that a 2.59 km2 plot was preferred for this program, per-plot mapping costs were one quarter of the per-plot costs for the NWI-S&T while still maximizing statistical precision. However, the ability to generalize these findings to other areas, such as California, is uncertain for a number of reasons. First, Minnesota has a higher mean wetland density, particularly for palustrine and lacustrine wetland types (Kloiber and Norris 2013). Second, wetlands in arid and semi-arid landscapes, such as California, are more unevenly distributed across the landscape. Finally, the goal of the California S&T program is to include both streams and wetlands, unlike the Minnesota program, which includes only wetlands. The different spatial distribution of streams and wetlands adds an additional factor to the consideration of optimum plot size. The objective of this study was to identify the plot size for monitoring stream and wetland extent in California that would balance statistical precision, program costs, and ability to monitor both evenly distributed streams and rare or unevenly distributed wetlands. This study used simulated sampling to evaluate different plot sizes for estimating aquatic resource extent. Results compared sample precision against estimated survey costs to select an appropriate plot size for a California S&T program. Specifically, we aim to answer the question of which plot size costeffectively meets program objectives for sample accuracy and precision? Although the study focused on California, the approach and results, combined with previous results from the MN analysis, will be helpful to programs in other heterogeneous landscapes attempting to make decisions about mapped plot size for design-based efforts.
Materials and methods General approach We utilized simulated sampling to evaluate the performance of various plot sizes for estimating stream and
Environ Monit Assess (2014) 186:2599–2608
wetland extent and distribution. Simulated sampling repeatedly samples from an existing dataset to model results based on different design options, allowing evaluation of the statistical accuracy and precision of each plot size option for estimating mean aquatic resource density. The cost of each option was then compared with its statistical performance in order to recommend an optimum plot size for the California S&T program.
2601
Bay Area wetland maps are viewable online through the California Wetlands Portal,1 and geodatabases were obtained directly from the map producer. Central Coast wetland maps are viewable and available for download as part of the NWI.2 We obtained South Coast maps directly from the producer, but these maps will be made publicly available through the NWI at a future date. Simulated sampling
Study areas We simulated sampling in three representative areas of California, the San Francisco Bay Area, the Central Coast Region, and the South Coast Region (Fig. 1). These areas were selected based on the availability of pre-existing, high-quality, contemporary maps of aquatic resources. Multiple areas were utilized to better represent the diversity of landscapes and aquatic resource types present in California. All wetland maps were produced using 1-m 2005 National Agriculture Imagery Program (NAIP) imagery as the primary source, and all were mapped using equivalent protocols. Unfortunately, the only statewide data source, the NWI, was not considered suitable for this study. The NWI covers approximately two thirds of the state and was produced between the 1970s and the 2000s. Comparability of maps produced at different points in time is limited by evolution in mapping techniques and protocols and by increased availability of highresolution, color, and infrared imagery.
Four continuous grids were produced for each study area with grid cell (plot) sizes of 1, 4, 9, and 16 km2. These sizes cover the range of plot sizes used by existing S&T programs. The grids were clipped to the boundaries of the respective study areas and then intersected with the aquatic resource maps. Finally, we determined area density of aquatic resources (streams and wetlands) for each grid cell. For each grid size, we simulated ten different plot sample sizes (40 to 400, by 40, for the 1 km2 grids; 30 to 300, by 30, for the 4 km2 grids; 20 to 200, by 20, for the 9 km2 grids; and 10 to 100, by 10, for the 16 km2 grids). We simulated sampling in R using the spsurvey package (Kincaid and Olsen 2011; R Development Core Team 2011). We utilized generalized random tessellation stratified (GRTS) sampling (Stevens and Olsen 2003, 2004). Previous work has established unstratified GRTS sampling is more reliable than stratified or simple random sampling designs for monitoring wetland and stream extent (Lackey and Stein 2013). Simulations were repeated 5,000 times to produce an empirical distribution of four sample parameters calculated by the spsurvey package and the grts function: the sample mean, standard error of the sample mean, and upper and lower 95 % confidence interval bounds of the sample mean. Based on the work of Stevens and Olsen (2003), the grts function utilizes an alternate estimate of sample variance with a narrower confidence interval. Sampling performance We used three metrics to assess simulated sampling performance. First, we determined the fraction of sample plots that contained any aquatic resources and the fraction that contained each of four aquatic resource subtypes: estuarine and marine wetlands, lacustrine 1
Fig. 1 Extent of three study areas
2
www.californiawetlands.net/tracker/ba/map http://www.fws.gov/wetlands/Data/Data-Download.html
2602
wetlands, palustrine wetlands, and riverine wetlands. This metric allowed us to determine which sampling design provides the greatest coverage for rarer or spatially limited wetland subtypes. By area, approximately three-fourths of mapped riverine wetlands would be considered buffered streamlines while approximately one-fourth are in or near-channel riverine wetland. This approximation is based on differences between wetland maps produced with linear streamlines and maps produced with two-dimensional streambeds. Estuarine, marine, and lacustrine wetlands are relatively rare and are constrained to the costal boundary of each study area while lacustrine wetlands are more evenly distributed. Palustrine and riverine wetlands are significantly more common, but palustrine wetlands are less evenly distributed than riverine. Second, for each simulated sampling repetition, statistical accuracy was assessed by recording the 95 % confidence interval of the sample mean and determining whether it contained the true mean density for the study area (defined by dividing the total aquatic resource area by the total area of the study region). After 5,000 repetitions, the overall accuracy of the sample confidence intervals was determined by dividing the number of intervals that contained the true mean by 5,000. The result indicates the likelihood that a given plot size can produce a statistically reliable estimate of aquatic resource density. Expected accuracy for the 95 % confidence interval was 0.95. Finally, statistical precision was assessed for each repetition as half the width of the 95 % confidence interval of the mean, divided by the sample mean. After 5,000 repetitions, the average sample error was then calculated. This metric indicated the statistical precision of sample estimates, important for detecting changes in aquatic resource density over time. Estimated survey costs Estimated program costs considered imagery acquisition and map production. We developed cost estimates through expert elicitation. Estimates were further supported by cost estimates obtained by the California Natural Resources Agency while developing the State S&T program. For imagery costs, we considered both no-cost imagery from the NAIP and contracted imagery acquisition. NAIP imagery is available from the U.S. Department of Agriculture and is currently available free of charge. Contracted imagery costs were based
Environ Monit Assess (2014) 186:2599–2608
on estimates provided by a local aerial photography company that has experience producing imagery for wetland mapping. Map production costs were estimated by two groups that routinely produce wetland maps for state and federal resource and regulatory programs.
Results and discussion Estimated costs: imagery and mapping Estimated costs for S&T programs were primarily driven by two components: imagery and map production. Imagery costs could include no-cost imagery from publicly available sources, such as NAIP, or newly acquired imagery. NAIP imagery is commonly used for aquatic resource map production and meets all of the technical requirements including 1-m resolution and four-band, color plus IR (Dahl 2011). However, use of NAIP or other freely available imagery sources limits the season of acquisition. For example, NAIP imagery is typically acquired between June and August. In Southern California, the limited rainy season puts the ideal timeframe for imagery acquisition for wetland mapping in June or even earlier. In addition, while NAIP has recently been acquired every 1 to 3 years, future acquisition frequency and image details would be outside the control of the S&T program managers. Estimates of costs to acquire new imagery were provided by a private company with substantial experience in aerial image acquisition for scientific surveys. Estimates were also consistent with those obtained by the California Natural Resources Agency while developing budget estimates for the State program. Based on these consultations, we estimated that 1-m or better resolution imagery for the plot sizes considered in this study could be acquired through single-pass acquisition at a rate of $150 to $450 per plot, depending on remoteness. Plot size did not affect cost estimates as all sizes considered could be captured with a single image and aerial pass. Map production costs were developed by two groups with many years of experience mapping aquatic resources for academic, government, and non-profit groups. The two groups reviewed their labor and contract records to produce estimates of time and salary costs and independently arrived at a mapping cost of $25 per km2. Both estimates considered all segments of map production including source imagery and data
Environ Monit Assess (2014) 186:2599–2608
management, streamline modeling and polygon production, classification of aquatic resources, and quality assurance and quality control. Mappers indicated that mapping costs would scale linearly with plot size. In addition, neither group believed that cost would be influenced by the total number of plots, that is, there would be no additional costs associated with stopping and starting mapping to produce aquatic resource maps for a greater number of discrete locations. We acknowledge that the above cost estimates are based on a limited survey of image acquisition and wetland mapping professionals. However, we believe that the results are still relevant for evaluating the statistical efficiency of different plot sizes. The three image acquisition costs levels—$0, $150, and $450—were (as presented below) sufficiently divergent to produce separation in results between cost levels. In addition, map production costs are a function of area mapped and therefore reflect the relative difference in area more than the cost of mapping each plot. Changing map production costs would not have an impact on the relative cost to produce sample plot maps, i.e., a 4 km2 plot will always be four times as costly to map as a 1 km2 plot. Finally, additional support for the estimates has come from the California Natural Resources Agency, as part of budget estimation for a California S&T program.
Study regions: density and wetland subtypes The three study regions had different density and diversity of wetland resources, allowing us to evaluate the statistical accuracy and precision and cost effectiveness of different plot sizes under a range of conditions. The Bay Area region had the highest wetland density overall, 0.17 km2 wetland per 1 km2 of area, due to the significant estuarine and marine wetland area contributed by the San Francisco Bay—75 % of the total wetland area or a total of 1,500 km2. The remaining wetland area in this region was 67 % palustrine, 16 % riverine, and 13 % lacustrine. The Central and South Coast regions had significantly lower overall densities, 0.035 and 0.053 km2 km−2, respectively, dominated in almost equal parts by palustrine and riverine wetlands (42 % and 37 % in the Central Coast and 38 % and 36 % in the South Coast). Lacustrine wetlands accounted for 14 % of wetland area in both of these regions, and estuarine and marine wetlands made up the remaining 7 % and 10 % in the Central and South Coast, respectively.
2603
Wetland density in the Central Coast study region was the closest match to other regions of California based on estimates from NWI maps prepared within the last decade. Recent NWI maps suggest that wetland density in California is 0.032 km2 km−2, closest to the density of the Central Coast region, and comprised of 38 % palustrine, 34 % riverine, 21 % lacustrine, and 8 % estuarine and marine, similar to the Central and South Coast regions. In contrast, for maps produced more than a decade ago, statewide density estimates are 0.053 km2 km−2, closest to the South Coast region, comprised of 44 % palustrine, 41 % lacustrine, 10 % estuarine and marine, and 5 % riverine. The significant difference in the portion of wetlands mapped as riverine is likely due to a change in conventional wetland mapping protocols. Prior to the most recent decade, mapped streamlines were reserved as one-dimensional features. In contrast, current mapping protocols convert linear streamlines into two-dimensional approximations of streambed width.
Plot size: resource subtypes and inclusion frequency The difference in wetland subtype densities between the three regions also allowed us to examine the relationship between plot size and the likelihood sampled plots would contain wetland types of interest. Larger plots were more likely to contain each resource subtype, and the fraction of plots that contained certain subtypes decreased significantly between a 4- and 1-km2 plot size (Fig. 2), suggesting a potential minimum desirable plot size of 4 km2. In each study area, close to 100 % of plots contained wetlands, even for the smallest plot size considered. Mapped streamlines and riverine wetlands, which tend to be uniformly distributed, were present in more than 80 % of plots, regardless of plot size. However, for palustrine wetlands, which are less uniformly distributed, the fraction of plots drops from 90 % to 100 % for the largest plot size, 16 km2, to approximately 70 % for 1 km2 plots in the Central and South Coast, and less than 50 % for the Bay Area. The largest decrease in palustrine frequency occurred between the 4 and 1 km2 plot size—dropping from 88 % to 91 % to 70 % in the Central and South Coast and from 74 % to 47 % in the Bay Area. For the less-frequent wetland types, prevalence decreased by approximately half between each plot size.
2604
Environ Monit Assess (2014) 186:2599–2608
Plot size: statistical accuracy and precision
Fig. 2 Fraction of plots with aquatic resources, estuarine or marine, lacustrine, palustrine, and riverine subtypes for 1, 4, 9, and 16 km2 plot sizes in the a Bay Area, b Central Coast, and c South Coast study areas
For all three study regions, the 1 km2 plot size had the highest statistical accuracy for a given estimated cost when no-cost imagery was assumed (Fig. 3). For example, assuming no-cost imagery and a total cost of $10,000 in the Central Coast, accuracy for a 1 km2 plot size was 0.90 versus 0.69 for a 16 km2 plot size (Table 1). In addition, a 1 km2 plot size was also generally the most efficient mechanism for achieving a statistical accuracy of at least 0.90, although differences were less pronounced for the Bay Area (Table 2). If we assume that contract imagery must be purchased, differences between plot sizes for the efficiency versus statistical accuracy relationship were much smaller (Tables 1 and 2). Under these circumstances, the 4 km2 plot size often had the best statistical accuracy and/or was the most cost-efficient. Smaller plots were also more efficient for achieving a narrower calculated confidence interval and lower mean error rate (Fig. 4). The difference in statistical precision between the 1 and 16 km2 plot sizes was largest for the Bay Area. For example, assuming no-cost imagery and a $10,000 total cost, a 1 km2 plot size was associated with an 11 % error rate while a 16 km2 plot size was associated with a 70 % error rate (Table 1). Similar assumptions in the Central and South Coast had error rates of 16–17 % for a 1 km2 plot size and 40–41 % for a 16 km2 plot size. Differences in unit error rates between plot sizes decreased when contract imagery was assumed (Table 1). Under the contracted imagery scenarios, a 4 km2 plot size became the most efficient choice for achieving a 25 % error rate (Table 2). Of note related to sample accuracy was the “underperformance” of sample results related to expected accuracy in the Central and South Coast study areas (Fig. 2). For these two regions, the accuracy of the 95 % confidence interval did not reach 0.95 for any of the plot sizes or sample sizes tested. Instead, accuracy plateaued at approximately 0.90, indicating that only 90 % of the 95 % confidence intervals contained the true mean. In contrast, in the Bay Area, which also had a significantly higher aquatic resource density than the Central or South Coast (0.17 km2 km−2 vs. 0.04 and 0.05 in the Central and South Coast, respectively), accuracy was greater than 0.95 for all but a few of the smallest simulated sample sizes. We considered a number of factors in an effort to explain the lower than expected accuracy for the Central
Environ Monit Assess (2014) 186:2599–2608
2605
with values bounded by zero and one—did not reduce this discrepancy (results not shown). Second, the variance estimator used under GRTS sampling produces a narrower confidence interval than the more commonly used estimator of sample variance (Stevens and Olsen 2003). However, because reduced accuracy was only observed in two of the three regions, we do not believe that the GRTS variance estimator is driving the discrepancy. Therefore, we believe that the discrepancy may be related to the low mean aquatic resource density in those regions (0.03 and 0.05 km2 km−2, respectively) compared with the Bay Area (0.17 km2 km−2). Others have identified the inadequacy of the sample variance estimator when small sample sizes are utilized (Kupper and Hafner 1989; Liu 2009). Statewide generalization
Fig. 3 Accuracy versus estimated costs assuming no-cost imagery for 1 (circle), 4 (triangle), 9 (plus), and 16 (cross)km2 plot sizes in the a Bay Area, b Central Coast, and c South Coast study areas
and South Coast. First, transformation of the population using the lognormal and the arcsine transformations— often used for right-tailed populations and populations
Small plot sizes (1 or 4 km2) were more efficient than large plot sizes (9 or 16 km2) for measuring aquatic resource extent in California. Differences between plot sizes were smaller when contract imagery was considered than when no-cost imagery was assumed (Figs. 3 and 4). Independent of plot size, regions with low average wetland density (0.03–0.05 km2 km−2) were associated with lower statistical accuracy and higher error rates than the region with high average wetland density (0.17 km2 km−2). In addition, cost differences between plot sizes were smaller for low-density regions compared with high-density regions when statistical precision was considered. When statistical accuracy was considered, cost differences were larger for lower-density regions. Finally, 4 km2 plots where preferred over 1 km2 plots when contract imagery was assumed, due to the interaction between plot size and sample variance. In general, we do not consider cost efficiency to be independent of statistical accuracy and precision in the design of a long-term monitoring program. Our objective was to determine the plot size that most efficiently satisfied requirements for statistical accuracy and precision. Removing cost from the considerations would be the same as implicitly assuming production costs were equal for all plot sizes. In our opinion, this assumption does not produce an accurate assessment of program efficiency. In all study areas, larger plots were more accurate and less variable than smaller plots but were significantly less efficient to map (Figs. 3 and 4). Fewer plots were necessary to achieve the same level of accuracy and precision, but because larger plots were also
2606 Table 1 Accuracy and error rates for a given survey cost
Environ Monit Assess (2014) 186:2599–2608
Plot size
No-cost imagery ($10,000)
Low-cost imagery ($50,000)
High-cost imagery ($75,000)
Accuracy
Error
Accuracy
Error
Accuracy
Error
1 km2
90 %
17 %
89 %
21 %
87 %
27 %
4 km2
85 %
29 %
89 %
19 %
88 %
23 %
9 km2
78 %
36 %
89 %
21 %
88 %
23 %
16 km2
69 %
41 %
85 %
25 %
85 %
25 %
1 km2
90 %
16 %
90 %
20 %
89 %
26 %
4 km2
86 %
28 %
91 %
19 %
90 %
22 %
9 km2
81 %
36 %
90 %
21 %
90 %
22 %
16 km2
76 %
40 %
89 %
22 %
89 %
22 %
1 km2
94 %
11 %
96 %
14 %
95 %
20 %
4 km2
96 %
28 %
96 %
16 %
96 %
20 %
9 km2
93 %
47 %
96 %
20 %
95 %
23 %
16 km2
91 %
70 %
97 %
27 %
97 %
27 %
Central Coast
South Coast
Bay Area
difference in mapping costs significantly influenced estimation of total costs, and the 1 km2 plot size remained the most cost-effective. In contrast, as-
more expensive to map, smaller plots were the most efficient mechanism for optimizing statistical accuracy and precision. When no-cost imagery was assumed, the
Table 2 Costs to achieve 90 % accuracy (95 % in the Bay Area) or a 25 % error rate
Plot size
No-cost imagery
Low-cost imagery
High-cost imagery
Accuracy
Error
Accuracy
Accuracy
1 km2
$8,000
$5,000
$56,000
$35,000
$152,000
$95,000
4 km2
$24,000
$12,000
$60,000
$30,000
$132,000
$66,000
$40,500
$22,500
$67,500
$37,500
$121,500
$67,500
>$40,000
$36,000
>$55,000
$49,500
>$85,000
$76,500
1 km2
$5,000
$4,000
$35,000
$28,000
$85,000
$76,000
4 km2
$15,000
$12,000
$37,500
$30,000
$82,500
$66,000
$27,000
$22,500
$45,000
$37,500
$81,000
$67,500
>$40,000
$28,000
>$55,000
$38,500
>$85,000
$59,500
1 km2
$5,000
$3,000
$35,000
$21,000
$95,000
$57,000
4 km2
$6,000
$12,000
$15,000
$30,000
$33,000
$66,000
$13,500
$22,500
$22,500
$37,500
$40,500
$67,500
$20,000
$40,000
$27,500
$55,000
$42,500
$85,000
Error
Error
Central Coast
9 km2 2
16 km
South Coast
9 km2 2
16 km
Bay Area
9 km2 2
16 km
Environ Monit Assess (2014) 186:2599–2608
Fig. 4 Error versus estimated costs assuming no-cost imagery for 1 (circle), 4 (triangle), 9 (plus), and 16 (cross)km2 plot sizes in the a Bay Area, b Central Coast, and c South Coast study areas
2607
sumption of contract imagery decreased the effect of plot size on survey costs and, in the case of 1 and 4 km2 plot sizes, meant that the 4 km2 plot became more cost-effective. The 4-km2 plots were also preferred over 1-km2 plots with regard to the diversity of aquatic resource types included in each sample plot (Fig. 2). Primary design considerations for the California S&T program are the cost considerations associated with achieving statistical accuracy and precision. Secondary design considerations for a large-scale S&T program include use of maps as a sample frame for ambient environmental field assessments. Less than 10 % of California has aquatic resource maps produced within the past decade; therefore, for large areas of the state, the S&T plot maps will represent the best-available aquatic resource maps. Therefore, larger plot sizes may provide better landscape context and elucidation of wetland complexes or specific resource subtypes. In addition, in the event that S&T plots show a gain or loss in aquatic resource area, proximal landscape elements, such as agricultural land use or urban or suburban development, within the mapped area can support hypothesis generation to help explain the drivers of gains or losses. Larger plots will also be better equipped to capture these interactions. Based on the balance of considerations—cost efficiency for achieving statistical accuracy and precision, capturing a diversity of aquatic resource and landscape types, and supporting hypothesis generation related to area gains and losses—a 4 km2 plot size optimized the objectives for development of the California S&T program. The work presented here is the first simulation study on the tradeoffs of appropriate plot size for monitoring the spatial extent of wetlands in a design-based program. Existing S&T programs utilize plot sizes between 1 km2, for the NILS, and 10.36 km2 (4 mi2), for the NWI-S&T (Dahl 2011; Ståhl et al. 2010). Neither program reported a cost–benefit analysis supporting their choice of plot size. The MN-S&T program (2.59 km2, 1 mi2) did consider the sample variance and estimated cost for different plot sizes but confined their analysis to a limited subset of the state (Deegan and Aunan 2006). In addition, the Minnesota analysis was limited to standard sample size calculations, focused on wetland density as opposed to streams and wetlands, and did not consider the empirical variability of sample estimates through simulation.
2608
Conclusions Overall, small plot sizes (1 or 4 km2) were more efficient for statistical accuracy and precision than larger plot sizes (9 or 16 km2) for the study areas considered; however, larger plot sizes contained a greater diversity of rare or geographically constrained wetland types such as estuarine, lacustrine, and marine wetlands. Balancing these considerations for the California S&T program led to selection of the 4 km2 plot size. The 4 km2 plot size also allows for more robust optimization, regardless of whether imagery is free or must be purchased. These results and tradeoffs should transfer well to other regional, tribal, or state programs using design-based methods to assess the extent and distribution of aquatic resources in heterogeneous landscapes. The approach presented here can also be easily replicated for programs that may want to conduct their own location-specific optimization. Acknowledgments The funding for this study was provided by the US Environmental Protection Agency (Grant CD-00 T22301) through the California Natural Resources Agency. Information vital to the cost analysis was graciously provided by Shawna Dark and Patricia Pendleton of California State University, Northridge; Kristen Cayce of the San Francisco Estuary Institute; and Nick Arentz of Skyview Aerial Photo, Inc., Murrieta, CA. Guiding input on analysis and interpretation was provided by the Technical Advisory Committee to the project, particularly Kerry Ritter of the Southern California Coastal Water Research Project, Steve Kloiber of the Minnesota Department of Natural Resources, and Paul Jones of the US Environmental Protection Agency; and by the dissertation committee to Dr. Lackey at the University of California, Los Angeles, particularly Richard Ambrose and Mark Handcock.
References Albert, C. H., Yoccoz, N. G., Edwards, T. C., Graham, C. H., Zimmermann, N. E., & Thuiller, W. (2010). Sampling in ecology and evolution—Bridging the gap between theory and practice. Ecography, 33(6), 1028–1037. doi:10.1111/j. 1600-0587.2010.06421.x. Baron, J. S., Poff, N. L., Angermeier, P. L., Dahm, C. N., Gleick, P. H., Hairston, N. G., et al. (2002). Meeting ecological and societal needs for freshwater. Ecological Applications, 12(5), 1247–1260. doi:10.1890/1051-0761(2002)012[1247: MEASNF]2.0.CO;2. Clare, S., Krogman, N., Foote, L., & Lemphers, N. (2011). Where is the avoidance in the implementation of wetland law and policy? Wetlands Ecology and Management, 19(2), 165–182. doi:10.1007/s11273-011-9209-3.
Environ Monit Assess (2014) 186:2599–2608 Dahl, T. E. (2011). Status and trends of wetlands in the conterminous United States 2004 to 2009. US Department of the Interior, Fish and Wildlife Service. Deegan, G., & Aunan, T. (2006). Survey design factors for wetland monitoring. Minnesota Department of Natural Resources, Forestry/Resource Assessment. Gregoire, T. G. (1999). Design-based and model-based inference in survey sampling: Appreciating the difference. Canadian Journal of Forest Research, 28, 1429–1447. Kincaid, T. M., & Olsen, A. R. (2011). spsurvey: Spatial survey design and analysis; Vienna, Austria: R Foundation for Statistical Computing Kloiber, S. M., & Norris, D. J. (2013). Status and trends of wetlands in Minnesota: Wetland quantity trends from 2006 to 2011. St. Paul, MN; Minnesota Department of Natural Resources. Kupper, L. L., & Hafner, K. B. (1989). How appropriate are popular sample size formulas? The American Statistician, 43(2), 101–105. doi:10.1080/00031305. 1989.10475628. Lackey, L. G., & Stein, E. D. (2013). Design-based sampling for aquatic resource extent monitoring in California. Wetlands. Liu, X. S. (2009). Sample size and the width of the confidence interval for mean difference. British Journal of Mathematical and Statistical Psychology, 62(Pt 2), 201–215. doi:10.1348/ 000711008X276774. Mitsch, W. J., & Gosselink, J. G. (2000). The value of wetlands: Importance of scale and landscape setting. Ecological Economics, 35(1), 25–33. doi:10.1016/S0921-8009(00) 00165-8. Nusser, S., & Goebel, J. (1997). The national resources inventory: A long-term multi-resource monitoring programme. Environmental and Ecological Statistics, 4(3), 181–204. Olsen, A. R., & Peck, D. V. (2008). Survey design and extent estimates for the Wadeable Streams Assessment. Journal of the North American Benthological Society, 27(4), 822–836. doi:10.1899/08-050.1. R Development Core Team. (2011). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.Rproject.org. Rossi, J. (2004). The effect of sampling unit size on the perception of the spatial pattern of earthworm (Lumbricus terrestris L.) middens. Applied Soil Ecology, 27(2), 189–196. doi:10.1016/ j.apsoil.2004.03.001. Ståhl, G., Allard, A., Esseen, P.-A., Glimskär, A., Ringvall, A., Svensson, J., et al. (2010). National Inventory of Landscapes in Sweden (NILS)—Scope, design, and experiences from establishing a multiscale biodiversity monitoring system. Environmental Monitoring and Assessment, 173(1–4), 579– 595. doi:10.1007/s10661-010-1406-7. Stevens, D. L., & Olsen, A. R. (2003). Variance estimation for spatially balanced samples of environmental resources. Environmetrics, 14(6), 593–610. doi:10.1002/env.606. Stevens, D. L., & Olsen, A. R. (2004). Spatially balanced sampling of natural resources. Journal of the American Statistical Association, 99(465), 262–278. doi:10.1198/0162145 04000000250.
