A.4.4 Results Using ISM
The following subsections provide results for the RDX and HMX DUs. Within each simulated DU subsection, 40 sets of results are shown using two different UCL calculation methods. Both systematic grid and random grid sampling routines for the grouped and standard IS patterns were used. Differences in results for these sampling routines were within the range of simulation (stochastic) error. Figure A-10shows an example of the equal coverage for both M2-A and M2-B using the three different standard IS sample selection patterns (random grid, simple random, and systematic random) for t-based 95% UCLs. For simplicity, only the results associated with the random grid sampling routines are presented in each section.
Figure A-10. A coverage plot comparing systematic grid (with random start), random grid,
and simple random sampling for the RDX DU (M2-A) and HMX DU (M2-B) when 2, 3, 4, or 5 ISs are collected from the DU.
The tables shown in each section will be separated into the three general sampling patterns—standard IS, grouped IS, discrete sampling. Each table summarizes the results from 2000 iterations. The first two columns are different for the IS and discrete summary tables. For the IS summary tables, the first column identifies the number of ISs sampled from within the DU, and the second column shows the number of increments in each IS. For the discrete summary tables, the first column identifies whether random or systematic sampling was used, and the second column lists the number of increments sampled from the DU that are used to calculate the mean and standard deviation. The third and fourth columns show the UCL coverage for the Chebyshev and t-UCL calculations. The last four columns summarize the RPD of the UCL values using the Chebyshev and t-distribution UCL multipliers. The "RPD above" column for each UCL multiplier is the average relative difference of the UCL from the true mean for those UCL values that were above the true mean. The "RPD below" columns for each UCL multiplier show the average relative difference of the UCL from the true mean for those UCL values that were below the true mean.
Each subsection contains plots depicting the pertinent information from the coverage tables for an easier visualization of the results from simulation studies. These plots show the designed UCL coverage level (dashed blue line) and the coverage performance of each sampling pattern as a function of the number of increments (in each IS for the IS designs and total for discrete designs). Each colored line represents a different sampling pattern with a separate plot for the discrete, grouped IS, and standard IS. The dashed line identifies the t-UCL calculations, and the solid line identifies the Chebyshev UCL values. Each plotted point represents the results from one line from the tables within the subsection. Coverage results based on 2000 iterations provide estimates accurate to within approximately ±1.5% to ±2.5%.
One figure of 40 UCL histograms with consistent axes is shown in each subsection. These figures are meant to show general distributional and coverage patterns of the calculated UCLs over all sampling patterns and may be difficult to use for evaluating any specific one.
The displayed t-distribution UCL calculations are based on a 95% UCL using t-distribution with the df equal to 1 minus the number of measures used to calculate the standard deviation for each scenario. For the IS sampling patterns df is the number of IS replicates gathered from the site minus 1. For the descrete sampling patterns df is the number of samples gathered minus 1. It is understood that the t-distribution is not appropriate for cases where the sample size is small and the measured values do not follow a normal distribution. This would generally be the case for the discrete sample designs with 9 and 16 samples as applied to the five simulated sites. In many instances a different UCL method would be needed for all discrete sample designs (16, 30, 49, and 100). Alternative UCL calculations that do not rely on normal theory should be used in those cases. Such UCL calculations can be found in software such as ProUCL (Singh et al. 2007) and Visual Sample Plan (Matzke et al. 2007) for use in environmental studies. There are a variety of choices depending on site-specific needs.
For the proposed IS sampling methods, the t-distribution may not provide adequate coverage, and with the limited number of available data values, it is difficult to use many of the tools in ProUCL for alternative UCL calculations. Thus, a more conservative Chebyshev multiplier is used for attaining an improved coverage percentage. The UCL coverage plots and tables also show the Chebyshev 95% UCL calculations. The standard error is multiplied by a prespecified value and added to the mean to identify the UCL. For the t-distribution this value is a function of the number of values used to estimate the mean and standard error. The Chebyshev multiplier is 1/sqrt(1 – 0.95) for a 95% UCL regardless of the sample size used. This generally conservative multiplier of 4.472 will shift the coverage statistics up for all sampling patterns except for the two IS designs. A t-distribution with 1 df results in a multiplier of 6.313. The most drastic effects of the Chebyshev multiplier are seen with the discrete designs, as their coverage and bias increases the most.