Monte Carlo analysis was used to repeatedly apply a specified sampling design (number of increments and ISs) to a DU scenario. Typically between 5,000 and 30,000 trials were used. The large number of trials can be expected to yield relatively stable (i.e., reproducible) results. Each trial represents a complete sampling event (i.e., “n” increments and “r” replicates) and yields an estimate of the population mean, the standard error of the mean, and the UCL. Collectively, the results yield a distribution of 95% UCLs that can be used to calculate the performance metrics. For example, ideally, the sampling method and UCL calculation yield a probability distribution of UCLs with a 5 th percentile equal to (or greater than) the true population mean. This would mean that one can expect that the sampling design applied to this type of population will achieve the desired coverage (or percentage of exceedences of the true mean) of 95%. Table A-2 provides examples of simulation experiments with coverages that vary from approximately 80% to 100%.
Multiple ISM samples (i.e., replicates) must be collected to calculate the standard error and UCL. The expected small sample sizes (e.g., three to seven replicates) for most implementations of ISM preclude the use of bootstrap resampling techniques to calculate a UCL; therefore, simulations were performed using only the Student’s-t and Cheybshev UCL methods, which are based on the sample size, sample mean, and variance (see equations at the end of this appendix). Because the distribution of sample means tends to exhibit less skew than the population due to the central limit theorem, the performance of the Student’s-t UCL can vary. The Student’s-t can be expected to yield the most reliable performance metrics for populations with a low (e.g., ≤1) CV. By contrast, the Chebyshev generally yields higher UCLs with higher coverage but also higher RPDs.
Generally, sampling designs were varied 15–100 increments and 2–7 replicate ISM samples. The mean of the distribution represents the population mean and is used to calculate the bias and relative percent difference metrics.
The number of replicates is used to represent the degrees of freedom in UCL calculations using ISM.