Publication:
Finite-sample performance of absolute precision stopping rules

Loading...
Thumbnail Image
Authors
Singham, Dashi I.
Schruben, Lee W.
Subjects
Simulation
Statistical analysis
Design of experiments
Advisors
Date of Issue
2012
Date
Fall 2012
Publisher
INFORMS
Language
en_US
Abstract
Absolute precision stopping rules are often used to determine the length of sequential experiments to estimate confidence intervals for simulated performance measures. Much is known about the asymptotic behavior of such procedures. In this paper, we introduce coverage contours to quantify the trade-offs in interval coverage, stopping times, and precision for finite-sample experiments using absolute precision rules. We use these contours to evaluate the coverage of a basic absolute precision stopping rule, and we show that this rule will lead to a bias in coverage even if all of the assumptions supporting the procedure are true. We define optimal stopping rules that deliver nominal coverage with the smallest expected number of observations. Contrary to previous asymptotic results that suggest decreasing the precision of the rule to approach nominal coverage in the limit, we find that it is optimal to increase the confidence coefficient used in the stopping rule, thus obtaining nominal coverage in a finite-sample experiment. If the simulation data are independent and identically normally distributed, we can calculate coverage contours analytically and find a stopping rule that is insensitive to the variance of the data while delivering at least nominal coverage for any precision value.
Type
Article
Description
The article of record as published may be found at http://dx.doi.org/10.1287/ijoc.1110.0471
Series/Report No
Department
Operations Research (OR)
Organization
Naval Postgraduate School (U.S.)
Identifiers
NPS Report Number
Sponsors
Funder
Format
12 p.
Citation
D.I. Singham, L.W. Schruben, "Finite-sample performance of absolute precision stopping rules," INFORMS Journal on Computing, v.24, no.4, (Fall 2012), pp. 624-635
Distribution Statement
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.
Collections