It has been our experience at Barraclough that contractors almost always skip the step of taking a probe sample when calculating the required sample size. Even though they do this, they frequently rely on RAT-STATS to make the sample size calculations. The inputs into RAT-STATS requires the variation in the variable being estimated, that is, RAT-STATS requires as one of its crucial inputs the variation (e.g., the mean and standard deviation) of the overpayments (which is the variable being estimated). Since the contractors skip taking a probe sample, they plug the wrong data into the RAT-STATS program, and then make their calculation of sample size by using the variation of the payments instead of the underpayments. This almost always results in RAT-STATS claiming that a smaller sample size is adequate. In the MPIM, Chapter 3, Section 3.10.2, we see a sketch of what a “properly executed” sample design is. In includes:
(1) defining the universe, (2) [defining] the frame, (3) [specifying] the sampling units, (4) using proper randomization, (5) accurately measuring the variables of interest, and (5) using the correct formulas for estimation
It can be argued that taking a probe sample so as to be able to plug the correct (and required) data into the RAT-STATS program falls under the fifth category “accurately measuring the variables of interest”. It follows that if the probe sample is not taken, then according to the MPIM, proper procedures have not been used. Note: RAT-STATS is a free statistical software package that providers can download to assist in a claims review. The package, created by OIG in the late 1970s, is also the primary statistical tool for OIG‘s Office of Audit Services.