Stop And Go Attribute Sampling
If the supporting documents for data being audited are contained in a central location, e.g. no travel or other logistics are involved, then stop and go sampling may be a more efficient and effective method for random sampling for the following reasons:
- There is no need to compute a required sample size,
- There is no need to perform a preliminary analysis of the population attributes such as expected error rate, and
- There is little or no risk in "over sampling", i.e. testing more samples than required and therefore spending excess audit time doing the testing.
Stop and Go sampling is a statistically valid process which involves the following steps:
- Assign a random number to each item in the population (e.g. using "Mersenne Twister" or other statistically valid random number generator)
- Sort the population by assigned random number, either ascending or descending
- Select the first 10 - 20 items (auditor judgment as to number), test them and put the results into an Excel spreadsheet.
- Run a "stop and go" sample report and review the results (see example below)
- If the resulting sample precision is too large, then select another group of transactions by sorted assigned random number (auditor judgment as to number)
- Test the samples and record the results in the same Excel spreadsheet.
- Run another "stop and go" sample an review the results.
- Repeat steps 5 through 7 until satisfactory results have been obtained.
The report from the Stop and Go Sample will show the intermediate results, sample statistics as well as calculate the estimate of the population at four confidence levels - 80%, 90%, 95% and 98%. The results will also be charted for easy review. The charts show the upper and lower bounds, as well as the point estimate for each calculation.
Shown below is an illustration of the process. The population consisted of 432 transactions. Random numbers were assigned to each transaction and the population was then sorted in ascending order by random number assigned.
Step 1 - Select the first 10 transactions
The auditor then tested the first 10 randomly sorted transactions and recorded the test results in an Excel worksheet. Running the Stop and Go Variable sample report resulted in the following results.
The box and whisker plot on the left shows the point estimate as the white line in the middle. The upper and lower limits for 95 and 98% confidence levels are shown as well. The box and whisker plot on the right shows the same statistics, but for 80 and 90% confidence levels. See example report.
Step 2 - Select an additional 15 transactions
The auditor determines that the precision level achieved with a sample of 10 transactions is insufficient and therefore selects another 15 transactions and tests them (auditor judgment to select 15 more). The audit results are entered into the Excel worksheet and the Stop and Go sample report re-run. The results are shown below.
The chart indicates improvement in the precision levels, but they are still insufficient. See example report.
Step 3 - Select an additional 57 transactions
After selecting and testing the next 57 transactions and entering the results in the worksheet, the auditor can then re-run the Stop and Go Report and obtain the following:
At this point the auditor has achieved the precision required and can stop further tests. See example report.
The Stop and Go Variable Sampling routine uses standard statistical calculations as follows:
- Count the number of items in the population (N)
- For the sample of items (n), count the number of items (e) which represent the error condition being tested for and compute the sample error rate (errrate)
- Computation of the point estimate is simply N * errrate
- The population standard deviation (S) is assumed to be the same as the sample, i.e. (s) or errrate * (1-errrate)
- A z-score (z) is computed based upon the confidence level required (done using Cephes library)
- The upper and lower limits are computed as the point estimate plus or minus (z * s/square root (n))