A test dataset was built containing 100,000 rows, each with 10 random numbers between -300 and 700 and another test dataset containing 1,000,000 dates between January 1, 2000 and present. Then various tests were run using this data on a 1.53 GHz CPU with 512 MB of memory. The timings for these tests are shown below. The numeric data, date data and the Excel program used to build the data and run the tests are all available for download (zipped file 10.5 MB).

Processing time for relatively small populations (e.g. under 50,000 items) is very fast. As the size of the population increases, the computational speed per transaction slows down. On the same processor mentioned above, computing statistics on a population of 1,250,000 took about three minutes (roughly 6,500 transactions per second). The same test on a population of a little over 80,000,000 transactions took about 6 hours and fifteen minutes, or about 1/2 the processing speed per transaction (3,100 transactions per second) of that required for a smaller population.

The results of the computations were compared with those obtained using SAS software version 6.12. There were no signficant differences between these computed amounts, although the SAS software is considerably faster for larger populations.

  Note that the times above include disk access and will vary depending upon the data being processed. Timings for just the computations alone are considerably faster than these reported amounts.

I welcome all comments and suggestions. I publish a variety of RSS feeds, as well as an audit blog, should you wish to be informed about software updates and new feature releases, or provide any comments at the blog.

e-mail: Mike.Blakley@ezrstats.com.