Benchmarking Performance on Amazon Servers

Bank Recon benchmarking is underway with the goal of providing measurable information as to what the optimal setup is, as well as to quantify the improvements gained from the recon rule changes that have been made. Remember, we are testing in a very specific environment so we can only give guidelines to maximise performance. Factors such as the number of applications running on your servers and the number of users will affect performance in your environment in ways we haven't tested. 

We decided to use the Amazon EC2 servers for the benchmarking exercise. The benefit of the Amazon setup is that hardware can be easily scaled to measure the effect of upgrading the processors, memory and disk space without the physical cost of buying and housing the servers . Amazon provides a number of 'instances' which have differing CPU's, cores and memory. Amazon uses the EC2 Compute Unit (ECU) to measure CPU output. One ECU unit is roughly equivalent to a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. More information about the Amazon EC2 servers can be found here.

Thus far we are testing on the following configurations

Instances Used

EC2 Large: 4 ECUs, 2 cores, 7.5 GB memory

EC2 Extra Large: 8 ECU's, 4 cores, 15 GB Memory

Server Setup

Windows Server 2008 R2 64 bit.

SQL Server 2005. 

Database and application running on the same server.

Databases:

160 GB with the Cashbook Transaction Table partitioned on Recon Id

6 GB with a standard schema

I will detail the testing done and the results in a later blog post when it is all completed, which should be in the next week or so. We have tested the recon performance improvement of the new code vs the old code as well as benchmarked the Import process on the Large instance. We are now testing how scaling the hardware affects the performance by running the recon rules and imports on the Extra Large Instance using the new code only. We may test on other instances as well to get a better idea of scalability. Every time we release a new version going forward we will benchmark it to ensure the performance is as good or better. 

And as soon as the testing is complete we will release the new build - watch this space!