Seldom, we discuss about the planning and implementation of the load test in our site. As such, we are taking a change in this and will be touching on some considerations you as a performance tester/test manager/consultant should be thinking for every load test. These considerations enable you to smooth up the process of the entire load test and reduce any load test risks associated with it. More than often, we focus on gathering the customers requirements for the load test such as user amount to generate, how long the load test will be conducted, where will it be conducted, the location that the virtual users will generate from, etc… That is basic information. However, additional information will be required to smoothen the load test project and below are some of the additional considerations, you as a performance tester/test manager/consultant should be asking yourself or client.
- Do you have sufficient hardware? – How may load generators are required for the entire load test? Refer to this document from HP Support, “Is resource (cpu and memory) footprint information available to help in estimating the load generators required for load testing with LoadRunner?” for estimating the load generators required by multiplying the observations with the total number of virtual users generated in the test. In addition to the processor utilization recommended in the document, do watch the memory and network consumption of the load generators. The network interface card (NIC) can become a bottleneck for load generation. A 1GB NIC will definitely generate and receive more load than a 100MB NIC. Therefore, do a check on the machines assuming the load generator roles and make appropriate measures to prevent the NIC in becoming the bottleneck. With the above, you will get a rough gauge of the required resource for your test.
- Do you need spare hardware? – Do cater some spare machines if additional load generators are required, such as an additional of 5 load generators. This is to stand by in the event of increasd virtual users and machine malfunction.
- Remember that it’s a team effort! – Can you secure human resources for the load test? You will need the web, application and database server teams to be around to observe the system behavior during the test or to assist in any reboot or refresh activities for the system. Network and firewall/security team may be required to ensure that connectivity is allowed for the load test. Therefore, in the planning, you have to get the relevant parties to commit the human resources!
- Installation of LoadRunner – This require administrative or power users privileges in order for installation to be successful. That raise another set of questions: Will you be able to install LoadRunner in the premises? Who can you look for to install the LoadRunner program? Do you need clearance for installation? Is the system being hardened to prevent installation?
- Offshore Installation – Is offshore installation required for the load test setup? Can you perform the installation via remote connection or do you require someone from the site to assist in the installation?
- Running of JVM/.NET Probes for Diagnostics – This will require installation and instrumentation on the application server, meaning it will be intrusive to the system as a whole. Do you require additional permissions for such installation?
- Post-installation – Do remember to test if the installation is complete and the programs are working before moving on to the next phase. Remember to do a connectivity check between the load generators to the controller to ensure they communicate prior the load test.
- Working in different environments – Are you required to work in different stages of a load test project? Example, you are required to run the test in (A) the data center followed by (B) the local client segment and lastly (C) the remote client segment? In this context, how do you ensure the scripts worked from A to C? It will be easy if it is a change of hostnames and URL addresses that is being used in these environments as it can be achieved through parameterization. But what if there are additional authentication mechanism introduced in environment B and C? Authentication mechanisms such as additional cookie in the clients are added into the URL or 2-factor authentication can render the scripts developed in environment A useless! How do you go about it? Should you cater more time to re-script for the worst scenario or is it better to disable the authentication mechansim during the load test?
These are some basic considerations we usually asked our ourselves and clients. Hope it will be useful in your planning. What are other considerations make for a load test for you?
http://www.loadrunnertnt.com/planning/planning-for-a-load-test/
