Performance testing is one of the most common tasks in web service testing and probably one of the areas where the confusion is most common. Not so much in the tasks involved, but rather in the terminology. We will try to give a short overview to performance testing here and what it means for web service testing.
Note: What is called a load test in SoapUI is actually a performance test. You can use SoapUI for several different kinds of performance related tests.
What is performance testing?
If we should explain our definition of performance testing in one sentence it would be:
“Performance testing is artificially creating or simulating load and measuring how your environment handles it.”
This means it does not necessary have to be how a system performs under high load, it can also be how it performs under base load or expected load. It doesn’t even have to be structured, automated or created in a tool like SoapUI; simply refreshing your web browser over and over again very fast is a load test.
What kinds of performance testing are there?
There are various definitions of performance testing:
• Baseline Testing
Technically speaking, this can be defined as pure performance testing. Baseline testing examines how a system performs under expected or normal load and creates a baseline with which other types of tests can be compared.
Goal: Find metrics for system performance under normal load.
• Load Testing
Load testing includes increasing the load and see how the system behaves under higher load. During load testing, you can monitor response times, throughput, server conditions, and more. The goal of load testing is not to break the target environment though.
Goal: Find metrics for system performance under high load.
• Stress Testing
The goal of stress test is exactly that, to find the load volume where the system actually breaks or is close to breaking.
Goal: Find the system breaking point.
• Soak Testing
To find system instabilities that occur over time, you need to run tests for a long period. That is what soak testing is for; run load tests or even baseline tests over a long period of time and see how the target environment handles system resources and if it works properly. The most common defect found by soak testing is memory leaks. The most common scenario for soak testing is turning on a number of tests when you leave your office on a Friday and let it run over the weekend.
Goal: Make sure no unwanted behavior emerges over a long period of time.
• Scalability Testing
The purpose of scalability testing is to check whether your system scales appropriately to the changing load. For example, a larger number of incoming requests should cause proportial increase in the response time. The unproportional increase indicates scalability issues.
Goal: Find metrics and check if the system performance changes appropriately to the load.
Download ReadyAPI and Start Performance Testing
What factors affect the web service performance?
If we look into unique characteristics of web service performance, two aspects stand out: a large amount of XML/JSON processing on the server side (both XML/JSON parsing and serialization), and processing of payloads (request and response bodies). The reasons why this fails can be multifold; it can be in the platform in the shape of for example weaknesses in the application server, and in the implementation in the shape of unnecessarily complex WSDL’s. It could also be that the code is making a request to a database that is slow in responding.
One more factor that affect the performance frequently is security. Those who do web performance testing know that HTTPS sites have considerably lower performance that their HTTP analogues, and in web service testing, we can add a layer of WS-Security to the layer of HTTP security, decreasing the performance even more.
The complexity of parsing the XML/JSON payloads means that we have to put an extra focus on scalability testing, while the issue of security means we will have to focus some extra on doing testing of requests that are secure. If the entire web service is secure, this means load testing is more important especially if WS-Security and token handling is used.
It also means we have to examine the API specification closely. If the requests and responses are either complex or large, or if they include large attachments, we have to focus on emphasizing that complexity and see how it behaves under load.
No one knows APIs better than SmartBear. Find out what our Pro version of SoapUI can do to improve your testing.
Learn more about load testing: