Coverage is a ReadyAPI feature that allows you to analyze how much of a given SOAP or REST Web Service is being covered by your Functional Tests, MockServices, Mock Clients and SOAP Monitor clients and targets. This document aims at giving a brief overview of the concepts of Web Service Coverage, with links to the corresponding documentation for each Coverage point in soapUI. The best way to learn is to play around with it!
Web Services Coverage
Traditionally in environments such as Java or .Net , coverage is used to measure how much of the actual code base that is being used during a given execution scenario (usually during Unit Tests). The returned value tries to give an estimate of how much of the code base that is actuality being used or tested.
SOAP
SOAP Web Services in soapUI are defined by a WSDL contract, which specifies the operations, messages, faults, representations, et c. that make up the web services functionality. For each message there is generally a finite amount of defined elements and attributes that can be used to convey the message content, divided among message bodies, headers and faults.
The Coverage functionality analyzes each messages contract definition and then compares all corresponding messages during for example a TestRun to calculate how many of the defined elements or attributes that were actually used during the tests. The result is in soapUI presented as both a percentage and a visualization of the actual message; "Message Coverage"
Further, when running functional tests, soapUI analyzes the performed XPath assertions, and will try to calculate a corresponding value of how many of the defined elements or attributes that have actually been asserted. This result is added to the Message Coverage presentation and called "Assertion Coverage".
Coverage Measurement Points
ReadyAPI allows you to measure coverage in 3 distinct scenarios:
Functional Tests
Functional Test Coverage measures how much of your Web Service contract is actually covered by your tests. The corresponding Coverage Panel is available at the Project, TestSuite and TestCase level.
MockServices
MockService Coverage measures both how much of your WSDL is being mocked and how much of your WSDL is being utilized/sent by the Mocks clients.
HTTP Monitor
HTTP Monitor Coverage measures how much of your WSDL is being used in a real life scenario.
Working With Coverage in Tests
Project Coverage
Project Coverage is the main Coverage analysis point for Test Coverage, allowing you to run all tests in your project and see the resulting Coverage analysis.
TestSuite Coverage
TestSuit Coverage is an Unique Functionality for soapUI (ReadyAPI only) that allows you to see how much of the Web Services are being covered by you tests. In everyday work this let's you improve your test by adding assertions for uncovered elements as well as shows which elements are unused in your test suites.
The TestSuite Coverage tab contains the same Coverage information as available in the Project Coverage Panel. Enabling Coverage and running the TestSuite will dynamically calculate/update the messages exchanged during the tests and add their Coverage information to the Coverage Panel. See the Project Coverage document for details.
TestCase Coverage
TestCase Coverage is an Unique Functionality for soapUI (ReadyAPI only) that allows you to see how much of the Web Services are being covered by you tests. In everyday work this let's you improve your test by adding assertions for uncovered elements as well as shows which elements are unused in your test cases.
The TestCase Coverage tab contains the same Coverage information as available in the TestSuite and Project Coverage Panels. Enabling Coverage and running the TestCase will dynamically calculate/update the messages exchanged during the tests and add their Coverage information to the Coverage Panel. See the Project Coverage document for details.