Home Docs TestCase Execution

TestCase Execution

Lets have a quick look at what actually happens when a TestCase is executed, both in regards to individual TestSteps, the TestCase log and triggered events, as this will give you a better understanding of how to build TestCases so they run as you want them to.

Starting up the TestCase

When starting a TestCase the following actions are taken internally:

  1. A context for the execution of the TestCase is initialized. This is passed to all scripts, property-expansions, etc that will occur during the execution of the TestCase allowing you to share resources, data, etc between them. For example, SoapUI uses this internally to store an HttpState object in the context when the "Maintain HTTP Session" option has been selected in the TestCase Options dialog.
  2. The setup script of the TestCase is executed.
  3. If a maximum execution time has been specified in the TestCase options dialog, a corresponding times is started that will terminate (and fail) the TestCase if it hasn't finished in the configured time
  4. A beforeRun event is triggered to all applicable TestRunListeners (defined either externally or as Event Handlers at the project level)
  5. Any WS-RM or AMF related session or transaction initializations are performed as configured under the corresponding tabs in the TestCase Options dialog.

Both 2 and 4 above have the possibility of terminating the TestCase if any external conditions are not met by calling testRunner.cancel or testRunner.fail.

When running from the UI, an entry marking the start of TestCase will be written to the Execution log:

test-started-log-entry

Execution of TestSteps

Ok, the TestCase is all set! Now the actual step-by-step execution of your TestCase takes place starting with the first enabled TestStep (or with the currently selected TestStep if started with the "Run from here" TestStep popup menu action). For each TestStep the following happens:

  1. The beforeTestStep event is triggered to all applicable TestRunListeners
  2. The TestStep itself is run and its TestStepResult is added to the testRunner.results collection
    • If the TestStep itself executes other TestSteps (for example a Run TestCase TestStep or Script TestStep calling testRunner.runTestStepXXX), these are recursively executed as described here
  3. The afterTestStep event is triggered to all applicable TestRunListeners
  4. If running in the UI, the corresponding log is updated with the TestStepResult saved in (2);

teststep-results-in-log

Depending on the outcome and internal logic of the TestStep, the following happens:

  1. If the TestStep fails (due to assertion failures, internal errors, etc) the TestCase is aborted if the "Abort on error" option is checked in the TestCase Options (default setting).
  2. If the TestStep does not fail (or the TestCase continues anyway), the data contained in the TestStepResult is discarded from memory if the TestCase Options "Discard OK Results" option is checked (for example the underlying request and response messages). Checking this option can save a lot of memory when executing long-running tests!
  3. If the current TestStep transfers execution to another testStep via the testRunner.gotoTestStepByXXX method, that TestStep is executed next instead of the one "next in line". The following TestSteps do this internally:
    1. Conditional Goto if a condition is met
    2. DataSource Loop if more data is available

Finishing the TestCase

Once execution passes the last TestStep the TestCase is finished and the following events are triggered:

  1. All TestSteps are internally "finished", which allows them to close any resources, write results to files, etc.
  2. The TearDown script of the TestCase is called, for example for cleaning up any resources or doing final validations.
  3. The TestRunListener.afterRun event is triggered for applicable listeners

The Status of the TestCase will be set to FAILED if any of the following conditions is met:

  • The TestCase was aborted due to an internal error
  • The TestCase was aborted because of a failed TestStep (as described above) or if there were failed TestSteps and the "Fail TestCase on Error" option in the TestCase Options dialog is selected
  • If a script in any of the triggered events, scripts or handlers explicitly failed the TestCase with testRunner.fail

The Run Log once again reflects the final state of the TestCase;

testcase-finished-in-log

Running a LoadTest

When SoapUI runs a LoadTest for your TestCase, it internally creates complete copies of the TestCase for each executing Thread. Each thread then runs the TestCase exactly as described above with a number of LoadTest specific additions:

  • The context available in the TestCase contains a LoadRunContext property holding the context of the LoadTest. Use this to share data or resources between executing threads in your LoadTest
  • LoadTestRunListener events are triggered for TestRunListener counterparts described above, allowing you to create handlers/listeners that operate on a LoadTest level for all running TestCases (or Threads)
  • Any TestStep result that leads to the failure of a LoadTest Assertion is never discarded from memory,
  • Any TestStep result that does not lead to a LoadTest Assertion failure is always discarded from memory if the "Discard OK Results" option is set