It is important to check that the server response is valid under load. This is to make sure not only that the scenario works as expected, but also that the load imposed does not cause errors in the application
NeoLoad automatically detects requests containing errors, in particular using the HTTP response code. For example, NeoLoad will log a request error if the response contains '500 internal error'. However, many web applications do not return the appropriate HTTP error code when handling an error, which means that for these, NeoLoad cannot automatically detect the fault.
These cases must be processed individually by checking the validity of the content returned by the server at key points within the application. This may entail checking for the presence of an expected text, for example "The operation was successful", or making sure the response does not contain "Error".
To define a container value:
In the Repository, click on the request whose server response requires validating.
Click on the "Validation..." button
underneath, to the right.
Add a Validation for the content.
Select a validation method: contains or does not contain a simple text, or use a regular expression.
Once the test is finished, select the requests containing errors, or those where the validation failed, in the errors panel. The content of the corresponding server response may then be analyzed to determine the cause of the problem.
Note that validation uses up resources (CPU, memory) on the Load generator, and that it is usually wiser to restrict its use to testing key pages only (for example, those that provide access to a database, or those more likely to produce errors).
Proceed step by step:
Use the Check
function to validate the scenario for a Virtual User type. This will
play the Virtual User once, displaying the details of the sent
request and the server response to each HTTP request.
Begin with a short test and a light load. Correct any application errors occurring under the light load before carrying out tests using heavier loads.
When a Virtual User receives an error, it should normally stop running. If this does not happen, it could continue playing requests that have no meaning. For example, if the user login fails, there is little point sending further browsing or search requests to the application as it will only distort the response time statistics for those pages.
Each Virtual User type may be configured to stop running in case of error or failed assertion.
An efficient way to highlight the performance variations between two sets of test results is to plot the statistics for a same item (page, request, Container) in both tests simultaneously.
This provides a visual comparison of the application's behavior under different scenarios, or further to its modification (e.g. update or optimization).
NeoLoad provides two advanced statistics:
Standard deviation - allows the variation in values to be measured, compared to the average. A high standard deviation indicates that response times vary widely, whereas a low standard variation is a sign that response times are consistent and even.
Average 90% - calculated by removing the highest and lowest 5% of values. This eliminates any outliers and produces values that are close to those that will be obtained by most end users.
After multiple test runs, the volume of data can become difficult to manage. It is important therefore to add a short description to each test prior to its running. This description is included in each test summary and the reports generated.
The Results Manager allows the user to delete the results of a previous test session or use them to generate a report (XML, HTML, PDF).