Manage test results

All test results are automatically saved after each test execution.

The Results manager screen displays and allows managing the results of a test completed in the current project. It can be accessed with the Tools > Results Manager command.

Columns displayed by default are:

• Status: Status of the test. See General information about test results for more information.
• Name: Name of the test result
• Date: Date of the test result
• Scenario: Name of the scenario
• Errors: Number of errors
• Avg Req/s: Average number of requests per second since test started
• Duration: Duration of the test in HH:MM:SS

Columns hidden by default and available with the button are:

• Author: First and Last name of the person who launched the test
• Description: Description of the test
• Avg Page duration: Average duration in seconds of the pages including think time—not played during the checking process.

Open a test result

In the Results Manager dialog, clicking Open displays the results of the selected test in the Results section of NeoLoad.

Test folder

Clicking on the Test folder button opens the directory containing the test results.

Generate a report

To launch the Generate Report wizard, it is necessary to select the appropriate test result then click Generate report. This wizard allows generating either a simple report or a results comparison report.

• Information: For more information, see Publish a test report.

Remove a test

Clicking on Remove permanently deletes a set of test results.

Deleting superfluous test results is recommended to free up disk space.

• Warning: Removing a set of test results deletes the files stored on the hard disk. This operation is irreversible.

Configure results

The lower part of the Results Manager dialog includes a number of categories which help configure results:

• Details
• Statistics
• SLA Profiles
• Filters
• Comparison
• Import Data

Details

A click on the Details category displays the Details group box.

In Name and Description, the label and descriptive text of the selected test result can be modified.

The Status of a test can also be modified: a test that has been deemed "Passed" can be changed to "Failed" and vice versa.

In the Time group box, the time scale of the x-axis for the graphs can be modified:

• Absolute is the option to display time in calendar dates.
• Relative displays time elapsed from a test start.

• Information: For more information about default graphs options, see Graphs and Advanced settings.

Statistics

Truncated Mean: This area allows obtaining a subset of results with the extremes removed. This excludes non-representative values and provides values close to those obtained by the majority of end users.

• The percentage of results considered pertinent.
• Policy for selected result distribution.
  • Centered range (removes the same number of high and low values).
  • Remove only the highest values.

Percentiles: This area allows you to define three percentile values to be shown in the Test Summary, Values and Graphs panels.

Possible values for these fields are between 0.1 and 99.9 with a 0.1 granularity.

Time: In this group box, the default time scale of the x-axis for the runtime and results graphs can be specified:

• Absolute is the option to display time in calendar dates.
• Relative displays time elapsed from a test start. This is the default display mode.

Service Level Agreements (SLA)

Click the SLA icon to manage the Service Level Agreements linked to the selected test results.

For more information, see Service Level Agreement (SLA) profiles.

Filters

A click on the Filters category allows applying a filter to the selected test results according to different criteria.

• Information: For more information, see Filter test results.

Comparison

A click on the Compare category allows comparing two sets of test results in the Results view.

• Information: For more information, see Compare test results.

Monitor data import

Despite NeoLoad extensive list of monitoring modules, there may be some network constraints which makes monitoring servers difficult (unavailable module in NeoLoad, firewalls, security policies, isolated environments, remote databases, or others).

The NeoLoad Monitoring Data Import function makes it possible to:

• import external CSV monitoring data files
• display monitoring information within the test results and graphs
• correlate performance measures between the NeoLoad data and the monitored servers
• take advantage of all the capabilities of NeoLoad analysis view

Import a monitoring data file

NeoLoad 9.0 can import external monitoring data files from tools like Windows 2000 Performance Monitor, or any other solution generating CSV monitoring file. NeoLoad makes it possible to set side by side as many monitoring reports as necessary.

• Warning: To get meaningful data, the monitoring tool must be active on the tested server when the NeoLoad load test is started. Similarly, an external monitoring file can be imported only when there is a NeoLoad test result for the same period of time.

Before starting the import, the generated monitoring file must be copied on the machine hosting the NeoLoad Controller.

The Import Data function is accessible:

• with a click on Edit > Results Manager, or
• in the Results section tabs with a click on Tools > Import Data.

In the Results Manager screen, the Import Data category is dedicated to importing a monitoring data file.

The Monitoring data file group box allows selecting the file to import and declaring its format:

• File: A click on the Browse button helps select the monitoring file that is available on the machine hosting the Controller.
• Format: The drop-down list helps select a predefined format that populates the File settings group box. When the format is unknown, the Generic CSV option is available.

• Information: The generic CSV format supported by NeoLoad complies with the following framework:
  "<any>"<separator>counterName1<separator>counterName2<separator>...
"<Timestampvalue1><separator><counterValue1><separator><counterValue2><separator>...
• For example:
  "TimeStamp","Memory used (%)","CPU Used (%)"
"12/17/2012 15:52:55.324","57","7"

In the NeoLoad settings group box, the Monitor name field can be completed with a free unique value. The default labels—"Imported monitor" for CSV imports, "Windows PerfMon" for Windows Performance Monitor imports, and "Imported NeoLoad monitor" for NeoLoad imports—are incrementally numbered once imported.

In File settings, the structure of the monitoring file must be defined. When a predefined format is selected in Format, the structure characteristics are set by default and dimmed. When importing a specific monitoring file, the drop-down boxes are all available:

• Date pattern: The date format of the monitoring file should be ISO 8601 compliant. However the field can be edited for custom date formats.
• Column separator: The column separator of the monitoring file can be a comma, a tab, or a semicolon.
• Decimal symbol: The decimal symbol of the monitoring file must be selected: period or comma.

The Times group box is useful when the monitoring file has been recorded in a different time zone or when the server and the controller are not sychronous. The drop-down list makes it possible to Add or Substract as many hours, minutes, and seconds as necessary to synchronize the file and the NeoLoad results. The reference time is the load test time, that is the time on the Controller machine.

• Tip: To identify the time available on the Controller and on the tested server, it is recommended to start a command on both machines at the same time to get their exact dates and times (date and time commands on Windows).

A click on OK starts the import. Once the import is complete, it can be viewed, as documented in View imported monitoring data. NeoLoad automatically detects whether the imported file includes measures from different servers, and displays them distinctively.

• Warning: If a monitoring file is not compatible or if the syntax defined in File settings is invalid, NeoLoad displays a message error.

• To import a monitoring data file

1. Copy the monitoring file from the tested server to the Controller machine.
2. In the NeoLoad Controller, to access the Import Data function:
   • click Edit > Results Manager, or
   • click Results > Tools > Import Data.
3. Click Browse to select the monitoring data file for the File field.
4. In Format, select:
   • a predefined Windows Perfmon or NeoLoad file format, or
   • Generic CSV.
5. In Monitor name, you can enter a custom label to view the counter easily once imported.
6. When necessary, define the structure of the file:
   • In Date pattern, select or enter the date format of the monitoring file.
   • In Column separator, select whether it is a comma, a tab, or a semicolon.
   • In Decimal symbol, select whether it is a period or a comma.
7. In Times, define the time gap between the tested server and the Controller machine, if any.
8. Click OK.

View imported monitoring data

NeoLoad can read external monitoring files and integrate the results into the analysis views of the Results section. The monitors are identified with the name of tested server and the name of the file specified at the import.

In the Test Summary tab, the Monitors section displays the Performance Counters of the imported monitoring files together with the NeoLoad Performance Counters.

In the Values tab, it is necessary to select Counters in the Type drop-down list to have the Performance Counters of the imported monitoring files together with the NeoLoad Performance Counters.

In the Graphs tab, a click on the Monitors sub-tab gives access to all the measures of the imported monitors together with the NeoLoad monitors.