September 2021 Release

What’s New in NeoLoad 7.11?

Easy, more flexible burst licensing options

NeoLoad is licensed on a virtual user (VU) basis, meaning each license accounts for how many VUs can be generated as load to stress a system under test. The VU licenses are used ongoing for continuous performance testing. To account for the use case where teams need to have less frequent very high load testing, NeoLoad now also offers complementary virtual user hour (VUH) licensing to bring high flexibility for occasional high volume tests.VUH licenses can be added to a VU base license to optimize license cost for a broad range of use cases. Examples include spikes in load from Black Friday shopping or during open enrollment season for insurance companies.

VU and VUH licenses can be managed, administered and used dynamically from NeoLoad. With VUH activated, you can set a number of VUs in a test higher than your VU license quota, and NeoLoad will dynamically compute the number of VUH needed “on top” of your VU license.For instance, if you want to do a one off 2,500 VU test but your VU license is only for 1,000, you can use 1,500 VUH licenses to combine with the existing 1,000 licenses to reach the 2,500 VU test.

You can also check NeoLoad to see how many VUH licenses are left to be consumed for testing.

VUH licensing is available for all web protocols and the Citrix protocol. VUH licensing for SAP GUI is not available.

Learn more with the VUH guide

Enhancements

Direct project export to NeoLoad Web
NeoLoad GUI can now directly export an optimized project file to an existing NeoLoad Web test to reduce the time and effort to test an updated project.

Tricentis qTest integration
We introduced a new option to create defects in qTest when NeoLoad SLAs are breached. The URL to qTest that is inserted in NeoLoad reports now targets the qTest test run.

Tricentis TOSCA integration
Tosca can send end user experience metrics to NeoLoad. Compatible with Tosca 14.3.

Dynatrace integration enhancements
• Dynamic monitoring now creates counters during a test to observe and optimize within auto-scaling systems. Now any change in the infrastructure under test is reflected in NeoLoad. For example, when a new container is started during a load test session, it will be added to NeoLoad, and in a case where the environment is sized down, the chart corresponding to the component (a Docker container) is stopped.
• Other Dynatrace enhancements include an option to handle self-signed certificates for a Dynatrace instance deployed on-premises and the ability to set a proxy specifically for the
Dynatrace integration.

Dashboard improvements
The notification displayed in case of concurrent modification now displays the user name of the other editor.

NeoLoad Web API
Download reports as PDF with “POST /v3/workspaces/{workspaceId}/test-results/{resultId}/report”. See the NeoLoad Web API documentation.

SAP GUI
NeoLoad 64 bit now supports the 64 bit version of SAP GUI for Windows so that you can design and run SAP GUI tests without the memory limitation of the 32 bit version.

What’s New in NeoLoad 7.10?

Tricentis qTest Native Integration

NeoLoad and qTest now have native integration which delivers customers the ability to manage their entire test suite from one view. Users can now view performance tests results and status within qTest Manager, allowing all stakeholders using qTest to manage performance tests and release KPIs side-by-side with functional test management. 

You’ll now be able to navigate between qTest and NeoLoad Test Results easily for analysis where your management statistics will now encompass performance tests. 

Dashboard Notifications

When users are modifying the same dashboard concurrently, NeoLoad sends a notification to those users that an update is necessary. Also, the Dashboards view now shows the user name and time stamp of who last edited it and when. 

CSV Export  

You can export the content of the table displayed in the values tab to a CSV file. 

Tosca Integration Update 

NeoLoad’s Tosca integration is now compatible with Tosca version 14.2. When reusing SAP GUI functional tests from Tosca as performance tests in NeoLoad, the integration can now automatically create a NeoLoad transaction for each Tosca test case, making both design and analysis easier in NeoLoad.  

New Citrix Licensing 

Citrix licensing in NeoLoad has changed. Now licenses to test Citrix are included in the web VU licenses and Citrix specific licenses are no longer required and will not work for Citrix-virtualized application testing. NeoLoad 7.10 requires the installation of a new license key to test Citrix, one has been generated in your customer area. Simply use Web VUs to test Citrix with this new key.  

Enhancements

Personal Access Token for Git authentication
GitHub is modifying its security policies and will disallow the use of “password” for authenticating with its API. NeoLoad Web now supports Personal Access Token. 

Encoding for Framework Parameters
Variables Extractors could already handle the extraction of values which are HTML, URL or custom encoded. Now Framework Parameters can also handle this encoding setting. 

Java 11
NeoLoad is now based on Java 11. It means that your custom Advanced Actions or Java libraries now may rely on Java 11 features. 

Hidden Access Tokens 
Access tokens are now hidden by default in the profile section of NeoLoad Web and in the preferences panel of NeoLoad. This improves confidentiality during screensharing sessions. 

Easily retrieve user’s workspaces
User settings now allow admin to easily view the workspace list a user belongs to. 

View reservation author
The reservation tooltip in NeoLoad Web now displays the user name who created the reservation. 

April 2021 Release

What’s New in NeoLoad 7.9?

Automatic License Leasing at Runtime

We have streamlined the runtime experience when using Neoload Web as your licence server.
With the auto-lease option activated, you won’t have to manually lease from your licence to run tests — Neoload will manage the lease/release process for you.

Click on image to enlarge

If you have multiple licenses, you can easily toggle the option to lease from your license manually as you do today, to remain compatible with your current workflows.

Real-time Reporting Collaboration

Sharing a dashboard through a URL
In addition to the recent export-as-PDF feature, you can now share your dashboard with colleagues, stakeholders and clients through a URL to ensure that everyone always has the most up-to-date report.

Click on image to enlarge

List of Dashboards
In addition to the update to our dashboard reporting, now you can now access, sort and filter your entire collection of dashboards from a single page.

Click on image to enlarge

Dynatrace Integration Enhancements

The Dynatrace integration has been enhanced with numerous targeted updates for improved user experience of first time configurations and usage.

See improvements for the initial integration setup on the screenshot below, including:

• Proxy management to access a specific Dynatrace instance
• Token obfuscation and encrypted storage for enhanced security
• Client side validation of API token rights/scope issue detection during connection test

Click on image to enlarge

Tag coverage verification during configuration
The tag validator helps you verify that your tag combination matches the desired number of Dynatrace entities you want to monitor for your scenario, helping you avoid configuration issues.

Before verification

After verification

Python CLI Improvements

Now you can easily export test results data in structured format (.csv, JSON, simple HTML) directly from the Python CLI, allowing you to simplify interfacing with other systems or specific data processing in your pipelines.

Simple export of transaction related data into csv format in the CLI

See the dedicated section in the online documentation for more information.

Other Enhancements

Configuration-free controller to easily connect a controller to NeoLoad Web

Click on image to enlarge

• Enhancements to the Microsoft dynamic monitors:
  • You can now re-edit the regular expressions for your instances.
  • We added a specific icon for dynamic monitors using regular expressions to easily recognize them.
• Percentiles are now updated at runtime on NeoLoad Web

February 2021 Release

What’s New in NeoLoad 7.8?

Dynamic Monitor Support

NeoLoad supports the ability to retrieve metrics from a broad range of technologies including APMs, operating systems, application servers, web servers, databases, network devices, and now – monitoring systems. NeoLoad’s support for dynamic monitors will allow you to get and analyze metrics from your auto-scaling systems.

Compatible monitors will also now generate new counters during test runs.
See below how we leverage this new capability within the Prometheus and Microsoft-based monitors.

Dynamic Prometheus Monitor

The NeoLoad Prometheus monitor now creates counters all along the run phase of your tests, allowing you to retrieve and store metrics needed in some high-value use cases like sizing your auto-scaling systems.

These metrics are as usual available real time while the test is running in the Controller and in the NeoLoad web platform.

Examples of monitor counters being created dynamically during the RUN phase:

Click on image to enlarge

Dynamic Microsoft Monitors

The Microsoft monitors (Windows, SQL Server, IIS, .NET) using instances now create counters all along the run phase of your tests. When you specify a regular expression, all instances matching this expression (including ones spawned during your tests) will return metrics that you can analyze in your dashboards.

Configuration example for dynamic regular expressions counters being created dynamically during the RUN phase:

Click on image to enlarge

Download a Dashboard in PDF

Any dashboard can now be exported as a PDF. Once your report is designed you can download it as a PDF file in order to share it with collaborators and stakeholders or you may archive it.

Click on image to enlarge

Edit Test Scenarios in NeoLoad Web Platform

You can now define and edit settings of your test scenario directly from the NeoLoad Web Platform. For each population defined in your project you can set the number of virtual users, the test duration and the load profile (constant or ramp-up).

The new visual preview allows you to create the load profile you are looking for.

Click on image to enlarge

You can also still run your scenario as defined in your uploaded project as usual.

Click on image to enlarge

Requirements

• Test execution requires at least a Controller from the 2020 October release (7.6)
• The test must be newly created or its project needs to be reuploaded.

Other Enhancements

• [Controller] The integration with version control systems (GIT and SVN) is always enabled. Enterprise users do not need anymore to lease a license from the license server to use this feature.
• [Controller] Command line tests using the Neotys Cloud Platform are now compatible with SSO. Command line tests accept an authentication token in addition to the existing login/password authentication method.

December 2020 Release

What’s New in NeoLoad 7.7?

Dashboards: Preconfigured Reports

Standard reports are now available from the NeoLoad dashboard. You can access the standard, or “canned”, reports directly from the test overview or from the dashboards section. Once generated, the standard report can be customized.

Click on image to enlarge

Export Data Tables as CSV

Export a data table into a CSV formatted file.

Single Sign-on (SSO)

NeoLoad SaaS login is now configurable to use Single Sign-on (SAML).  The neotys.com login is also SSO enabled to access Neotys portal (Neotys Academy, Neotys Labs, etc.). If you are on a SaaS Enterprise plan, contact support to get set up.

Click on image to enlarge

Prometheus Monitor

Neoload can now natively connect to your Prometheus monitoring system and get all the relevant metrics you need for your performance analysis.

Create real time visibility into multiple Prometheus requests in the Neoload Prometheus Monitor.

Click on image to enlarge

Structure the values of your metrics with labels and legends to customize how you analyze the application.

Redefine existing Prometheus monitors with as-code variables to achieve a high-degree of automation.

The promQL requests are resolved during the initialization phase of the run to define the counters list.

Manage Instances for Microsoft-family Monitors

To avoid reconfigurations between test runs, you can now specify instances with matching regular expressions instead of picking them in the instances list. This is particularly useful with instances that change names between runs and that answer to a naming pattern in dynamic environments (.e.g. Hyper-V Network Adapters, IIS process names etc.).

Click on image to enlarge

The regular expressions are resolved during the initialization phase of the run to define the counters list.

Dynamic Infrastructure Improvements

Check Parameters

Test providers’ settings from the settings panel.

Click on image to enlarge

Cluster Logs

Cluster events now appear in the logs.

Click on image to enlarge

Labels

Customize labels for the orchestrator of your choice: this allows for integration with all your enterprise systems that use labels as criteria, such as self-service or billing systems.

Click on image to enlarge

Python CLI: Bamboo CI integration examples

Find ready-to-use examples of Bamboo CI configurations amongst the growing library of pre-configured Neoload Python CLI examples (Azure DevOps, Gitlab, AWS CodePipeline…).

Use Bamboo as-code V1 and V2 scripts to connect to your Neoload platform, run your tests and automatically decide to pass/fail your CI pipeline according to your SLAs.

Have a look!

Click on image to enlarge

Other Enhancements

• [API]: SCIM-compatible Delete user endpoint: SaaS account admins and on-prem admins are now able to programmatically remove all roles or delete a user from their Neotys user base through an API endpoint. The endpoint is SCIM compatible, making it possible to integrate it into an Identity Management system if needed. Configurations using Neoload LDAP connector can not use this mechanism as the user provisioning is managed by the LDAP directory.
• [Controller] Case insensitive flags – When designing a test, an option allows you to flag requests based on a case insensitive search.
• [Controller] Automatic login to the Cloud Platform – As long as the Controller is configured to connect to NeoLoad Web SaaS then the authentication token will be used to connect to the Cloud Platform instead of requesting a login/password. Login and Password are still required when the Controller is connected to an on-premise NeoLoad Web.
• Java 11 Support (preview) – A technical preview of the Controller and Load Generators are available if your Java code or libraries used by NeoLoad needs Java 11. Contact the support to request access to the preview. 
• [API]: Get all workspaces: Admin now get a list of all workspaces, whether they are a member of it or not. 
• SaaS Account Identifier – The SaaS Account Identifier is now displayed next to the account name in the user’s profile page.

October 2020 Release

What’s New in NeoLoad 7.6?

Dashboard Improvements

New analysis capabilities

Dashboards now include tables and text-editable comment fields to organize analysis. These new analysis capabilities come in addition to previously released line graphs.

Click on image to enlarge

Download graphs as PNG

When analysing your test results through dashboards you can share a specific graph by downloading it as a PNG file.

API Improvements

Export raw data

Extract raw data from Neoload Web through the Rest API.
See GetTestRawResult and GetTestRawResultElement.

Custom events

When synchronizing test scenarios with external processes (monitoring, chaos) you can declare external events that will appear in Neoload Web test results values.
See PostTestResultEvents.

CRUD-based Workspace management 

Programmatically manage (create, update, delete) your workspaces and their settings like quotas and members.
See Workspace Management API

JavaScript API to stop a test

Use the JavaScript API to stop a test in the event where a virtual user experiences an issue during a test. 
See RuntimeContext.stopTest()

As-Code Improvements

Assertions

Use as-code based tests to validate server response to ensure that the test is running as expected.
See assert_content on a request.

Advanced setting for a scenario

Define advanced settings for an as-code scenario like monitoring options, rendezvous policies, excluded URLs, or APM configuration.

Custom Load Policy

You can define an as-code Custom Load Policy.

Search on Tests

Search test results by their test name in the advanced search window or in the test result picker on the Dashboard.

There’s also a shortcut to retrieve the test results related to a test.

Click on image to enlarge

Other Enhancements

• Support of client certificate for Websocket
• [Dynatrace native integration] Several fixes for large scale use cases
• [Silverlight] Ability to disable base64 encoding in case it contains clear text needed for correlation
• [Citrix] TextGet action inserted by Citrix capture helper contains the recorded response
• Compare response content with recorded content for custom actions in the Check Virtual User dialog.
• Testers, not only Admins, can see the roles of members of the Workspace.

August 2020 Release

What’s New in NeoLoad 7.5?

Workspaces

Overview

NeoLoad now introduces a new capability called Workspaces. Workspaces allow administrators to group users into teams and assign quotas for virtual users (see more below in the license quota section). Workspaces enable teams to organize and collaborate on their performance testing tasks, assets and results.

Workspaces enable people to separate assets such as tests, test results and webhooks in a designated Workspace and associated team members or share assets among multiple Workspaces and teams.

Users may belong to several workspaces but will only see the contents of one Workspace at a time.

When running tests from NeoLoad Web, the Zone’s hosting Controllers and Load Generators are shared between all Workspaces to optimize resource usage. You may use the Reservation feature to control the access to these shared resources and guarantee that they are available for important planned tests.

Quotas on Licensed Virtual Users

Administrators can define for each Workspace a quota to limit the maximum number of Virtual Users of the license that all members of the workspace can use.

The quota is checked for tests run by NeoLoad Web as well as for independent Controllers that use NeoLoad Web as their license server.

Click on image to enlarge

Reorganized Settings

The Settings panels have been reorganized so that all settings at Account level are in a single location (for Admin only) and the settings related to a Workspace are grouped together.

New API

The NeoLoad Web API version 3 handles Workspaces. Previous versions of the API support lists of assets and creation of assets in the default Workspace.

Click on image to enlarge

See the API definition.

Moving Assets

NeoLoad Web has the capability to move assets like tests and test results from one Workspace to another. This allows you to organize your current test assets to new Workspaces as you create them.

Dynatrace Native Integration

Dynatrace and NeoLoad are now natively integrated in a bidirectional way, allowing users to push Dynatrace metrics into Neoload, or NeoLoad metrics into Dynatrace, native monitors through a very simple configuration. Once configured, this integration discovers the system under test in the Dynatrace Smartscape and pushes Neoload generated performance engineering context and metrics into Dynatrace itself, or Dynatrace metrics into NeoLoad, allowing more powerful analysis in both NeoLoad and Dynatrace.
Choosing easily which Dynatrace metrics will be pulled in Neoload

Click on image to enlarge

Dynatrace metrics within a NeoLoad Web Dashboard

Click on image to enlarge

NeoLoad metrics within Dynatrace Dashboards

Click on image to enlarge

Neoload Event in Dynatrace

Click on image to enlarge

Tosca Integration v2.2

NeoLoad’s integration with Tosca already supports the conversion of any Tosca web test, including SAP web tests, and Tosca SAP GUI tests. The NeoLoad-Tosca integration now supports a new Tosca test type, API testing. And, the integration is also now compatible with the latest Tosca release, Tosca 13.3

Other Enhancements

• Minor GUI enhancements when defining the variables to be exported with the raw data.
• MySQL driver upgraded to the latest version

June 2020 Release

What’s New in NeoLoad 7.4?

Command Line Interface 

The Python CLI is a client to the runtime API of NeoLoad Web (SaaS or on-premise) that allows  you to define and control a test from anywhere a Python command line is accessible. Using the Python CLI for NeoLoad is a simple and speedy approach for automating performance testing in CI pipelines.

For instance to: 

1. Install the CLI client
2. Connect to your Neoload Platform (in the example, the default is our SaaS platform)
3. Create and configure a test
4. Associate a project file
5. Run your test

All you need to do is:

Click on image to enlarge

The CLI can be combined with NeoLoad’s performance testing as-code capabilities in any of your automation environments.

A comprehensive documentation is available on the Neotys GitHub repository: https://github.com/Neotys-Labs/neoload-cli 

Preconfigured popular CI pipeline examples (such as Jenkins, Azure Devops or GitLab) are also provided to get started even faster: https://github.com/Neotys-Labs/neoload-cli/tree/master/examples/pipelines

Filtering Result Values 

When analyzing results in NeoLoad Web, you can filter Transactions, Pages and Requests by Zones, Populations and User Paths, to focus your analysis for greater accuracy.

Click on image to enlarge

Custom Fields for Raw Data

When exporting raw data for transactions from the NeoLoad Controller, you now can add custom fields to tell in which context the execution of the transaction occurred. As an example, you will not only see which specific transaction was slow but you can also see which data was used (example ContractID for each “Edit Contract” transaction).

See the tutorial on exporting contextualized raw data.

Click on image to enlarge

The following image illustrates an example of NeoLoad load testing data exported to an analytics platform:

Click on image to enlarge

Dynamic Infrastructure Improvements

Neotys adds Google GKE to the list of supported dynamic infrastructure providers for NeoLoad Web: OpenShift, Kubernetes, Microsoft AKS and Amazon EKS.

Also, NeoLoad Web now can launch tests using dynamic infrastructure even when the reservation feature is enabled on the account. In this case, you need to use auto-reservation, since planned reservations are not supported for dynamic infrastructure.

Dynamic zones settings are now more flexible and powerful allowing to switch from one cluster to another.

Tosca Integration v2.1

Support for Tricentis Tosca web tests has been enhanced with the automatic creation of a Neoload Transaction for each Tosca step in the converted testcase. With automated userpath update, when you make a change to your Tosca testcase, Neoload merges the new user path with the updated one, keeping all the changes you made to your initial NL userpath such as variables, loops, SLA and so on.

Click on image to enlarge

This version also brings support for Tosca 13.1 SP3.

Other Enhancements

• Citrix improvement – Random popup, asynchronous notifications or potential warnings can interrupt your Citrix scenario. Neoload now manages these intermittent popups with a simple Fork action in your design.
• NeoLoad Web Licensing improvement – A license now can be leased in parallel by two Controllers having the same hostname which may occur when starting Controllers in different Kubernetes pods.

Compatibility Notes

• New project version – Projects designed with NeoLoad 7.4 can only be opened by NeoLoad 7.4 and more. It means that you need to upgrade your load generators to be able to run projects designed with this new release. Note that the new Load Generators will be able to run your older projects.