October 2020 Release

What’s New in NeoLoad 7.6?

Dashboard Improvements

New analysis capabilities

Dashboards now include tables and text-editable comment fields to organize analysis. These new analysis capabilities come in addition to previously released line graphs.

Click on image to enlarge

Download graphs as PNG

When analysing your test results through dashboards you can share a specific graph by downloading it as a PNG file.

API Improvements

Export raw data

Extract raw data from Neoload Web through the Rest API.
See GetTestRawResult and GetTestRawResultElement.

Custom events

When synchronizing test scenarios with external processes (monitoring, chaos) you can declare external events that will appear in Neoload Web test results values.
See PostTestResultEvents.

CRUD-based Workspace management 

Programmatically manage (create, update, delete) your workspaces and their settings like quotas and members.
See Workspace Management API

JavaScript API to stop a test

Use the JavaScript API to stop a test in the event where a virtual user experiences an issue during a test. 
See RuntimeContext.stopTest()

As-Code Improvements

Assertions

Use as-code based tests to validate server response to ensure that the test is running as expected.
See assert_content on a request.

Advanced setting for a scenario

Define advanced settings for an as-code scenario like monitoring options, rendezvous policies, excluded URLs, or APM configuration.

Custom Load Policy

You can define an as-code Custom Load Policy.

Search on Tests

Search test results by their test name in the advanced search window or in the test result picker on the Dashboard.

There’s also a shortcut to retrieve the test results related to a test.

Click on image to enlarge

Other Enhancements

• Support of client certificate for Websocket
• [Dynatrace native integration] Several fixes for large scale use cases
• [Silverlight] Ability to disable base64 encoding in case it contains clear text needed for correlation
• [Citrix] TextGet action inserted by Citrix capture helper contains the recorded response
• Compare response content with recorded content for custom actions in the Check Virtual User dialog.
• Testers, not only Admins, can see the roles of members of the Workspace.

August 2020 Release

What’s New in NeoLoad 7.5?

Workspaces

Overview

NeoLoad now introduces a new capability called Workspaces. Workspaces allow administrators to group users into teams and assign quotas for virtual users (see more below in the license quota section). Workspaces enable teams to organize and collaborate on their performance testing tasks, assets and results.

Workspaces enable people to separate assets such as tests, test results and webhooks in a designated Workspace and associated team members or share assets among multiple Workspaces and teams.

Users may belong to several workspaces but will only see the contents of one Workspace at a time.

When running tests from NeoLoad Web, the Zone’s hosting Controllers and Load Generators are shared between all Workspaces to optimize resource usage. You may use the Reservation feature to control the access to these shared resources and guarantee that they are available for important planned tests.

Quotas on Licensed Virtual Users

Administrators can define for each Workspace a quota to limit the maximum number of Virtual Users of the license that all members of the workspace can use.

The quota is checked for tests run by NeoLoad Web as well as for independent Controllers that use NeoLoad Web as their license server.

Click on image to enlarge

Reorganized Settings

The Settings panels have been reorganized so that all settings at Account level are in a single location (for Admin only) and the settings related to a Workspace are grouped together.

New API

The NeoLoad Web API version 3 handles Workspaces. Previous versions of the API support lists of assets and creation of assets in the default Workspace.

Click on image to enlarge

See the API definition.

Moving Assets

NeoLoad Web has the capability to move assets like tests and test results from one Workspace to another. This allows you to organize your current test assets to new Workspaces as you create them.

Dynatrace Native Integration

Dynatrace and NeoLoad are now natively integrated in a bidirectional way, allowing users to push Dynatrace metrics into Neoload, or NeoLoad metrics into Dynatrace, native monitors through a very simple configuration. Once configured, this integration discovers the system under test in the Dynatrace Smartscape and pushes Neoload generated performance engineering context and metrics into Dynatrace itself, or Dynatrace metrics into NeoLoad, allowing more powerful analysis in both NeoLoad and Dynatrace.
Choosing easily which Dynatrace metrics will be pulled in Neoload

Click on image to enlarge

Dynatrace metrics within a NeoLoad Web Dashboard

Click on image to enlarge

NeoLoad metrics within Dynatrace Dashboards

Click on image to enlarge

Neoload Event in Dynatrace

Click on image to enlarge

Tosca Integration v2.2

NeoLoad’s integration with Tosca already supports the conversion of any Tosca web test, including SAP web tests, and Tosca SAP GUI tests. The NeoLoad-Tosca integration now supports a new Tosca test type, API testing. And, the integration is also now compatible with the latest Tosca release, Tosca 13.3

Other Enhancements

• Minor GUI enhancements when defining the variables to be exported with the raw data.
• MySQL driver upgraded to the latest version

June 2020 Release

What’s New in NeoLoad 7.4?

Command Line Interface 

The Python CLI is a client to the runtime API of NeoLoad Web (SaaS or on-premise) that allows  you to define and control a test from anywhere a Python command line is accessible. Using the Python CLI for NeoLoad is a simple and speedy approach for automating performance testing in CI pipelines.

For instance to: 

1. Install the CLI client
2. Connect to your Neoload Platform (in the example, the default is our SaaS platform)
3. Create and configure a test
4. Associate a project file
5. Run your test

All you need to do is:

Click on image to enlarge

The CLI can be combined with NeoLoad’s performance testing as-code capabilities in any of your automation environments.

A comprehensive documentation is available on the Neotys GitHub repository: https://github.com/Neotys-Labs/neoload-cli 

Preconfigured popular CI pipeline examples (such as Jenkins, Azure Devops or GitLab) are also provided to get started even faster: https://github.com/Neotys-Labs/neoload-cli/tree/master/examples/pipelines

Filtering Result Values 

When analyzing results in NeoLoad Web, you can filter Transactions, Pages and Requests by Zones, Populations and User Paths, to focus your analysis for greater accuracy.

Click on image to enlarge

Custom Fields for Raw Data

When exporting raw data for transactions from the NeoLoad Controller, you now can add custom fields to tell in which context the execution of the transaction occurred. As an example, you will not only see which specific transaction was slow but you can also see which data was used (example ContractID for each “Edit Contract” transaction).

See the tutorial on exporting contextualized raw data.

Click on image to enlarge

The following image illustrates an example of NeoLoad load testing data exported to an analytics platform:

Click on image to enlarge

Dynamic Infrastructure Improvements

Neotys adds Google GKE to the list of supported dynamic infrastructure providers for NeoLoad Web: OpenShift, Kubernetes, Microsoft AKS and Amazon EKS.

Also, NeoLoad Web now can launch tests using dynamic infrastructure even when the reservation feature is enabled on the account. In this case, you need to use auto-reservation, since planned reservations are not supported for dynamic infrastructure.

Dynamic zones settings are now more flexible and powerful allowing to switch from one cluster to another.

Tosca Integration v2.1

Support for Tricentis Tosca web tests has been enhanced with the automatic creation of a Neoload Transaction for each Tosca step in the converted testcase. With automated userpath update, when you make a change to your Tosca testcase, Neoload merges the new user path with the updated one, keeping all the changes you made to your initial NL userpath such as variables, loops, SLA and so on.

Click on image to enlarge

This version also brings support for Tosca 13.1 SP3.

Other Enhancements

• Citrix improvement – Random popup, asynchronous notifications or potential warnings can interrupt your Citrix scenario. Neoload now manages these intermittent popups with a simple Fork action in your design.
• NeoLoad Web Licensing improvement – A license now can be leased in parallel by two Controllers having the same hostname which may occur when starting Controllers in different Kubernetes pods.

Compatibility Notes

• New project version – Projects designed with NeoLoad 7.4 can only be opened by NeoLoad 7.4 and more. It means that you need to upgrade your load generators to be able to run projects designed with this new release. Note that the new Load Generators will be able to run your older projects.

April 2020 Release

What’s New in NeoLoad 7.3?

Dynamic Test Infrastructure with Kubernetes, AKS, and EKS

NeoLoad’s Dynamic Infrastructure is the capability to interact with container orchestrators, currently OpenShift, Kubernetes, Microsoft AKS and Amazon EKS, to provision and use load testing infrastructure automatically, on demand.

When a test is run through NeoLoad Web, controllers and load generators are spun up for the duration of the test and released once the test is finished. The load testing resources are automatically provisioned with pre-set settings within a minute of executing a test on demand and the Docker containers are shut down and released once the test is complete. NeoLoad leaves the cluster automatically cleared and ready for any use. 

Each zone now can be customized in terms of CPU, memory, Docker image whether your cluster is managed using a cloud orchestrator or self managed using Kubernetes directly.

Click on image to enlarge

Click on image to enlarge

Tricentis Tosca Web Tests as Performance Tests

Previously, NeoLoad released the capability to convert SAP GUI functional tests created in Tosca, to performance tests in NeoLoad. Now, NeoLoad can convert any web tests (not limited to SAP) created in Tosca too. With one click, Tosca web or SAP GUI functional tests can be exported and converted into performance tests. 

Support for SAP GUI test analysis has been enhanced. When exporting or re-exporting an SAP GUI test, you now have the option to automatically create a Neoload Transaction for each SAP TCode detected in your test.

The install process has also been simplified.

Citrix Enhancements

NeoLoad contains several improvements to ease recording and maintenance of Citrix applications testing:

• Update recorded content now effective for Citrix
• Additional tolerance level for image detection
• Constant variable for Citrix timeout
• Support of NoPrivileges LG launcher for Terminal Services
• Set back focus to Citrix client after NL citrix capture
• Screenshot comparison panel to help identify differences between the recording and the playback.

Click on image to enlarge

New public API v2 for NeoLoad Web

A new version of the public API now enables a much more powerful and more flexible automation of tests in the context of continuous integration and continuous load testing.

The API has been enhanced to leverage the recent concept of Tests in NeoLoad Web. It allows you to automatically run a Test that has been defined manually in NeoLoad Web’s interface or fully automate the whole process from creating a Test, uploading the NeoLoad project and running it.

Changes to the API

• New Tests related functions (create, update, upload project, list, delete)
• Running a test is now on the Test (/tests/{testId}/start) 
• Ability to stop a running test  (/test-results/{resultId}/stop) 
• Test result related functions renamed from /tests/ to /test-results/

See the API documentation.

Other Enhancements

• Force release – When it’s not possible to release a license lease from NeoLoad Web normally, you can now request Neotys support to unlock the lease.
• When launching a test in the context of a reservation for a group, only group members can stop the test in NeoLoad Web.
• New JUnit Report Format – NeoLoad can generate a JUnit report for the SLA results for command line tests. When using ‘-SLAJUnitResults’ the old format is kept for backward compatibility. When using ‘-SLAJunitReport’, the XML grammar has been fixed to be compatible with more tools handling JUnit format and the way JUnit tests are named and structured is more actionable.
• NeoLoad Web users now have the ability to pick licensed Citrix or SAP VUs to run their Web tests.
• The screenshot comparison panel now helps identify differences between the recording, the playback or debugging – for any action that has a screenshot (Web Pages, Citrix, SAP GUI …).

January 2020 Release

What’s New in NeoLoad 7.2?

Citrix Support

NeoLoad now can record and replay Citrix deployed applications. The set of Citrix actions is compact and consistent for better readability and easier script maintenance. Design faster with NeoLoad’s live detection of texts and get a more reliable text recognition at runtime with modern and configurable Optical Character Recognition (OCR) engine (based on Neural Networks).

See the Citrix Testing Guide.

Changes since the beta version:

• Windows Interactive Logon is now possible.
• Various UI improvements and bug fixes.

License Key Management

Click on image to enlarge

Split Licenses Between Several NeoLoad Web Accounts
An Enterprise License can now be split between several NeoLoad Web Accounts and/or instances (On-premise or SaaS) for a given duration and a given number of licensed VUs.

Learn more in the documentation.

Offline Lease with NeoLoad Web
In order to test from isolated environments, a NeoLoad Controller can already request a lease from an Enterprise License installed in Neotys Team Server with an offline process.

A NeoLoad Controller now can also request an offline lease from an Enterprise License installed in NeoLoad Web.

Learn more in the documentation.

Other Enhancements

• Some default settings when creating an element can now be overridden and shared, for example the error handling policy for a VU Path or the stop policy of a Population at runtime. See the ‘Advanced’ section of the general settings.
• The NeoLoad Web’ Tests page displays new columns: ‘‘updated by’, ‘last modified’, ‘last run’’
• NeoLoad Web’s graph legend now includes the Test name.
• New NeoLoad Web Resources API to access the list of Reservations and the list and the content of Zones. See the API Definition.
• The bundled Controller Java VM is now an OpenJDK which has an impact on how to install NeoLoad in a headless Linux environment.
• The default ‘Change Policy’ setting for new List, File and Sql variables has been changed from “For each Virtual User instance” to “Each Iteration”.
• Browser Profiles picker has been updated with the latest browsers.
• You now can sort test results returned by Neoload Web API. See the API Definition
• Neoload Web now officially supports MongoDB 3.6.

Note About Installation

When installing NeoLoad with the command line in a headless environment, NeoLoad 7.2 has new requirements. See how to install NeoLoad in a headless Linux environment.

Bug Fixes

LOAD-19894
[QFT] Project settings/Runtime parameters problem

LOAD-19889
[NLW > UPLOAD PROJECT] Scenario summary are missing when user creates a new test

LOAD-19873
An offline lease can be installed as a License and so used while it has been released

LOAD-19872
Starting Test screen: space missing for ‘Number of Virtual Users’

LOAD-19867
[offline ]Another Admin user cannot use the generated rls file to release on the Master

LOAD-19837
NLG – Dashboard are created 2 times when opening a project saved using XML_MULTI format

LOAD-19674
[NLW] ${runID} should start at 1.

LOAD-19666
Force JS Variable initialisation to be evaluated to make sure no value stays in cache

LOAD-19483
SOAP parameter “no encode” value does not work as expected

LOAD-17905
[NLW Doc] – OnPremise – In SSL deployments, max file upload size documentation should also talk about nginx limitation.

LOAD-17891
The NeoLoad as code feature does not expect extension file in upper case but only in lower case

November 2019 Release

What’s New in NeoLoad 7.1?

NeoLoad 7.1 marks NeoLoad’s beta support of Citrix performance testing capability, new continuous integration extensions to Azure DevOps, GitLab and AWS CodeBuild and updates to test management and transaction data. Tests setting can now be saved in NeoLoad web, shared and reused. Tests can also be scheduled to be run during off-hours. Test reports and dashboards now display the 50th, 90th, 95th and 99th percentiles metrics for transactions.

Citrix Support

NeoLoad now can record and replay Citrix deployed applications. The set of Citrix actions is compact and consistent for a better readability and easier to maintain scripts. Design faster with a live detection of texts and get a more reliable text recognition at runtime with modern and configurable Optical Character Recognition (OCR) engine based on Neural Networks. 

Citrix support is currently being released as a Beta. Please ask your Neotys contact or contact us here to get a trial license key.

NeoLoad Web Tests

The test settings for launching a test from NeoLoad Web are now saved as ‘Tests’ which can be shared with and reused by teammates.

Click on image to enlarge

Scheduled Tests for Reservations

Schedule your tests to leverage your test infrastructure during off-business hours.

NeoLoad Web Reservations can now automatically start a test when the reservation starts.

Click on image to enlarge

Percentiles in NeoLoad Web Results

NeoLoad Web test results now provide the 50th, 90th, 95th and 99th percentiles for transactions. These metrics are available through NeoLoad Web’s public REST API. Dashboards also provide a percentile graph on transactions.

Click on image to enlarge

Cloud CI Integration (GitLab, AWS CodeBuild, Azure DevOps)

Integrate NeoLoad load tests in Docker friendly Continuous Integration pipelines such as GitLab, AWS CodeBuild and Azure DevOps. Neotys provides a Docker image that easily uploads a project to NeoLoad Web and triggers its execution from environment variables. Examples are provided with the documentation.

See neoload-web-test-launcher on DockerHub.

Functions

Speed up design time and increase design readability by using function calls rather than variables, and skip the extra steps of inserting JavaScript actions for simple functions calls. 

Wherever you can use a variable (ex: ${myVariable}) when designing a User Path, you can now use a Function with the following syntax: ${__myFunction(arg1,arg2)}. 

NeoLoad provides a number of predefined functions for new projects as well as allowing you to define your own.

Get more information in the documentation.

Click on image to enlarge

Summary Panel for a Scenario

When using several Populations for your tests, use the new summary panel to quickly check that your populations are properly and consistently configured, notably just before running a test. It also enables you to check the total number of Load Generators used and the total number of Virtual Users that will be generated.

Click on image to enlarge

Project Format for Externally Managed Version Control Systems

NeoLoad native integrations with SVN and GIT make it easy to use a Version Control System to version and collaborate on NeoLoad projects. For advanced usage and when the NeoLoad integration cannot be used, NeoLoad projects can now be saved in a multi-XML files format so that it’s easier to use your preferred VCS tools and commands independently of NeoLoad.

See how to change the storage format in the documentation.

Click on image to enlarge

Other Enhancements

• Search & Replace now work on Advanced Actions, SAP GUI and Citrix Actions.
• Bulk delete and duplicate of Populations. 
• ECMA 6 support by the NeoLoad Javascript engine.
• Copy/Paste of screenshots on Pages, SAP GUI and Citrix Actions.

Compatibility Notes

The Script Converter must be used as part of a global migration plan and not independently, so it is now disabled by default. Please contact support to learn how to continue using it.

July 2019 Release

What’s New in NeoLoad 7.0?

The NeoLoad 7.0 release introduces the ability for teams to load test on-demand using the new dynamic infrastructure capability for users of OpenShift. In tandem, resource reservation is now generally available, out of its previously released preview mode. Two new enhancements are also included: enhanced Dynatrace integration and a new format for exporting raw data.

Dynamic Infrastructure for Open Shift

Dynamic Infrastructure is NeoLoad’s new capacity to interact with container orchestrators, currently OpenShift, to provision and use load testing infrastructure automatically, on demand. When a test is run through NeoLoad Web, controllers and load generators are spun up for the duration of the test and released once the test is finished. The load testing resources are automatically provisioned with pre-set settings within a minute of executing a test on demand and the Docker containers are shut down and released once the test is complete. NeoLoad leaves the machine automatically cleared and ready for any use.

No need to write lengthy scripts to provision machines, manually connect dynamic testing resources to CI pipelines or manually change the number of load generators used for a test. With dynamic zones, connect to testing resources in OpenShift with a few clicks. Infrastructure is set up only once. Later, increase or decrease the number of load generators easily with a slider scale. Reduce the total cost of hardware ownership as you do more with the same testing resources without the need to tie up machines indefinitely or manual steps to release and reconfigure machines. Reuse the same machines for multiple purposes beyond testing.

Click on image to enlarge

Resource Reservation Generally Available

Resource reservation, previously released in preview mode, is now *generally available.  With resource reservation, reserve load testing infrastructure (controllers and load generators) and virtual users by date and duration to guarantee resources are dedicated for application performance testing for a specific team or team member and avoid testing schedule conflicts among the teams.
Share and reserve resources as a team or an individual – any member of the team can cancel, modify, and use the reservation. Optimize test infrastructure usage and improve team collaboration and productivity with resource reservation.

Click on image to enlarge

Other Enhancements

• Enhanced Dynatrace integration – The HTTP header name used between NeoLoad and Dynatrace can now be customized to use the same value as other testing tools in your stack. Creating Dynatrace settings is now easier and homogeneous for all tools. The working default value is the one recommended by Dynatrace.
• New CSV data format – The format of the CSV created when exporting raw data at transaction level has been changed and is not compatible with the previous format. Updates include the elapsed time which is now in milliseconds, a new absolute time and a virtual user ID to identify the instance of the virtual user.

May 2019 Release

What’s New in NeoLoad 6.10?

The NeoLoad 6.10 release updates and enhances many of our recently added features to automate API and end-to-end application performance testing. The main enhancements include updates to the Test as Code capabilities, GIT support, APM integration, and more.

Test as Code

Prior NeoLoad releases offer the ability to automate API tests within CI pipelines or run ad hoc API smoke tests to see if recent code modifications degraded performance. Previously test as code functionality was available through the NeoLoad command line. Now, NeoLoad offers the option to create, run and manage performance tests as code via a standalone YAML file, as multiple semantic files and execute tests as code via command line, the NeoLoad Web API or the NeoLoad Web user interface.

• Test as Code: as Standalone YAML
  Design and run tests in YAML as standalone code without dependency on a NeoLoad project, similar to a unit or non-regression tests. Execute tests as code in YAML through command line, the NeoLoad Web API or within the NeoLoad Web user interface.

  Test as Code: as One or Many YAMLs
  Concentrate tests as code in one unique file or split the logic into several, smaller, semantically consistent files (Scenarios, Variables, UserPaths, etc.) and reference them using includes. Includes promote test code reuse and sharing and ease switching between target environments.

  Test as Code: More Expressions
  Test as Code now supports control structures (IF, THEN, ELSE), inline JavaScript and new variable types (random number and counter).

Click on image to enlarge

Learn more with the NeoLoad as-code guide

Improved GIT Integration

With prior NeoLoad releases, teams can collaborate, share and manage all test assets through their GIT version control system. Now teams can also run a performance test from NeoLoad Web based on a project retrieved from GIT – whether the test asset is a Neoload project, a pure YAML file, or a hybrid version. 

Click on image to enlarge

APM Integration

NeoLoad Web’s API now offers the ability to retrieve a list of events (errors, failed SLAs, and failed monitoring thresholds) and to push them into third-party tools, like APMs. Testers can use API events to correlate performance and APM data to get more intelligent insights.

Click on image to enlarge

Other Enhancements

Enhanced Proxies – Define and set proxies through the command line for both the NeoLoad Controller and Load Generators to ease continuous integration and Docker deployments.
Improved Security – NeoLoad now has stronger encryption between its Controller and Monitoring Agent.