June 2020 Release

What’s New in NeoLoad 7.4?

Command Line Interface 

The Python CLI is a client to the runtime API of NeoLoad Web (SaaS or on-premise) that allows  you to define and control a test from anywhere a Python command line is accessible. Using the Python CLI for NeoLoad is a simple and speedy approach for automating performance testing in CI pipelines.

For instance to: 

1. Install the CLI client
2. Connect to your Neoload Platform (in the example, the default is our SaaS platform)
3. Create and configure a test
4. Associate a project file
5. Run your test

All you need to do is:

Click on image to enlarge

The CLI can be combined with NeoLoad’s performance testing as-code capabilities in any of your automation environments.

A comprehensive documentation is available on the Neotys GitHub repository: https://github.com/Neotys-Labs/neoload-cli 

Preconfigured popular CI pipeline examples (such as Jenkins, Azure Devops or GitLab) are also provided to get started even faster: https://github.com/Neotys-Labs/neoload-cli/tree/master/examples/pipelines

Filtering Result Values 

When analyzing results in NeoLoad Web, you can filter Transactions, Pages and Requests by Zones, Populations and User Paths, to focus your analysis for greater accuracy.

Click on image to enlarge

Custom Fields for Raw Data

When exporting raw data for transactions from the NeoLoad Controller, you now can add custom fields to tell in which context the execution of the transaction occurred. As an example, you will not only see which specific transaction was slow but you can also see which data was used (example ContractID for each “Edit Contract” transaction).

See the tutorial on exporting contextualized raw data.

Click on image to enlarge

The following image illustrates an example of NeoLoad load testing data exported to an analytics platform:

Click on image to enlarge

Dynamic Infrastructure Improvements

Neotys adds Google GKE to the list of supported dynamic infrastructure providers for NeoLoad Web: OpenShift, Kubernetes, Microsoft AKS and Amazon EKS.

Also, NeoLoad Web now can launch tests using dynamic infrastructure even when the reservation feature is enabled on the account. In this case, you need to use auto-reservation, since planned reservations are not supported for dynamic infrastructure.

Dynamic zones settings are now more flexible and powerful allowing to switch from one cluster to another.

Tosca Integration v2.1

Support for Tricentis Tosca web tests has been enhanced with the automatic creation of a Neoload Transaction for each Tosca step in the converted testcase. With automated userpath update, when you make a change to your Tosca testcase, Neoload merges the new user path with the updated one, keeping all the changes you made to your initial NL userpath such as variables, loops, SLA and so on.

Click on image to enlarge

This version also brings support for Tosca 13.1 SP3.

Other Enhancements

• Citrix improvement – Random popup, asynchronous notifications or potential warnings can interrupt your Citrix scenario. Neoload now manages these intermittent popups with a simple Fork action in your design.
• NeoLoad Web Licensing improvement – A license now can be leased in parallel by two Controllers having the same hostname which may occur when starting Controllers in different Kubernetes pods.

Compatibility Notes

• New project version – Projects designed with NeoLoad 7.4 can only be opened by NeoLoad 7.4 and more. It means that you need to upgrade your load generators to be able to run projects designed with this new release. Note that the new Load Generators will be able to run your older projects.

April 2020 Release

What’s New in NeoLoad 7.3?

Dynamic Test Infrastructure with Kubernetes, AKS, and EKS

NeoLoad’s Dynamic Infrastructure is the capability to interact with container orchestrators, currently OpenShift, Kubernetes, Microsoft AKS and Amazon EKS, to provision and use load testing infrastructure automatically, on demand.

When a test is run through NeoLoad Web, controllers and load generators are spun up for the duration of the test and released once the test is finished. The load testing resources are automatically provisioned with pre-set settings within a minute of executing a test on demand and the Docker containers are shut down and released once the test is complete. NeoLoad leaves the cluster automatically cleared and ready for any use. 

Each zone now can be customized in terms of CPU, memory, Docker image whether your cluster is managed using a cloud orchestrator or self managed using Kubernetes directly.

Click on image to enlarge

Click on image to enlarge

Tricentis Tosca Web Tests as Performance Tests

Previously, NeoLoad released the capability to convert SAP GUI functional tests created in Tosca, to performance tests in NeoLoad. Now, NeoLoad can convert any web tests (not limited to SAP) created in Tosca too. With one click, Tosca web or SAP GUI functional tests can be exported and converted into performance tests. 

Support for SAP GUI test analysis has been enhanced. When exporting or re-exporting an SAP GUI test, you now have the option to automatically create a Neoload Transaction for each SAP TCode detected in your test.

The install process has also been simplified.

Citrix Enhancements

NeoLoad contains several improvements to ease recording and maintenance of Citrix applications testing:

• Update recorded content now effective for Citrix
• Additional tolerance level for image detection
• Constant variable for Citrix timeout
• Support of NoPrivileges LG launcher for Terminal Services
• Set back focus to Citrix client after NL citrix capture
• Screenshot comparison panel to help identify differences between the recording and the playback.

Click on image to enlarge

New public API v2 for NeoLoad Web

A new version of the public API now enables a much more powerful and more flexible automation of tests in the context of continuous integration and continuous load testing.

The API has been enhanced to leverage the recent concept of Tests in NeoLoad Web. It allows you to automatically run a Test that has been defined manually in NeoLoad Web’s interface or fully automate the whole process from creating a Test, uploading the NeoLoad project and running it.

Changes to the API

• New Tests related functions (create, update, upload project, list, delete)
• Running a test is now on the Test (/tests/{testId}/start) 
• Ability to stop a running test  (/test-results/{resultId}/stop) 
• Test result related functions renamed from /tests/ to /test-results/

See the API documentation.

Other Enhancements

• Force release – When it’s not possible to release a license lease from NeoLoad Web normally, you can now request Neotys support to unlock the lease.
• When launching a test in the context of a reservation for a group, only group members can stop the test in NeoLoad Web.
• New JUnit Report Format – NeoLoad can generate a JUnit report for the SLA results for command line tests. When using ‘-SLAJUnitResults’ the old format is kept for backward compatibility. When using ‘-SLAJunitReport’, the XML grammar has been fixed to be compatible with more tools handling JUnit format and the way JUnit tests are named and structured is more actionable.
• NeoLoad Web users now have the ability to pick licensed Citrix or SAP VUs to run their Web tests.
• The screenshot comparison panel now helps identify differences between the recording, the playback or debugging – for any action that has a screenshot (Web Pages, Citrix, SAP GUI …).

January 2020 Release

What’s New in NeoLoad 7.2?

Citrix Support

NeoLoad now can record and replay Citrix deployed applications. The set of Citrix actions is compact and consistent for better readability and easier script maintenance. Design faster with NeoLoad’s live detection of texts and get a more reliable text recognition at runtime with modern and configurable Optical Character Recognition (OCR) engine (based on Neural Networks).

See the Citrix Testing Guide.

Changes since the beta version:

• Windows Interactive Logon is now possible.
• Various UI improvements and bug fixes.

License Key Management

Click on image to enlarge

Split Licenses Between Several NeoLoad Web Accounts
An Enterprise License can now be split between several NeoLoad Web Accounts and/or instances (On-premise or SaaS) for a given duration and a given number of licensed VUs.

Learn more in the documentation.

Offline Lease with NeoLoad Web
In order to test from isolated environments, a NeoLoad Controller can already request a lease from an Enterprise License installed in Neotys Team Server with an offline process.

A NeoLoad Controller now can also request an offline lease from an Enterprise License installed in NeoLoad Web.

Learn more in the documentation.

Other Enhancements

• Some default settings when creating an element can now be overridden and shared, for example the error handling policy for a VU Path or the stop policy of a Population at runtime. See the ‘Advanced’ section of the general settings.
• The NeoLoad Web’ Tests page displays new columns: ‘‘updated by’, ‘last modified’, ‘last run’’
• NeoLoad Web’s graph legend now includes the Test name.
• New NeoLoad Web Resources API to access the list of Reservations and the list and the content of Zones. See the API Definition.
• The bundled Controller Java VM is now an OpenJDK which has an impact on how to install NeoLoad in a headless Linux environment.
• The default ‘Change Policy’ setting for new List, File and Sql variables has been changed from “For each Virtual User instance” to “Each Iteration”.
• Browser Profiles picker has been updated with the latest browsers.
• You now can sort test results returned by Neoload Web API. See the API Definition
• Neoload Web now officially supports MongoDB 3.6.

Note About Installation

When installing NeoLoad with the command line in a headless environment, NeoLoad 7.2 has new requirements. See how to install NeoLoad in a headless Linux environment.

Bug Fixes

LOAD-19894
[QFT] Project settings/Runtime parameters problem

LOAD-19889
[NLW > UPLOAD PROJECT] Scenario summary are missing when user creates a new test

LOAD-19873
An offline lease can be installed as a License and so used while it has been released

LOAD-19872
Starting Test screen: space missing for ‘Number of Virtual Users’

LOAD-19867
[offline ]Another Admin user cannot use the generated rls file to release on the Master

LOAD-19837
NLG – Dashboard are created 2 times when opening a project saved using XML_MULTI format

LOAD-19674
[NLW] ${runID} should start at 1.

LOAD-19666
Force JS Variable initialisation to be evaluated to make sure no value stays in cache

LOAD-19483
SOAP parameter “no encode” value does not work as expected

LOAD-17905
[NLW Doc] – OnPremise – In SSL deployments, max file upload size documentation should also talk about nginx limitation.

LOAD-17891
The NeoLoad as code feature does not expect extension file in upper case but only in lower case

November 2019 Release

What’s New in NeoLoad 7.1?

NeoLoad 7.1 marks NeoLoad’s beta support of Citrix performance testing capability, new continuous integration extensions to Azure DevOps, GitLab and AWS CodeBuild and updates to test management and transaction data. Tests setting can now be saved in NeoLoad web, shared and reused. Tests can also be scheduled to be run during off-hours. Test reports and dashboards now display the 50th, 90th, 95th and 99th percentiles metrics for transactions.

Citrix Support

NeoLoad now can record and replay Citrix deployed applications. The set of Citrix actions is compact and consistent for a better readability and easier to maintain scripts. Design faster with a live detection of texts and get a more reliable text recognition at runtime with modern and configurable Optical Character Recognition (OCR) engine based on Neural Networks. 

Citrix support is currently being released as a Beta. Please ask your Neotys contact or contact us here to get a trial license key.

NeoLoad Web Tests

The test settings for launching a test from NeoLoad Web are now saved as ‘Tests’ which can be shared with and reused by teammates.

Click on image to enlarge

Scheduled Tests for Reservations

Schedule your tests to leverage your test infrastructure during off-business hours.

NeoLoad Web Reservations can now automatically start a test when the reservation starts.

Click on image to enlarge

Percentiles in NeoLoad Web Results

NeoLoad Web test results now provide the 50th, 90th, 95th and 99th percentiles for transactions. These metrics are available through NeoLoad Web’s public REST API. Dashboards also provide a percentile graph on transactions.

Click on image to enlarge

Cloud CI Integration (GitLab, AWS CodeBuild, Azure DevOps)

Integrate NeoLoad load tests in Docker friendly Continuous Integration pipelines such as GitLab, AWS CodeBuild and Azure DevOps. Neotys provides a Docker image that easily uploads a project to NeoLoad Web and triggers its execution from environment variables. Examples are provided with the documentation.

See neoload-web-test-launcher on DockerHub.

Functions

Speed up design time and increase design readability by using function calls rather than variables, and skip the extra steps of inserting JavaScript actions for simple functions calls. 

Wherever you can use a variable (ex: ${myVariable}) when designing a User Path, you can now use a Function with the following syntax: ${__myFunction(arg1,arg2)}. 

NeoLoad provides a number of predefined functions for new projects as well as allowing you to define your own.

Get more information in the documentation.

Click on image to enlarge

Summary Panel for a Scenario

When using several Populations for your tests, use the new summary panel to quickly check that your populations are properly and consistently configured, notably just before running a test. It also enables you to check the total number of Load Generators used and the total number of Virtual Users that will be generated.

Click on image to enlarge

Project Format for Externally Managed Version Control Systems

NeoLoad native integrations with SVN and GIT make it easy to use a Version Control System to version and collaborate on NeoLoad projects. For advanced usage and when the NeoLoad integration cannot be used, NeoLoad projects can now be saved in a multi-XML files format so that it’s easier to use your preferred VCS tools and commands independently of NeoLoad.

See how to change the storage format in the documentation.

Click on image to enlarge

Other Enhancements

• Search & Replace now work on Advanced Actions, SAP GUI and Citrix Actions.
• Bulk delete and duplicate of Populations. 
• ECMA 6 support by the NeoLoad Javascript engine.
• Copy/Paste of screenshots on Pages, SAP GUI and Citrix Actions.

Compatibility Notes

The Script Converter must be used as part of a global migration plan and not independently, so it is now disabled by default. Please contact support to learn how to continue using it.

July 2019 Release

What’s New in NeoLoad 7.0?

The NeoLoad 7.0 release introduces the ability for teams to load test on-demand using the new dynamic infrastructure capability for users of OpenShift. In tandem, resource reservation is now generally available, out of its previously released preview mode. Two new enhancements are also included: enhanced Dynatrace integration and a new format for exporting raw data.

Dynamic Infrastructure for Open Shift

Dynamic Infrastructure is NeoLoad’s new capacity to interact with container orchestrators, currently OpenShift, to provision and use load testing infrastructure automatically, on demand. When a test is run through NeoLoad Web, controllers and load generators are spun up for the duration of the test and released once the test is finished. The load testing resources are automatically provisioned with pre-set settings within a minute of executing a test on demand and the Docker containers are shut down and released once the test is complete. NeoLoad leaves the machine automatically cleared and ready for any use.

No need to write lengthy scripts to provision machines, manually connect dynamic testing resources to CI pipelines or manually change the number of load generators used for a test. With dynamic zones, connect to testing resources in OpenShift with a few clicks. Infrastructure is set up only once. Later, increase or decrease the number of load generators easily with a slider scale. Reduce the total cost of hardware ownership as you do more with the same testing resources without the need to tie up machines indefinitely or manual steps to release and reconfigure machines. Reuse the same machines for multiple purposes beyond testing.

Click on image to enlarge

Resource Reservation Generally Available

Resource reservation, previously released in preview mode, is now *generally available.  With resource reservation, reserve load testing infrastructure (controllers and load generators) and virtual users by date and duration to guarantee resources are dedicated for application performance testing for a specific team or team member and avoid testing schedule conflicts among the teams.
Share and reserve resources as a team or an individual – any member of the team can cancel, modify, and use the reservation. Optimize test infrastructure usage and improve team collaboration and productivity with resource reservation.

Click on image to enlarge

Other Enhancements

• Enhanced Dynatrace integration – The HTTP header name used between NeoLoad and Dynatrace can now be customized to use the same value as other testing tools in your stack. Creating Dynatrace settings is now easier and homogeneous for all tools. The working default value is the one recommended by Dynatrace.
• New CSV data format – The format of the CSV created when exporting raw data at transaction level has been changed and is not compatible with the previous format. Updates include the elapsed time which is now in milliseconds, a new absolute time and a virtual user ID to identify the instance of the virtual user.

May 2019 Release

What’s New in NeoLoad 6.10?

The NeoLoad 6.10 release updates and enhances many of our recently added features to automate API and end-to-end application performance testing. The main enhancements include updates to the Test as Code capabilities, GIT support, APM integration, and more.

Test as Code

Prior NeoLoad releases offer the ability to automate API tests within CI pipelines or run ad hoc API smoke tests to see if recent code modifications degraded performance. Previously test as code functionality was available through the NeoLoad command line. Now, NeoLoad offers the option to create, run and manage performance tests as code via a standalone YAML file, as multiple semantic files and execute tests as code via command line, the NeoLoad Web API or the NeoLoad Web user interface.

• Test as Code: as Standalone YAML
  Design and run tests in YAML as standalone code without dependency on a NeoLoad project, similar to a unit or non-regression tests. Execute tests as code in YAML through command line, the NeoLoad Web API or within the NeoLoad Web user interface.

  Test as Code: as One or Many YAMLs
  Concentrate tests as code in one unique file or split the logic into several, smaller, semantically consistent files (Scenarios, Variables, UserPaths, etc.) and reference them using includes. Includes promote test code reuse and sharing and ease switching between target environments.

  Test as Code: More Expressions
  Test as Code now supports control structures (IF, THEN, ELSE), inline JavaScript and new variable types (random number and counter).

Click on image to enlarge

Learn more with the NeoLoad as-code guide

Improved GIT Integration

With prior NeoLoad releases, teams can collaborate, share and manage all test assets through their GIT version control system. Now teams can also run a performance test from NeoLoad Web based on a project retrieved from GIT – whether the test asset is a Neoload project, a pure YAML file, or a hybrid version. 

Click on image to enlarge

APM Integration

NeoLoad Web’s API now offers the ability to retrieve a list of events (errors, failed SLAs, and failed monitoring thresholds) and to push them into third-party tools, like APMs. Testers can use API events to correlate performance and APM data to get more intelligent insights.

Click on image to enlarge

Other Enhancements

Enhanced Proxies – Define and set proxies through the command line for both the NeoLoad Controller and Load Generators to ease continuous integration and Docker deployments.
Improved Security – NeoLoad now has stronger encryption between its Controller and Monitoring Agent.

March 2019 Release

What’s New in NeoLoad 6.9?

Performance Test APIs to Full Applications

The Neotys team continues working hard to release in our agile framework to provide new functionality often. The 6.9 release marks improvements in our test as code feature set, giving Enterprises a path for building performance testing into CI environments. New Worksoft Certify Integration allows testers to reuse functional test assets as performance tests from the most widely used SAP functional testing tool on the market. The highly demanded resource reservation feature offers individuals and teams the ability to reserve testing infrastructure and licenses for application testing.

Resource Reservation – Preview

NeoLoad introduces the ability to reserve load testing infrastructure (controllers and load generators) and virtual users by date and duration to guarantee resources are dedicated for application performance testing. Share and reserve resources as a team or an individual – any member of the team can cancel, modify, and use the reservation. Optimize test infrastructure usage and improve team collaboration and productivity with resource reservation.

This feature is available as a fully functional, feature complete preview for a limited time. Pricing and packaging are to be announced.

Click on image to enlarge

Worksoft Integration

Certify by Worksoft is the most widely used functional testing tool for SAP applications. NeoLoad is the only performance testing platform on the market that has built-in integration with Worksoft Certify to make functional test asset reuse a cinch. The animated GIF below shows how functional test assets from Certify are converted into performance tests in NeoLoad. And, as code changes demand updated functional tests, NeoLoad’s user path update automatically updates the performance tests with the new changes.

Accelerate SAP GUI and web performance test design and test maintenance with the NeoLoad Certify integration.

Available on GitHub: Click here

Click on image to enlarge

Test as Code: Define and Use SLAs

Building on NeoLoad’s capabilities to design and execute performance tests as code, NeoLoad now offers the ability to define and use SLAs from code. Below’s image illustrates SLA definition and usage within a YAML file, for example. SLAs as code is in addition to our current support for SLA definition natively within the NeoLoad GUI if that’s your jam. Performance test APIs one at a time or automate performance testing within CI pipelines with NeoLoad’s test as code feature set.

Click on image to enlarge

Feature Enhancement: Load Generator Availability Status

NeoLoad now provides status information regarding load generator availability. Select load generators with the knowledge of which are available and which are not during the process of setting up a test. Teams can share load testing resources across different applications more seamlessly.

January 2019 Release

What’s New in NeoLoad 6.8?

Automate Performance Testing in Enterprise Bimodal IT Environments: NeoLoad Covers Traditional and Agile Approaches

NeoLoad continues to build support for the Enterprise bimodal IT organizations who develop and test applications with two separate styles of work:

1. Waterfall development for existing application maintenance and internal business operations
2. Agile development for digital innovations

The NeoLoad performance testing platform covers the range of these application and API testing needs and optimizes performance testing within both development methodologies.

Testing, updating and maintaining the range from modern to legacy technologies continues to get easier with NeoLoad. New integrations include Tricentis Tosca, an SAP GUI functional testing tool, and enhancements to NeoLoad’s Data Exchange API. To support rapid application development of innovative applications, use the additional capabilities in the Shift Left feature set. NeoLoad 6.8 introduces Test Design as Code and the ability to branch and tag tests within GIT. Enhancements to NeoLoad’s Data Exchange API also help developers and testers performance test in agile environments.

Tricentis Tosca Integration

Use NeoLoad’s pre-built integration with Tricentis Tosca, a functional testing tool for SAP GUI applications. Convert functional test assets from Tosca into performance tests in NeoLoad with one click. As functional scripts change, NeoLoad uses the changes to update the performance test. Be more productive when creating and maintaining performance test designs for the monolithic SAP application updates and new implementations. Share and repurpose test assets across applications throughout the develop and test cycles and collaborate better as a team. NeoLoad is the only platform on the market that has this capability.

Available on GitHub.

Data Exchange API

Import third party data into NeoLoad’s dashboard from multiple sources like APM tools that monitor the same applications being tested under load. Show performance test results from NeoLoad and monitoring results from APM tools that test the same application, in one dashboard. Alternatively, expose the NeoLoad performance test results in the APM tools as well with the two-way data exchange.

Teams can share results across dev, ops, testers and business owners.

Test Design as Code

Previous NeoLoad releases enabled the definition of test settings in domain specific language (YAML). Now, design performance tests as you code with NeoLoad 6.8. Integrate performance testing into continuous integration pipelines. Define test script and settings as code to enable automated CI/CD processes to update and maintain tests and to consume and use results. (This is not possible if you rely on a GUI definition that requires a human to open NeoLoad.) Use IDEs and CLIs to develop, run, and maintain performance tests for APIs. Find performance issues early and often when they are easier and less costly to fix. With minimal coding, set the tests’ population, VU path, transactions, requests, delays, variables and etc.

Read more in the documentation.

GIT Branches and Tagging

Integrate with GIT to version control and collaborate on performance tests using NeoLoad’s GIT integration. NeoLoad now provides the ability to choose branches and align branches for the Integrate with GIT to version control and collaborate on performance tests using NeoLoad’s GIT integration. NeoLoad now provides the ability to choose branches and align branches for the different versions of the tested application. GIT users work on branches (Push/Get) and with Tags (Tag and retrieve a Tag) to improve test asset reuse and team collaboration. Build performance testing into agile SDLC pipelines to continuously improve software delivered.

Enhanced Webhooks Support

Introduced in NeoLoad 6.5: Integrate performance testing into test ecosystem with Webhooks. Configure the applications to be notified, for example, inside SMS or Slack, from triggers like a test begins or ends, into a complex CI workflow and automate performance triggers (for example, stop the CI process if a performance test fails). Or use webhooks as a simple notification mechanism, for example, adding a row to a Google Sheet as a log for load test activity.

Now: NeoLoad’s support for webhooks is advanced. Test IDs are now provided as variables for the webhook payload – the NeoLoad Web API can be called to perform custom actions in the CI pipeline. A link to the graph summary of the test is also accessible in the payload.