#NeotysPAC – Performance Tooling Evolution in Response to Change, by Ian Molyneaux

[By Ian Molyneaux, MolySOFT]

I was pleasantly surprised to be invited by Neotys to participate in the inaugural #NeotysPAC summit recently held in Borthwick Castle near Edinburgh.  A meeting of seasoned professionals to discuss current and future trends in performance engineering.

It was great to enjoy the scenery, hospitality, and refreshments on offer and to digest the very interesting and thought-provoking presentations of the other delegates.

For my presentation, I decided to look at how disruptive technologies like Cloud, Mobile, and IOT has impacted the evolution of performance test automation tooling in response to client requirements and how this may change in the future.

Performance Test Automation (Past)

The historical record of performance automation tooling is pretty familiar to anyone who has been consulting in the performance test space for any time. The appearance of Load Runner from Mercury Interactive in the mid to late 90’s enabled serious performance testing for the first time in that you could create a reusable set of test assets representative of application UI behavior and then use these assets to drive distributed load to whatever level required. Before this, performance testing was pretty much a manual exercise almost impossible to repeat consistently at anything other than very modest scale.

Following LoadRunner other tools came to market including QA load from Compuware, Borland Silk Performer, Facilita Forecast (now eggPlant Performance), Neoload of course and others. Many of these toolsets have subsequently been acquired and relaunched, sunsetted or remain available today.

They all offer (or offered) pretty much the same basic capability of script record, correlation, and playback differing mainly in supported protocols, native and integrated monitoring, the sophistication of test analytics and of course licensing costs. Later generations added integration with IDE (Eclipse, MSVS), APM solutions like DynaTrace and Appdynamics and most recently Cloud and Automation Server frameworks like Jenkins looking to embrace the move to DevOps.

Performance Test Automation (Present)

Protocol support requirements are now almost exclusively for Web technology with a very limited requirement persisting for remote desktop (ICA, RDP) and packaged solution clients such as SAP GUI (ABAPI) and Oracle eBusiness Suite.

Cloud support has become mandatory as can combine cloud and LAN based load injection as part of performance test configuration. Cloud-based test environments now make it possible to test production SLA’s against components in Dev, something previously unheard of.

The current trend in tooling has also seen a resurgence in popularity for Open Source offerings. JMeter remains a very popular toolset and lends itself well to performance testing in Dev where the powerful scripting engine and the limited requirement for distributed load injection play to its strengths.

With the evolution of performance testing to embrace the full SDLC rather than simply a late life-cycle NFR testing requirement, it has become a vital validation step that performance engineering is alive and well in the Enterprise.  

As an example – Dev performance testing in-sprint promotes performance awareness and aligns well with modern component-based architecture choices including Microservices, Containers, and Node.js. Being able to trend performance and scalability across builds at such an early stage in the SDLC greatly benefits the quality and speed of release.

Performance Test Automation (Future)

Future trends for performance automation tooling are likely to focus on increased automation server integration, easier data exchange with application monitoring solutions and additional protocol support for IOT based testing requirements.

The emergence of AI may (eventually) lead to some form of smart testing including automated creation and maintenance of test assets, test scenario execution, and reporting although, in the short-term, it is most likely to simply benefit test analysis as AI increasingly becomes part of the Big Data model.

I predict Open Source will remain a strong player as the desire to bring performance engineering skills in-house as part of DevOps including tooling will only increase.

It goes without saying that to remain competitive licensed tool vendors will need to ensure they keep providing the features client want in a cost-effective package. It is interesting to note some established toolsets now offering the ability to integrate and execute tests using Selenium and JMeter scripts. Sign of the times.

Ian Molyneaux

Originally from Auckland, New Zealand, Ian Molyneaux ended up in IT purely by chance after applying for an interesting looking job advertised as “junior computer operator” in the mid-’70s.
The rest is history: 36 years later Ian has held many roles in IT but confesses to being a techie at heart with a special interest in application performance.

Ian is a published author on Performance Testing and related disciplines (The Art of Application Performance Testing, O’Reilly 2009 and 2014) as well as being an established technical writer.
Ian currently works as an independent consultant offering advice and support to clients on:

– DevOps Implementation
– Performance Testing
– Application Performance Management (APM)
– Performance Trouble-Shooting

Ian also speaks at many IT events and works in an advisory capacity with many IT tooling vendors.

Learn More about Performance Tooling Evolution

Do you want to know more about this event? See Ian Molyneaux’s presentation here.

Leave a Reply

Your email address will not be published. Required fields are marked *