The first step in performance or load testing a web application is to create a realistic test script. This script, representing a specific type of end user, contains steps that are fully automated transactions flows. Often, complex behaviors need to be emulated as part of these flows. To incorporate the needed behavior or to handle complex scenarios, testers need to customize these load scripts. The question is whether you utilize a tool that takes a GUI driven approach for script customization/manipulation, or whether you choose a tool which requires the use of a programming language.

As Performance testers, we typically evaluate several load testing tools before diving into a project. Some of us prefer the GUI driven approach to creating load scripts because, being results driven, we want to get the test up and running as quickly as possible. The GUI approach saves us time and there is no programming involved thus the learning curve is also extremely fast. These GUI scripts are typically easily modified by having their parameters and values organized in a visually obvious format. And just because a tool is easy to use, doesn’t mean that it’s only for beginners. Remember the good old days of MS-DOS? Eeek, thank goodness for GUI driven DOS (Windows).

Some testers (mistakenly) dismiss the GUI driven approach as being too limited for complicated behaviors. Old school testers insist on only utilizing program-based scripts because past experience has led them to think that this is the most flexible or the only choice for handling complicated behaviors. All too often, the perception is that GUI driven scripts limit the level of customizations possible to emulate a real user. This opinion is likely based on experiences with rudimentary GUI tools on the market that are too basic and limited in their capabilities such as handling dynamic content and varying the transactions flows.

However, there is a new breed in load testing solutions, designed and built to handle the most complex advanced scenarios. These are true enterprise-class solutions which provide a hybrid approach by containing both GUI and programming features. The power behind this hybrid approach enables the user to emulate even the most complex behavior but it also includes time saving features which can only be offered by a GUI solution.

Let’s get really technical for a minute to highlight an example: Some tools provide the common “programming” tools like loops and conditions. Recorded values of parameters are easily replaced by variables with a click of the mouse. NeoLoad, an example of a modern enterprise solution, goes further to provide even more advanced features. Some specific examples include: 1) “Wait Until” Action to the current execution thread until certain conditions have been verified 2) “Shared Queue” Action to use values from one VU’s response as input in an entirely different VU execution 3) a “While” Action allows various transactions, such as web pages, to be iterated while a condition remains false 4) “Try …Catch” Action to break out of the script based on Validation or Assertions and go to the End phase. The use of nested actions creates even more complex behaviors. All of these advanced scenarios are accomplished without programming! PUSH technology which is rapidly being deployed in today’s RIA’s adds even more complexity. With asynchronous updates pushed to/from the page independent of complete page refreshes…imagine this programming nightmare! NeoLoad has a way to solve this problem using cutting edge technology that can “listen” for PUSH behavior (all polling or streaming requests) and wrap all these asynchronous requests into a GUI fork Action, even calculating the average time interval between execution, then presents you with a newly modified script to emulate these load pattern characteristics. Again, all of these GUI centric features allow further manipulation without the need for programming.

Admittedly, there are rare occasions which call for programming and the new breed solution has a Javascript Action with a complete API to give it a “hybrid” of GUI and coding capability. Again, simply drag the Javascript action into the place where you want it executed and add the java or javascript required. Accomplish things like computation of variables, logging values to files, manipulating cookies, etc.. Incorporating the javascript feature allows for unlimited flexibility in scripting.
As the script becomes more complicated, it’s nice to have this hierarchical GUI-type view.

New breed solutions not only allow for complex behaviors…the GUI approach has other time saving embedded assets. To handle dynamic parameters, NeoLoad is delivered with an ever evolving list of known architectural framework dynamic parameters which are automatically detected and handled by a wizard. The search and replace feature finds strings embedded in any request (body, header, URL’s, XML tree, partial matches, etc.) and replaces or concatenates to the string with any data or variable – choose to replace entire string or just part of the string. There’s an automatic extraction feature to take data from any response and use it to populate the value of a variable. You can enable/disable specific transactions at a page level or embedded request level which a click of the mouse to isolate transactions of interest. Target an entirely different QA/Performance/Production environment – change the Servers field from host where you initially made the recording to the new server name and port. Debug scripts easily using the side-by-side comparison feature with a tab isolating only the differences of recorded vs. replayed values.

From this perspective, the great debate over choosing a GUI driven or programming approach is over…eliminate the tradeoffs and choose a hybrid which has a powerful GUI and also extends the flexibility for programming. Skip most of the manual programming and only use coding where it’s required. Both experts and beginners can agree there is now a solution which meets all of the load & performance testing challenges of your own unique web application. Introducing the “GUI hybrid”.

, ,

10 Responses to “GUI Versus Script. Is This Old Load Testing Debate Now Over?”

  1. James Beckles says:

    Can NeoLoad be installed on the same machine with Loadrunner?

  2. Steve Weisfeldt says:

    Absolutely. They’re two entirely separate tools – there’s nothing with LoadRunner that should interfere with a NeoLoad installation….

  3. Ben says:

    Be careful not to alienate your intended audience with opening statements like “The first step in performance or load testing a web application is to create a realistic test script.”

    I stopped reading after that sentence. I don’t know any good performance engineers who would agree that performance testing starts with scripting.

    • **100%** completely agree with you, Ben.

      In our haste to make sure tool users pay close attention to creating absolutely real-word scripts, we shortchanged the start of the process.

      Scripting should not be embarked upon until one has carefully identified test requirements, goals, required resources, etc. (i.e. created a comprehensive test plan). The projects where test engineers jump immediately into scripting are the ones that are most likely to fail.

    • Jerry Hogan says:

      We actually go through an assessment to make sure that the “client” understands their responsibilities, that we have project management and an SME on board, and that the full scope of the work is understood and agreed to by both parties. There is a great deal that has to happen before we ever start scripting.

      Because of the nature of our business and the number of sites supported all our applications run through Citrix. So even while there may be a GUI it doesn’t matter.

      We may be able to see a GUI for certain HQ only applications but few of those are on the radar at this time.

  4. Super Kevy says:

    I was confused by the statement

    Again, simply drag the Javascript action into the place where you want it executed and add the java or javascript required. Accomplish things like computation of variables, logging values to files, manipulating cookies, etc.. Incorporating the javascript feature allows for unlimited flexibility in scripting.

    So I am programing? And is this a special api like the loadRunner Lr api calls or am I creating tme from scratch?

    • Generally speaking, the NeoLoad UI allows you to develop your Virtual User Profiles without any programming. All requests – including customization and parameterizations – are easily handled by the GUI-driven approach that NeoLoad offers.

      The JavaScript action that you refer to allows you to augment the VU Profiles you’ve developed by writing custom JavaScript code *IF YOU WISH*. It is not necessary by any means but can be useful in the rare cases where you have unique logic that you need to introduce into your profiles. You may also have client-side computation that would occur inside a browser that you wish to include in your VU Profiles — the JavaScript action can be useful here as well.

  5. Everseeker says:

    Have a Question about the other half… Report Presentation.
    Management likes pretty pictures/dashboards, BLOTs, Tl;Dr docs, etc…
    LoadRunner is kinda falling flat here… their new 11.52 really did not do much for presentation… So, how does NeoLoad look, out of the box?
    (And yes, We are taking the LR data and “rolling our own” but I feel it should not be that way…)

    • NeoLoad’s results engine allows you to create all sorts of graphs, tables, reports, etc., to help you analyze the behavior of your test application during test execution. There are many different vehicles available to do your analysis and reporting (including some very slick Comparison Reports).

      The best way to answer your “how does it look?” question is really to suggest that you try it our yourself with our free evaluation so you can get a better sense of this for yourself. Alternatively, feel free to contact at and we’d be happy to see that you’re exposed to a demonstration of our solution.

  6. Dancho says:

    My concern about GUI versus traditional HTTP/HTTPS web has always been footprint during execution of test not customizing because all tools (LoadRunner, SilkPerformer) give you many functions to use. Currently we can only put 50 users per agent approximately with GUI versus about 3000+ for HTTP/HTTPs protocol. This is because it has to put the whole application in memory for each user.

Leave a Reply