Validates a server response during a test by defining and asserting a condition that must be met. Validation can be based on the response content, size or duration. NeoLoad and Neotys Design Studio can be configured to throw assertion errors when the conditions defined by an assertion are not met. Other than clearly pointing out unexpected or faulty behavior in load scenarios, assertions and assertion errors also reveal functional issues.
Within the context of an HTTP transaction, basic authentication is an operation whereby the browser includes user account information (login and password) in a request. The information is sent as plain text (merely base64 encoded) in the Authorization header.
A Cloud Load Generator is special in that it is hosted on a virtual machine at a Cloud hosting service provider.
A Cloud session can contain several Cloud Load Generators in several geographical zones. It starts at a start date and stops at a scheduled end date. A Cloud session also may have an availability date that pre-dates its start date. In this case, its Load Generators IP addresses will be available from the availability date, although the Load Generators cannot be used for testing until the start date.
A Cloud workgroup is made up of a number of technical or administrative contacts, plus the credits available, reserved and used. These credits are used to start Cloud sessions; a credit is consumed every hour while a session is open.
Count statistics are the number of full executions of an element of a User Path. If the element is interrupted (because of error or end of test), then the count number is not incremented.
A file variable takes its values from a file with a format such as a comma separated values (CSV) file. To use a file variable, the file from which the values will be read has to be configured.
The form extractor is used in "Submit a form in the previous request" -type requests. This process extracts a form in the server response HTML code so that its values can be injected into the current request.
The HTTP request is the request sent to the server to retrieve an HTML page, image or CSS file. Requests are grouped within pages.
The HTTP response is the Web server answer to an HTTP request.
The link extractor is used in a "Follow a link from the previous request" -type request. This process extracts a link in the server response HTML code so that it can be used in the scenario.
A list variable is a variable whose values are sourced from a specific list.
The Load Generator is a program piloted by the Controller, which plays a scenario while simulating a predefined number of users. A Load Generator is started by its Load Generator Agent for each User Path validation. NeoLoad has an agent and Load Generator built in for ease of use. Load Generator may refer either to the program simulating the users, or to the machine on which it is installed.
The Load Generator Agent responds to the Controller instructions to start and stop the Load Generator for each test. The Controller has an agent and Load Generator built in for ease of use. Additional agents may be installed on other machines if required.
Load testing consists in simulating real-life workload conditions for the tested web application. It helps determine if the application works fine with the loads that result from large numbers of simultaneous users, transactions and to determine whether it can handle peak usage periods.
A logical action is used to enrich the content of a User Path. Typical examples of logical actions are loops, to repeat a sub-series of web pages, pauses, to represent a user's waiting time, and Transactions, which can group several pages within a business transaction.
Long polling is a method used by the push technology and is a hybrid of polling and streaming. As with polling, it uses a mechanism whereby a request is sent by the client to the server at regular intervals. As with streaming, the server keeps the connection open to send messages in real time. The connection is periodically interrupted in order to generate a new request.
The median is the number that divides a set of results into two equal halves, each half containing the same number of values.
A Monitor observes the behavior of a component part of a monitored machine (web server, application server, operating system, database, etc.). A Monitor groups together several performance counters and indicators.
A monitored machine is a server, or other device in the infrastructure, that is to be monitored. A monitored machine may be a physical machine or a network device. A monitored machine groups together several Monitors.
NTLM (NT LAN Manager) is an authentication protocol used by many Microsoft servers. The protocol uses encryption to securely transmit passwords. See the Microsoft website for further details.
The pacing delay is the minimum time a Transaction must take to execute. If the Transaction execution takes less time than the pacing time set, the User Path will wait until the end of the pacing time before continuing the scenario.
The last element of the monitoring infrastructure which retrieves the numeric values on a Monitor.
A performance indicator is a text value, comprising several lines of text, that shows the state of one of the monitor elements.
Polling is a mechanism used by the push technology whereby a request is sent by the client to the server at regular intervals. In return, the server updates the status of each connected client. Between two polling requests, the server stores the updates sent to the client in its memory until the next polling request is received.
A Population is a group of Virtual User types which allows the application to be tested using different business actions. One typical example is when testing an e-business website with 90% of users browsing the catalog and 10% making an on-line purchase.
Push technology is a type of communication where information is "pushed" by the server to the browser, contrary to the normal "pull" principle whereby the browser initiates the request for information. There are several Push methods: polling, long polling and streaming.
In push technology, a push message is a Container being composed of the actions or business transactions to be executed when a message is received. A push message is defined by a list of conditions that must be met for it to be executed.
The recording proxy is the NeoLoad and Neotys Design Studio component which records the requests between the application and the server in order to facilitate the creation of a load test scenario.
Regular expressions are used to find sub-strings that match a specified pattern. In NeoLoad and Neotys Design Studio, they are used to define character strings for extraction (see Variable extractor) or rules to validate text (see Assertion).
Rendezvous points help emulate heavy user load on the server. Rendezvous points instruct Virtual Users to wait during test execution for multiple Virtual Users to arrive at a certain point, in order that they may simultaneously perform a task.
NeoLoad response times are calculated from the time the first byte of a request is sent, until the time the last byte of the response is received, namely including network time to send request and receive response.
Scalability testing helps ensure that a web application is scalable, that is, it can handle an increased workload with added resources and determine whether it best scales up, by upgrading the server hardware, or out, by improving the application code to distribute processing across several servers. Scalability testing is performed as a series of load tests with different server hardware or software configurations while keeping other test environment characteristics unchanged.
A scenario sets out all the test parameters: duration, load policy (constant, ramp-up, peak or custom), number of Virtual Users to be generated, and so on. NeoLoad can manage several scenarios for each project.
SOAP (Simple Object Access Protocol) is a lightweight protocol for the exchange of information in a decentralized, distributed environment. It is an XML-based protocol that consists of three parts: an envelope that defines a Framework for describing what is in a message and how to process it, a set of encoding rules for expressing instances of application-defined data types, and a convention for representing remote procedure calls and responses. See W3C for more details.
Standard deviation is a measure of the variation in response times compared to the average. A high value indicates wildly fluctuating response times, whereas a low value indicates that the response times are consistent and stable.
Streaming is a mechanism used by the Push technology whereby a blocked request is sent by the client to the server. In other words, the client never completely finishes reading the server response to the request, the server sending back several separate response packets in the same flow.
Stress testing helps identify performance bottlenecks in a web application, determine the peak load (for example, the number of users) that brings the application into an unstable state and verify that it successfully recovers from fails. Stress testing means testing beyond the limits of the expected production conditions.
Think time is the time the user normally takes to make a decision to do the next task. It is the simulation of the time taken by a real user to read one page before clicking to the next one.
It is possible to use a global think time for all pages (use a variable to obtain a dynamic value within an interval), or define a think time for each separate page.
A Transaction groups the different web pages that make up a "business transaction". In other words, a Transaction is a set of web pages that carry out a business exchange. A typical example is the set of pages used to connect to an application. Transactions are clearly marked in the test results to make analyzing download times easier.
The truncated mean is calculated on a reduced set of results, excluding the extremes. This enables non-representative values to be ignored, giving values close to those generally obtained by end users.
The time taken, in seconds, before receiving the first byte of the response. This is the time taken by the server to start sending its response, excluding the download time. It includes the time to create the connection to the server, the time to send the request and the time for the server to generate the response. This indicator is sometimes useful in detecting network bottlenecks. If the TTFB is short, but the response time long, it may indicate a network delay issue.
A User Path is a set of Transactions, web pages, requests and/or logical actions that describes the browsing actions performed on an application or a website. It can be recorded or manually defined in order to be replayed by n number of Virtual Users performing the same actions.
A proxy in the usual sense: a proxy server used to connect to the web server.
Variables are used to add dynamic content to a scenario, for example, by submitting dynamic form parameters or connecting to the server using different user accounts. The different types of variables include: counter, list, random integer, random string, date, etc.
A variable extractor is used to extract content from a server response during a test and assign the extracted content to a variable. This variable can then be used for the rest of the scenario.
A Virtual User simulates a user by executing a User Path. A typical load test can contain one or several User Paths, each User Path being executed by several Virtual Users.
A Virtual User instance is a specific execution of a User Path. During a load test, several User Paths are running at the same time, each of them being a Virtual User instance.
See Virtual User
Page as rendered by a web browser. It consists in the HTML page and all its sub-components, such as image files and CSS style sheets.
WSDL (Web Service Definition Language) is a specification defining how to describe web services in a common XML grammar. See W3C for more details.
XPath is a non-XML language used to describe one part of an XML document.