Performance Testing Using RPA – To Move or Not to Move

Robotic Process Automation (RPA) is on an upward trend throughout the enterprise. Every day more companies are replacing tedious, manual activities performed by humans with digital automation processes handled by computers. RPA will prove useful for streamlining clerical tasks such as documentation management, regulation compliance, and supply chain fulfillment.

To ensure that RPA implementations work as intended, companies will need to design and implement unique testing strategies. Such planning is sound and necessary. However, there is an entire class of robots in the Robotic Process Automation that will require companies to rethink the way they approach RPA testing fundamentally. This class of robots moves through physical space to complete the work, e.g., a robot that pulls an item from the shelf in a warehouse, loading it onto a truck for transport. Testing of these robot types requires moving beyond the standard techniques traditionally used for software testing. The challenges are real, the implications profound, particularly when it comes to performance testing the robot.

Performance Testing in Time and Space

Nearly all software performance testing is executed on a device that is stationary. While it’s true that cell phones and tablets are devices intended to move around through time and space, when it comes to performance testing scenarios, these devices are motionless. It’s not like the tester has to “herd cats,” pinning one down before testing can begin. Typically such devices sit on a shelf in a mobile device farm at a test lab. The device is motionless. Moving the device around has never been a significant consideration in most performance testing scenarios. But, what if the device movement does matter?

Consider this scenario as described by Helen Bally last year at the June 2018 gathering of Neotys’s Performance Advisory Council. She was the performance testing lead on the implementation of an SAP Enterprise Warehouse Management System. The primary device for inputting data into the system was a handheld RF tool that warehouse workers carried as they moved about the warehouse. They used the device to read barcodes that identified a particular shelf in the warehouse as well as items on the shelf. Also, the device had a small keyboard with a reduced set of keys for performing other types of data entry.

Bally’s testing objective was to get the rate of entry into the system to under a second. She thought the task would be easy but learned otherwise:

“My initial thought was that isn’t shouldn’t be that difficult. We’ll just have to measure them. But, I started talking to the experts who knew a lot more about the Extended Warehouse Management System. I figured out that testing what the clicks would be under real-world conditions and demonstrating the repeatability and accuracy of those was going to be quite challenging.”

What Bally came to understand is that movement matters most when performance testing a mobile device in a warehouse. Workers used to an RF tool for data entry only when they were physically near an item of interest. So, during a light day in the warehouse, with only a few workers walking the aisles, data entry was faster compared to the busier times when many workers were active. It’s a simple fact. The more workers trying to access items in the warehouse, the more clogged the aisles became preventing other workers from getting through. At that point, the performance issue had nothing to do with the digital piece of the process. The problem was physical access. Fewer workers translated into clearer aisles. More workers resulted in clogged aisles and degraded system performance.

Improving system performance was more of scheduling vs. a capacity issue. Additional computers wouldn’t have mattered. Increasing network bandwidth – not that either. Optimizing algorithmic efficiency was not going to yield benefit. To get the most out of the SAP implementation, the warehouse had to optimize the number of workers in the aisles. It was a matter of time and space.

Moving Beyond Data Processing

Despite the tablet market’s suggested decline (according to, and the cell phone market expected to see slower rates of growth by 2022, warehouse robotics is a $2.28 billion industry with projected yearly growth of more than 10% for the next five years. More robots will be appearing in everyday commerce, all the way from driverless vehicles to chambermaid machines that will clean hotel rooms tirelessly day in and day out. The performance testing of these devices in an all-inclusive manner integrating hardware, software, and conditions from the physical environment will become more critical. Remember, the term is not Office System Process Automation or Mobile Device Automation. It’s Robotic Process Automation. Therefore, to determine the real benefit of RPA, businesses will need to conduct performance testing regarding the moving machine as well as the software and the process.

RPA performance testing software that resides on an array of computers in the Cloud or cell phones and tablet in a device farm will still be useful in the short term. But, as Robotic Process Automation evolves to the point where intelligent, agile, moving robots become a more significant part of business processes, a new way of implementing performance testing will need to be found. Forward-looking thinkers contemplating the future of performance testing are already preparing for this eventuality. This new avenue for performance testing is a path we could all benefit from adopting.

Learn More

Discover more load testing and performance testing content on the Neotys Resources pages, or download the latest version of NeoLoad and start testing today.


Bob Reselman 
Bob Reselman is a nationally-known software developer, system architect, test engineer, technical writer/journalist, and industry analyst. He has held positions as Principal Consultant with the transnational consulting firm, Capgemini and Platform Architect (Consumer) for the computer manufacturer, Gateway. Also, he was CTO for the international trade finance exchange, ITFex.
Bob’s authored four computer programming books and has penned dozens of test engineering/software development industry articles. He lives in Los Angeles and can be found on LinkedIn here, or Twitter at @reselbob. Bob is always interested in talking about testing and software performance and happily responds to emails (

Leave a Reply

Your email address will not be published. Required fields are marked *