Are Speed and Quality Both Possible for Performance Testing in DevOps?

Digital transformation is urging organizations to adopt Agile and DevOps principles to deliver software on recurring, accelerated schedules. While many aspects of the software development process have made great strides towards Agile via Shift Left and automation, performance testing is still highly-manual and typically executed at the end of the SDLC.

This blog is the first in a series in which we will review three topics associated with digital transformation as we address the speed AND quality possibility for performance testing in DevOps:

  • Part 1 – The Coming Digital Transformation
  • Part 2 – What Makes a Great Tester in a DevOps Environment?
  • Part 3 – The Future of Performance Testing


Digital Transformation and its Impact on Speed/Quality for Performance Testing in DevOps 

Social Media, Mobile, Analytics, and Cloud are Driving Transformation

If you don’t believe we are in the midst of a global Social Media, Mobile, Analytics and Cloud (SMAC)-driven digital transformation, take a look at the level of effort established companies OUTSIDE of tech are exerting to deliver software/services to enable a better connection between their customers and their brands. They’re also making investments in self-re-invention as digitally-transformed businesses. One Japanese sensor company who’s adopted the notion that digital transformation is the key to future business growth is shifting their business model from manufacturer to service provider.

Another example is USA-based airline jetBlue. Recognizing it didn’t have all the internal expertise needed to transform itself, jetBlue setup their own Venture Capital arm to invest in tech startups who could help them improve passenger/employee experiences. jetBlue and other airlines companies have even opened up their vast data store to outside companies looking to leverage these data, using APIs.

To ensure survival and growth, companies must invest in, and be a part of, the coming wave of digital transformation. On top of all this change, the velocity with which we must deliver new applications and capabilities to market is faster than ever before and getting faster still! But you don’t need to do it in one giant leap. The least painful way is to go step-by-step – I’d like to share with you some of our findings.

The Market Impact of Digital Transformation

Let’s take a look at a few relevant statistics from Gartner:

  • By 2020, IoT devices will outnumber the total number of tablets, laptops, and mobile devices – estimated at >26B units => The technology is ubiquitous
  • CIOs expect 46% of the perceived value of their business being digital => The business impact of the technology continues to grow
  • 52% of the companies in the Fortune 500 from the year 2000 are gone => Even the mighty are mortal

The Impact of Poor Application Performance

What do poorly-performing applications mean for business? Since just about every business operation is enabled and mediated by applications, poor app performance impacts every area of the business.

  • 88% of 900 global large enterprise executives surveyed say their day-to-day work productivity is impacted by poor app performance by the apps they use to get their jobs done (Riverbed/Wakefield)

Poor app performance doesn’t just affect “the business” in some abstract way. Application performance is personal as well, impacting people directly and, when poor, making it harder to do their jobs and get things done. When apps are slow, what do people do in response?

  • In a recent survey[1], 37% of executives said their employees used apps not supported by the organization (“Shadow IT”) to get around problems using “approved,” production apps

Digital Transformation Impacts All Areas of the Business

The workplace itself is transforming as a result of:

  • Customer expectation fluctuation due to the continued react and adapt the approach to ever-changing skill set requirements
  • More diverse and globally distributed team orientation
  • A business environment with growing complexity by day

Of course, the digital transformation of the entire business has a much wider scope than simply network infrastructure and security. What is needed to move ahead with digital transformation across the business? Empowerment of test and development employees wherever they are located, stronger engagement with customers than ever before, operations optimization, and product transformation.

This transformation results in changes across the business, including changing requirements in three areas – Products (WHAT we make and sell), Processes (HOW we organize teams to make and bring to market these products), and People (WHO are the staff – makeup, levels, and skills needed within the organization).

Products (WHAT)

When we talk about the products and services companies bring to market, in this context, we are referring to the applications your teams are delivering to the business rather than what your company sells – I.e., building, automobiles, and banking or insurance products. These applications have become and will continue to be, the key drivers of customer engagement and business growth. Rather than an afterthought or alternative channel for goods/services distribution, the user perception of, and happiness with the performance of these applications has become mission-critical.

To be sure, application performance will always be a moving target. Every time a new player enters the market, the benchmark for application performance and capability raises. And, the new players may not even be on your radar as a competitor! Consider this:

  • 72% of companies are vulnerable to disruption within three years, according to Gartner
  • According to IBM, 60% of CMOs surveyed indicated that the greatest source of competition is expected to come from outside of their core industry

While companies are re-thinking their approach to app development to remain competitive, product architectures are evolving towards greater cloud integration of full-depth application stacks, and Microservices delivery in SaaS environments. Microservices architecture breaks a product into independent, bite-sized services “chunks” so that it can be deployed and managed solely at that level, with function indivisibility and autonomy.

Companies like Netflix and Amazon have adopted the Microservices concept in their products for ease of deployment and maintainability; but regarding performance, they do so to provide fast and highly-reliable apps. The most important aspect of Microservices culture is that whoever develops the service, it’s the team’s responsibility to manage it. Not having a baton pass, nor the problems typically associated with the hand-off fits with DevOps like a glove.

Because companies who adapt to, and embrace the impact of, the digital transformation will thrive and succeed, business leaders are hungry to empower their developers and testers to work more closely together. Such collaboration can transform and shape the work environment in a way that will allow more powerful, better-performing application delivery, faster.

Processes (HOW)

With these changes happening so fast, it’s no surprise we’ve seen the advent of DevOps and Agile. Using the Agile approach means delivery of many incremental changes, continuously, rather than the periodic launch of a new release via Waterfall methodology.

Testing methodologies and approaches also need to be re-engineered using advanced methods and tools – I.e., Microservices creating every functional aspect as an individual service, with many services in a big project. Each service communicates through API/remote calls, adding more overhead vs. monolithic software inter-process communication calls, which can impact performance. And, when there’s an SLA failure or performance bottleneck identified during performance testing in DevOps, isolation down to cause can be painstaking for the DevOps team.

A large enterprise currently moving to Microservices recently shared some of their findings along their way: no end-to-end visibility, a lack of audibility and traceability with some application support challenges, issues with domain-driven context/bounded context to define the right level of granularity, and designing for non-functional requirements [Performance/Scalability/Configuration] testing (E.g. what is the right amount of cache, etc.?).

Having a clear demarcation of responsibilities and thoughtful approach to application testing and deployment planning workflows is what is needed to be successful in transitioning to Microservices.

People (WHO)

The mix of skills needed by testers is changing as part of the business transformation. Testers are being called upon to learn skills once the sole responsibility of developers, and as coders become “full stack” devs, you have to hold onto the real value you bring to the business – collaboration, judgment, and analysis. The skills, experience, and knowledge of component interaction to successfully build applications which perform well, and ability to solve performance challenges during development and testing, are critical business assets. The demand for these credentials has never been greater, as well as building the experiences/tools that make it possible for every professional in any functional role to be more productive in getting things done.

[1] Source: Riverbed/Wakefield


In part two of this three-part series focused on speed AND quality possibility for performance testing in DevOps, we will explore “What Makes a Great Tester?” 

Leave a Reply

Your email address will not be published. Required fields are marked *