#NeotysPAC – Measure User-Perceived Time in Production without APM Tool, by Ankur Jain

Measure User-Perceived Time in Production without APM Tool

People are more aware of performance today than a decade back. All the benefits of including performance testing/engineering early in the development cycle are well documented. This has helped in debunking major performance myths to quite an extent. I still believe that most of us are unaware of the importance of doing UI performance. The majority of the population still consider server-side performance improvements to be the only way of making our websites faster. In this blog post, I will try to take on these myths and enable you to leverage the real user’s data to improve the end-user experience.

With ever-changing technology and web development techniques, almost every other website today relies heavily on JS, CSS, DHTML, AJAX, etc. Also, to stand out among competitors, businesses wanted to have high-resolution graphics and heavily animated websites. All this has led the user-perceived time to shoot well above the industry benchmark of 3 secs. As per a survey conducted by Speed Matters Vol. 3, 75% of the participants consider that the speed at which page loads to be a significant factor in user experience.

Source: Speed Matters, Vol.3

Hence it makes sense to measure UI/object load testing time to improve user experience. Measuring web performance can be achieved in a couple of ways, namely, Synthetic Monitoring and Real User Monitoring (RUM). As the name suggests, synthetic monitoring is done in a controlled pre-production environment or lab environment. It can help in the early detection of performance issues and benchmarking competitors. Few tools that I truly enjoy using are Webpagetest, Yslow, google page speed insight, Google’s lighthouse, and very convenient performance tab in the browser’s developer tool. Although Synthetic monitoring is a perfect way to begin discovering performance issues, they may not unearth all the answers to bad user experience for the actual end-users. This is because user-perceived times in the real world may depend on numbers of factors like:

  • Internet speed
  • Browser type and its version
  • Machine configurations (RAM, CPU, etc.)
  • Mobile or Desktop user
  • Geolocation/latency

Simulating and testing the website against all these conditions in a lab environment incur a lot of person-hours, and still, you might not be able to cover everything. Hence it makes a lot of sense to monitor real users in production. All the performance data gathered from the real chaotic world can also be correlated with business Key performance indicators (KPIs). Real User Monitoring (RUM) can thus help in better understanding of conversion rates, bounce rates, business revenue, and customer satisfaction, among others. It can easily be said that both Synthetic, as well as RUM, is required for a 360-degree view of the application.

But, measuring user-perceived time is not as straightforward as it seems. During the early days of the web, most of the websites were simple multipage applications with only HTML and images. It was perfectly fine to measure image download time and consider that to be a user-perceived time. Fast forward to today, the web has now become very complex with heavy reliance on JS, CSS DHTML, etc. The processing of these applications happens at the client-side or browser than on the servers. It became all the more important to look beyond server-side performance. Also, most of the websites these days are Single Page Applications (SPA). SPA usually has a single onload event with multiple page rewritten events based on user actions. Hence traditional performance metrics like window.onload or document complete events do not translate well with user-perceived timing. Other widely used metrics like first paint, first contentful paint, time to DOM Content ready, etc. also give misleading numbers. There are few newer alternatives developed like speed index, visually completed, and above the fold render time. But all of them consider every pixel on the viewport as equal. They are ignorant of what content the user is most interested in on the page. Moreover, all these newer methods capture a series of screenshots or video and require highly compute-intensive calculations to come up with performance numbers. This makes it impossible to be used for Real User Monitoring (RUM).

So, we need a metric that captures the user’s perception of when the useful content on the webpage is rendered. The solution should be:

  • Accurate
  • A standard across browsers
  • Measurable for real user
  • Lightweight and cost-effective

Let me introduce you to W3C User Timing Spec that enables web developers to add custom metrics to the web apps. User timing APIs consists of below essential functions:

  • mark(“MarkerName”) – stores time taken since navigation start
  • measure(“MeasureName,” “StartMarkerName,” “EndMarkerName”) – records the delta between two marks.
  • clearMark(“MarkName”) – clears any data associated with MarkName
  • clearMeasure(“MeasureName”) – clears any data associated with MeasureName

User timing APIs are arguably the most useful performance measuring method in our browser to create application-related timestamps. These timestamps are part of the browser’s performance timeline. The marks and measures when combined with a deep understanding of how browsers work can be quite useful in answering most of our questions related to user-perceived time. Moreover, user timing APIs are well supported by all the leading web browsers.

Let’s see a few ways of adding these marks and measures in the web app.

  • Adding mark below all the stylesheet LINK tags to capture when stylesheets have done page render-blocking.

<link rel=”stylesheet” href=”style1.css”>

<link rel=”stylesheet” href=”style2.css”>

<link rel=”stylesheet” href=”style3.css”>

<script>

      performance.mark(“stylesheets done blocking”);

performance.measure(“stylesheets”,undefined,”stylesheets done blocking”);

</script>

  • Similarly, placing mark below all the synchronous scripts can be useful in knowing when scripts blocking is finished.

<script src=”analytics.js”></script>

<script src=”page2.js”></script>

<script>

      performance.mark(“scripts done blocking”);

</script>

  • It can be added to the onload event of an image to capture when the image is downloaded and rendered on the page.

<img class=”imgclass” src=”https://www.w3schools.com/w3css/img_lights.jpg” onload=” performance.clearMarks(‘image10:displayed’); performance.mark(‘image10:displayed’); “>

<script>   

performance.clearMarks(‘image10:displayed’); 

      performance.mark(‘image10:displayed’);

</script>

  • But the most useful way of using user timing APIs are to add them in JS to get key JS execution milestones.

<script>

function test(){

performance.clearMarks(“start function test”)

performance.mark(“start function test”)

// Do some XHR or json requests.

performance.clearMarks(“end function test”)

performance.mark(“end function test”)

performance.measure(“test time”,” start function test”,” end function test”)

}

</script>

Now the last piece of the puzzle is to collect this data from the real users in production. This can easily be achieved by writing a lightweight JavaScript and add it to the webpage. This JS will retrieve this data from the user’s browser and sends it to a backend server, which then stores it to a time-series database, like InfluxDB. Fetching these marks and measures from the page can be done by using a performance timeline API. The Performance Timeline API extends the performance interface with three methods that provide different mechanisms to get a set of performance records (metrics), depending on the specified filter criteria. The methods are:

  • getEntries() – returns all recorded performance entries in the webpage
  • getEntriesByType() – returns the recorded performance entries based on the specified performance type (e.g., Mark, Measure)
  • getEntriesByName() – returns the recorded performance entries based on the specified name (e.g., MarkName, MeasureName)

For this year’s PAC, I created a demo photo gallery application. This application is choked with marks and measures all over the webpage. As the end-user interacts with the application, user-perceived timing data is stored in the browser. Here is how the photo gallery application looks.

Photo gallery app

As you can see in the above image, I have retrieved the timing of these marks and measures in the console log of the developer tool. This is one of the ways these values can be read. To have a more automated approach, I retrieved these timing data using an embedded JavaScript. This script sends the timing information to a backend web server. This webserver then saves the data into an InfluxDB. For visualization purposes, I connected this InfluxDB to a Grafana dashboard. Here is how my Grafana dashboard looks.

Grafana Dashboard

As suggested by other PAC experts, I tried a scalability test of my backend server and found out that I could easily send up to 100 write requests/second (>200 million hits/month) to my Influx server running on t3.large EC2 instance. This simple and cost-effective solution can easily be implemented for the majority of web applications having less than 100 hits/sec web traffic. For enterprises receiving higher traffic, they can look into Influx’s hardware guidelines to have proper configuration to scale. Happy Monitoring!!

How do you measure your real users? Let us know in the comments section below.

 

 

Learn More about the Performance Advisory Council

Want to see the full conversation, check out the presentation here.

Leave a Reply

Your email address will not be published. Required fields are marked *