Aditya Ganjam is Conviva‘s Chief Product Officer and Co-founder.
Delivering an exceptional user experience is table stakes for running any app or digital service today. While major outages and service disruptions can frustrate users, the real battleground lies in addressing the everyday experience issues that collectively have a greater impact on business metrics, such as user satisfaction, sales conversions and customer retention.
Traditional monitoring and observability approaches primarily focus on system performance and uptime, failing to capture the nuances of the user experience. Without visibility into user experience, businesses lack insight into all the issues impacting users: pauses or delays in an app’s performance or failed credit card payments.
To truly understand and optimize users’ experiences, companies must adopt an experience-centric approach that starts from the client-side device, meticulously tracking all user actions and system responses across all devices in real time.
By computing quality of experience (QoE) metrics from this data for every user and across every critical user flow (e.g., the user flow from sign-up to search to purchase to watch) and analyzing them across fine-grained user cohorts (e.g., all combinations of device models, operating systems version and app versions), businesses can identify specific users experiencing issues, diagnose the causes by correlating them with system performance data and enable real-time remediation through AI-powered alerts.
This outside-in, user-experience-centric methodology represents a shift from the inside-out approach of many traditional monitoring and observability tools, which start from system performance and often are blind to the nuanced impact on end users.
Understanding Quality Of Experience
QoE is a multifaceted concept that encompasses the various elements that directly impact a user’s satisfaction when consuming digital content or services. It goes beyond simple delivery, focusing on the entire experience across every critical user flow, such as sign-up, finding flights, booking flights and payment.
Factors affecting QoE can be found throughout the user journey:
• Sign-Up And Login: Complex registration forms, unreliable logins, error notifications and difficult password recovery processes
• Product Search, Selection And Add-To-Cart: Poor search relevance, slow loading, unclear product information and buggy cart functionality
• Checkout, Purchase And Payment: Complicated checkouts, slow payment processing or lack of clear confirmations
• Transactions: Slow processing, lack of real-time updates and transaction recording errors
• Video And Audio Quality: Poor resolution, inconsistent quality and lack of adaptive streaming
• Buffering And Playback: Frequent buffering, slow starts and navigation issues
• Latency: High latency in live-streaming interactive applications or e-commerce functions
• Ad Experience: Intrusive, frequent or poorly targeted ads and technical playback issues
• User Interface And Usability: Unintuitive navigation and inconsistent design across devices
It’s important to note the differences between QoE and quality of service (QoS). While QoS focuses on technical performance metrics, QoE takes an experience-centric perspective. QoE measurements are more aligned with user perceptions and satisfaction, whereas QoS typically deals with network and system performance indicators like load, errors and saturation.
Computing And Monitoring QoE
Traditional analytics approaches for user experience often rely on a combination of product analytics and observability solutions, both of which have gaps.
Product analysis tools measure past user engagement but lack real-time visibility into the user experience, including why engagements dropped or increased. Observability tools provide real-time insights but focus on signals from the backend systems that support apps that are disconnected from the user’s experience.
The move to experience-centric operations requires a new, superior approach. Digital apps expose a rich set of data, including every user action, such as a click, and system response, such as an API call. These data can be used to compute how users are experiencing an app across every critical user flow.
Experience occurs in fine-grained user cohorts. Checkout may be slow only for Android operating system users running version 2.4 of a retail store’s app. There might be hundreds or thousands of users in an impacted cohort within a company’s user base.
Finding these cohorts is a computational and AI problem that requires analysis across thousands of dimensions, such as all combinations of device models, app versions, geographical locations and other factors. Key user flows and QoE metrics to consider include:
• Search: Tracking the performance and effectiveness of users’ search, whether it’s to find a product, flight or content. Computed QoE metrics for this flow include time to search, time from search to selection, and search processing time.
• Login: Tracking authentication success rates and processing times from a user’s perspective. Low rates may indicate system or process issues. Computed QoE metrics for this flow include time to log in, login success rate, and login processing time.
• Purchase: Tracking purchase transactions and payment processing. Computed QoE metrics for this flow include time to complete purchase, purchase processing time, and repeated purchase failure rate.
• App Bounce Rate: Tracking the percentage of users quickly leaving the app. High rates suggest potential UX problems, such as slow app startup times.
• Video And Audio Quality: Monitoring audiovisual metrics to ensure high-quality content delivery. QoE metrics for this flow include time in buffering, video start failures and average bitrate.
Diagnosing And Resolving QoE Issues
The holistic view provided by computing user experience and system performance in real time enables technical and operations teams to eliminate blind spots in user experience analysis, reduce alert noise, enhance developer efficiency and lower operational costs.
By leveraging AI to understand the interplay between technical performance and user experience, businesses can make more informed decisions about where to focus their optimization efforts, enhancing overall user satisfaction and positively impacting business outcomes.
QoE encompasses all factors that influence and shape user satisfaction, like pauses, delays and failed credit card payments. This approach enables businesses to enhance the overall user experience and positively impact business outcomes, including reducing mean time to experience, meeting service-level agreements and improving technical and operations teams’ work experience and efficiency.
Understanding and optimizing QoE is not optional—it’s a necessity for businesses that want to thrive in today’s digital world. By adopting an experience-centric approach to monitoring and improvement, companies can meet user expectations, reduce churn and build lasting relationships with their customers.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
