Industry Best Practices & Top Insights delivered to your Inbox.
Blog General

Usability Testing: A Comprehensive 2022 Guide

Shivkumar M Shivkumar M has over 20 years of experience shaping technology product and GTM strategy. With B2B SaaS expertise across industries, he leads product launches, adoption, and GTM as Director of Product Marketing.
Usability Testing: A Comprehensive 2022 Guide

What if you worked hard to create a new app, website, product, or feature only to find out that your users just don’t vibe with it? Usability testing is the key to discovering how real users experience your product before you bring it to market.
The reality is, when you build something, you become deeply familiar with it. So familiar that you might find yourself blind to its flaws. 
Just because you and your team find your product perfectly intuitive and enjoyable doesn’t necessarily mean your users will feel the same. That’s where usability testing comes in — it’s the ultimate bias-remover. 
Unfortunately, not many businesses make use of it. Twenty-eight percent of small businesses still do not have a website and more than 70% of small business websites do not even use a call-to-action (CTA) button. To top it all off, only 55% of businesses conduct usability or UX testing.
What Is Usability Testing?
Usability testing is the evaluation of a product based on how a sample group of users interacts with it. Through hands-on testing, product developers and marketers get a better understanding of the user experience (UX) through the eyes of people who are unfamiliar with the product.
A usability test begins with marketers and researchers recruiting a group of people who represent a product’s target market. Then, a facilitator leads the group through a series of hands-on tasks under controlled conditions while a researcher observes and takes notes. The researcher asks questions, notes where problems or frustrations arise, and takes stock of how the user experiences the product. 
A quick note: From here on out, we’ll use “product” as a blanket term to refer to apps, websites, and other hands-on products that benefit from usability testing.

The Purpose of Usability Testing 

When you have a team hard at work creating a product from start to finish, they develop a deep understanding of what it’s intended to do and how it works. 
In other words, through no fault of their own, your team has blinders on. 
Your user, on the other hand, comes in with no prior understanding of things like your product’s value proposition and purpose or how to use its various functions.
Ultimately, usability testing benefits your product. The feedback gathered during this period is used to create a prioritized list of recommended changes to improve the user experience. It can help you identify things like:

  • Whether or not people can easily complete tasks (like finding and executing a specific function)
  • How long it takes to complete certain tasks
  • Bugs or performance issues that your in-house testing didn’t reveal
  • Points of confusion about the functionality of your interface
  • The effectiveness of your calls to action

By helping you analyze how people use your product, usability testing ensures that you release a product, new feature, or update that real users in your target market can navigate easily and find value in. 

Usability Testing Benefits

The goal of usability testing is to get a better understanding of how real customers engage with, and experience, your product and then use those insights to improve the user experience. 
Usability testing benefits include:

  • Removing blinders. Usability testing will expose biases and help your team see things from the user’s perspective.
  • Gaining insight into ease of use. Usability testing can reveal whether your product is intuitive enough for your users to interact with easily.
  • Determining whether your product is engaging. Will your users find it fascinating or dull?
  • Validating data points. If data shows that users aren’t interacting with your product as expected, watching them in action can help to reveal problems.
  • Enhancing quality. Usability testing can help detect small errors or glitches that might otherwise go unnoticed.

Usability testing is about seeing your product through the lens of your user’s experience with it, and then making your product better as a result.


When To Do Usability Testing

You may think you know when to do usability testing — when your product’s almost ready for launch, right?
Not quite.
For the best results, you should conduct usability tests at different stages throughout your development process, not just when you’re near the finish line. If you wait to perform testing until you’re almost ready to release your product to the world, you’ll lose valuable insight that could have guided the development process in its early stages. 
Instead, do usability testing when you have a minimum viable product, and then again when you have a working beta version, and again when you’re close to launching. Let your users guide you along the way. 
You should also conduct usability tests when you want to implement a new feature or find new ways to improve your product. After all, few products are truly ever finished when they’re released. There’s always room to grow and improve.
So, here’s the TL;DR. Usability testing is important when you:

  • Want to validate whether a prototype will gain user buy-in
  • Have a beta version that’s ready for outside testing
  • Need further insight into data points you’ve gathered
  • Are adding a new feature to an existing product
  • Have completed in-house testing and are nearing release

Usability testing helps you shift out of development mode, move past any blind spots you’ve developed, and empathize with your users so you’ll build a product that’s focused on both their needs and your goals.

Types of UX Testing: Functional Testing vs. Usability Testing

You may sometimes hear the terms “functional testing” and “usability testing” thrown around as if these two types of UX testing were the same. The truth is the two are very different.
Functional testing is a process to test — no surprise here — the functionality of a product. Basically: does the product work? It helps to ensure that the product behaves as it’s intended to without factoring in design or user experience. 
Meanwhile usability testing is basically: how does this group use the product?
Both functional testing and usability testing should happen throughout the development of your product. But always perform a functional test before moving on to a usability test. You want to ensure that your product is working as intended — at least as far as you can tell — before you try to gain insight from actual test users. 

How To Conduct Usability Testing

To perform an effective usability test, you’ll need to assign a facilitator and appoint some observers. Then you’ll assemble your user group and run them through your test tasks. You’ll wrap up by analyzing the data you’ve collected and recommending product improvements based on your findings.
Let’s look a little closer at the steps:

1. Establish an Internal Facilitator

A facilitator (also called a moderator) is the person who administers the usability test tasks to your user group. Their role is to direct the user group to perform various processes with as little how-to guidance as possible. 

2. Appoint UX Researchers or Observers

Your facilitator could take notes during a small-scale test. But with larger tests, it’s ideal to appoint one or more observers to gather intel. That way, the facilitator will be free to concentrate on leading the session while the observer remains laser-focused on the user’s experience.

3. Choose Your User Group(s)

Who’s going to use your product? Depending on your answer, you’ll need to assemble one or more user groups. 
If different segments of your audience will interact with our product in different ways, you’ll need separate groups for each. For example, if you’re developing an app that has both general users and admins, you’ll need separate groups and test tasks for each role.

4. Prepare Usability Test Tasks

What is usability testing without the actual tests? You’ll need to prepare about five to 10 tasks for your user groups to perform within a set time frame. 
Ask yourself what tasks your user needs to complete to accomplish their goals when they’re using your app. Think: 

“My user has to do [TASK] to [GOAL THEY WILL REACH].” 

e.g., “My user has to create an account to begin using the app.”

When you prep your usability test tasks, you should:

  • Consider your most crucial end-user goals first. As you outline the test tasks, follow the path you expect your user to take. If your user’s first step is creating an account, then the next might be logging in and then creating a user profile. Those are crucial goals.
  • Create an explanation for each task. The facilitator will share this explanation with the test group, so it needs to be descriptive and clearly written. You don’t want the facilitator to have to add information or provide clarity during the test, which could compromise the integrity of the results.
  • Establish metrics for evaluating performance. The point of testing is, of course, to assess whether your product works as it should or needs improvement, so make sure you know your test metrics in advance. They may include things like successful task completion, time on task, critical and non-critical errors, and subjective feedback scores on quality issues like ease of use and overall satisfaction.

5. Test Away!

Now it’s time to test. Gather your test participants, either in a physical location such as a meeting room or in a virtual environment. During the test, the facilitator should administer one task at a time by reading the task explanation.
Heads up! It’s important for the facilitator to explain the test task and then take a step back to let the user figure things out without further guidance. Remember, when your product is out in the world, there won’t be anyone standing over the user’s shoulder to show them how things are supposed to work. 
Imagine that the first thing you want your user to do after creating an account is set up a profile. If your facilitator says, “Click the menu to open your profile,” the user has been given directions they wouldn’t otherwise have. Instead, the facilitator should say, “Open your profile,” so your observers can learn whether they’re able to successfully complete the task without help.
As the user is performing the task, the facilitator may ask them to talk through certain steps. What are they trying to accomplish? What’s their decision-making process?
Whenever possible, record your testing process. Although researchers generally observe the tests and take notes, having a recording of the session (either captured live with video equipment or remotely with screen recording software) is invaluable for further analysis and communicating the results to other stakeholders.

6. Compile Data and Recommend Changes

Now that you’ve wrapped up testing, it’s time to comb through the piles of data you gathered. So dig in and analyze those recordings, session notes, surveys, and feedback forms. 
Of course, data analysis is a huge topic. But let’s review a few basic steps you’ll want to include in your review. 

  • Gather your data points. List your test tasks. Then, under each task, record every related data point you collected. Be sure to include relative notes about where in the process the problem occurred and any other details that will help your design team replicate the problem in further testing.
  • Prioritize issues. How much impact does the issue have on the overall user experience? Determine a scale. An example priority scale might be: Critical, Serious, Low-Impact, No Issue.
  • Recommend improvements. Once you’ve analyzed the data, present the findings to your development team and offer recommendations for improving your product. 


Usability Testing Methods

Usability testing isn’t necessarily a one-size-fits-all solution for product testing. There are a number of testing methods, and you’ll want to apply the one that feels most useful for your circumstances. 
Let’s look at a few of the most popular testing methods and where you might want to use them. 

Moderated Test

This is the classic usability test we’ve outlined in this post. The user is either in the same room with the researcher or observed remotely with screen-sharing software. The facilitator asks the user to accomplish a series of tasks one by one. The user’s process, reactions, questions, difficulties, and other points of interest are recorded by either the moderator or additional researchers.
Best for: When you need an in-depth analysis of user behavior.

Unmoderated Test

In this test, your user performs a series of tasks without someone watching them in real time. The data from the user’s device is captured for your team to analyze. Your user may also record feedback through a survey.
Best for: Capturing large quantities of data, especially when you’re further evaluating or trying to solve for an issue that cropped up during moderated testing.

Focus Groups

This is a variation on the moderated testing theme. Focus groups involve a small group of participants (about 5-10 sample users) gathered in a room to discuss their experience with your product. A moderator generally leads the group through a series of open-ended discussion questions over the course of a couple of hours. The session is recorded so the results can be analyzed.
Focus groups can also answer questions centered around the results of a previous usability test, providing you with further insight.
Best for: Homing in on your user’s attitudes, beliefs, challenges, and needs.

More Types of UX Testing

Beyond the structured usability testing process that we’ve talked about here, there are a host of other ways to test how users experience a product. They include:

  • Beta testing: Sometimes called early-stage testing, beta testing is a process used to gain insight into functionality, user behavior, and any bugs that need fixing. Beta testing might begin when you have a minimum viable product and continue until just before you bring it to market.
  • First-click test: This test simply reveals what a user would click on first within a product’s user interface. It helps you determine whether your users are immediately drawn to the part of your interface you want them to interact with.
  • A/B testing: The idea behind A/B testing is simple: Test a single variation in your UX against a control sample and see if there is any significant improvement in audience engagement (or any other marketing metric that matters for growth) due to the variation. An improvement can signify better usability or a better overall user experience.
  • Heuristic Evaluation: A heuristic evaluation is an analysis of your product’s usability by using 10 pre-selected criteria (AKA Nielsen’s 10 Heuristics for User Interface Design). To keep the results unbiased, the test is conducted by professional UX consultants from outside your team.
  • Surveys: Hands-on testing and careful documentation of feedback are essential to good usability testing. Although surveys don’t provide the level of detail that formal usability testing can, they’re a quick way to gain usable feedback and collect a large amount of data from a wide pool of sample users.

Usability Testing Cost

Here’s the big question: How much does usability testing cost? 
The answer isn’t as straightforward as you probably wish it was. Testing costs vary widely depending on a number of factors including:

  • Project scope
  • User sample size
  • Testing team size and compensation
  • Testing platform technology costs
  • Sample user recruiting costs
  • Sample user compensation
  • Testing facility rental fees

Speaking in very general terms, usability testing can cost anywhere from a few thousand dollars to hundreds of thousands. Naturally, the broader the scope of the test and the more moving parts it has, the more you’ll have to spend. 
You’ll also want to remember to budget for more than one usability test. To get everything just right, testing will always be an iterative process.
For an alternative video explanation of UX testing, particularly how to conduct moderated usability testing, watch this:

Real Insights from Real Users

As we’ve learned here, usability testing can be a complex, layered process. But ultimately, the goal is simple — to gather real sample user input to help you perfect your product. 
When you perform effective usability tests, you get insights quickly. Then, you can provide data-supported recommendations. 
Your recommendations will help your development team produce a product that functions smoothly, delights your users, and keeps them coming back to it again and again.
Schedule a demo to learn how CleverTap can help you A/B test and develop your customer engagement strategy.

Mobile Marketing is Easier with Expert Guidance

Posted on April 5, 2022