Evaluating Wireframes with Usability Inspection Methods
Wireframes are not only for idea generation. They can also be an ideal tool for validating design solutions early in the product design process.
In this article, we’ll suggest how to conduct Usability Inspections of your wireframes to evaluate the usability of your design ideas prior to writing any code.
Usability Inspection is a set of methods where evaluators inspect designs for usability issues, typically before they’ve been implemented. We’ll discuss how anyone working on a product can conduct these studies with some basic instruction. You’ll learn how to try out your solution and find out if you're on the right track with your product ideas, thereby prevent missteps down the line.
Why test the ideas in your wireframes?
You might wonder why you’d test out the ideas in your wireframes when you could just wait to test your working website or app.
First, let’s say that you do want to consider testing your working product if you can. But what if you wait until you have a functional product and uncover issues that require revisiting and altering your solution? You might have to go back to the drawing board, explore new ideas, and then rebuild. As a result, you've lost time in your pursuit of shipping.
As valuable as wireframing is to finding the right design, there is always the possibility that you’ve made decisions that render your solution sub-optimal, or even unusable. We want to prevent that from happening. If you consider that you spend less time crafting wireframes compared to the combined time of wireframing, visual design and development, testing wireframes should make a lot of sense. Changing a solution is much less painful when it exists in something that can easily be discarded, like a wireframe. Changing a working product can be costly.
We should note that there may be some potential downsides, however. You are testing out ideas with a static model of your product. Static screens may go only part-way toward representing its final state, so some things may be harder to test, like the “feel” of interactions such as animated responses in the interface. These kinds of behaviors might be more suitable to test on working code. However, the core functions and user flow are good candidates for testing.
About usability inspection
There are many types of usability study you could consider for evaluating wireframes. Usability inspection methods are useful because they allow you to assess your product without directly testing it on users.
Using these techniques, you evaluate your solution based on understanding user needs and usability principles. The goal is to discover where to focus on improving usability in your product.
Usability inspection is an attractive approach to finding flaws in your design because it tends to be inexpensive to conduct compared to formal usability testing, and these methods help discover issues before customers use the product. Additionally, since they can be run repeatedly quite easily, they may provide incremental improvements over each inspection and design iteration.
Cons and caveats
A downside of usability inspection techniques is that they are theoretical exercises. They are not adequate replacements for watching actual users perform tasks on your product for measuring the usability and success of your product. Ideally you perform your evaluation and follow up by testing users.
The evaluator’s bias may also alter the results compared to actual observation of users. It’s therefore beneficial to employ an expert if possible, and to run the evaluation with multiple evaluators.
Lastly, as the Interaction Design Foundation note in their article on conducting heuristic evaluation, the Nielsen principles were created in a different era for interactive products, compared to the technologies and conventions that exist today. They suggest that while the original heuristics are still applicable, companies should develop their own heuristics, relevant to their product and market.
If you’re interested in learning more about usability inspection, see the summary of usability inspection methods found on the NNg site.
Planning your evaluations
First, let’s discuss how to plan and prepare.
Leading up to evaluations, you should have the following in hand.
1. User profiles
You’ll use user profiles to describe your users and frame the scenarios of features you wish to test. These well-described profiles will be used as proxy actors for inspection techniques. A Jobs to Be Done approach would be ideal in framing your users, as it would contribute a description of jobs and goals without being prescriptive of how they’ll accomplish them.
The goals or use cases that describe what the user will accomplish with your product will form the direction your evaluation will take.
The scenario is made up of actions the user is expected to perform to complete tasks in pursuit of the goal. This is the flow and interaction in the interface that we are inspecting and evaluating. You might revisit the requirements and supporting task flows for the project to flesh out your scenarios.
Wireframes are the object of your inspection. They need to provide the applicable screens with enough detail to cover the scenarios to be tested. These can be static documents or linked as click-through prototypes.
Evaluators perform the usability inspection. Ideally they are usability experts, and typically they may be the user researchers and designers working on the product.
As part of usability inspection and usability testing, you will need to make a plan for what to test.
First, you’ll define the scope of what you want to assess and select user goals or use cases to evaluate within that scope. Along with that you can reference what is out of scope.
Next, describe the actions and ideal scenario to achieve each goal. For example, if the goal is to sign up to create a new account on a website, it might look like this:
Example goal and scenario:
- Open browser
- Navigate to website
- Select the Sign Up button
- Enter email address
- Enter password
- Select Sign Up to submit the form
- Read confirmation message instructions
- Check email to confirm
These scenarios will be used differently depending on the usability technique employed, but for each, you’ll want to outline these basic scenarios and include them in your plan. As you’ll see, they guide how you proceed through your evaluations.
Now you’re ready to write up an evaluation plan that you can review with your team. It should provide a description of the test scope and procedures. This will include, for example, a description and scope of what will be evaluated, details about how the exercise will be conducted (e.g. description of the prototype, naming of evaluators), and any notes about the measurement tools (e.g. heuristics used, definition of rating scale for evaluation), etc.
After reviewing the plan with your team, you’re ready to begin. In the next section, we’ll show you how to conduct 2 different kinds of evaluations: cognitive walkthroughs and heuristic evaluations.
A cognitive walkthrough is a method for evaluating product usability employing the user profile as a proxy for actual users. It’s a task-oriented inspection with the primary goal of answering fundamental questions like, “will users be able to perform this scenario to complete this task?”
Walkthroughs help assess the initial user experience, particularly with respect to how new users would perform.
Cognitive walkthrough procedure
1. Walk through the tasks
Evaluators work through each of the scenarios in your plan using the wireframes. As they attempt to complete each goal, they will record answers for each action in the walkthrough document.
2. Document the walkthrough
The cognitive walkthrough document will contain the following information. You can use a table or form to record your walkthrough.
The table should break the goal into actions, listing a task per row in the first column. There should be columns each evaluation question, and a column for your notes or recommendations.
Cognitive walkthrough document
- Describe the steps the user will take to complete a task.
- Evaluate the action from the user’s perspective.
- Will the user try and achieve the right outcome?
- Will the user notice that the correct action is available to them?
- Will the user associate the correct action with the outcome they expect to achieve?
- If the correct action is performed; will the user see that progress is being made towards their intended outcome?
As you work through the scenario, mark each evaluation Yes or No. In the last column, add notes to explain your answers and provide recommendations if applicable.
You can make a copy of the example Cognitive Walkthrough template spreadsheet below if you’re signed in to a Google account.
3. Summary and next steps
The final activity is to review all of the walkthroughs performed by the evaluators. As you discuss each evaluator’s results, you’ll get a better understanding of your assumptions about users and what you think they can or cannot accomplish. As a team, you can identify the problem areas and discuss recommendations for improving them. You may also need to present your findings to your larger product team, so preparing a report may be important at this point.
Heuristic evaluation is a method for finding flaws in a design by judging it against usability heuristics (“rules of thumb”) that are known to make interfaces easy to use. Evaluators perform a set of tasks on the product, assessing how it performs against a set of usability heuristics. Areas that do not measure up provide insight into what to focus on to improve the product.
While it is ideal to engage a usability expert, this evaluation can be also performed in-house by someone in your UX team who has familiarized themselves with the heuristics and procedure.
The usability heuristics that we commonly use for evaluating a product’s usability are those established by Jakob Nielsen. They have been popularly utilized in the software development industry for years as a reliable set of principles that we can compare our products against, and we would suggest starting with them.
- Visibility of system status
- Match between system and the real world
- User control and freedom
- Consistency and standards
- Error prevention
- Recognition rather than recall
- Flexibility and efficiency of use
- Aesthetic and minimalist design
- Help users recognize, diagnose, and recover from errors
- Help and documentation
IXD Online has a video showing students doing a Heuristic Analysis on a paper prototype that you can watch on YouTube. Seeing a session in progress should give you a practical idea of how you might conduct your evaluation.
Nielsen’s heuristics were written in the 1990s, and while still applicable, you may also be interested in learning how to apply them with current technologies. The series of articles on heuristic analysis by Melissa Velazquez and Harsh Gorasia provide examples that are relevant and up-to-date.
Heuristic evaluation procedure
As with cognitive walkthroughs, you will have your prerequisites in hand and select your evaluators. Unlike the cognitive walkthrough, however, heuristic evaluations tend to be less directed explorations of the interface, so you will not direct the evaluator on how to work through scenarios.
1. Evaluate the wireframes
To begin, an evaluator may simply be given a description of what is being tested. They may start to explore the product and make observations about the user experience from any perspective.
Using a heuristic evaluation form to document the evaluation (often in the form of a table or spreadsheet), evaluators will work through the product wireframes, evaluating them against the heuristics as they work.
2. Document usability issues
The evaluator will use a document or form including the following information.
Heuristic Evaluation Document
- Wireframe Name and UI Element
Identify the area affected on the screen.
- Usability Heuristic that Applies
Indicate which heuristic applies.
- Rating / Severity
Ordinal scale describing how severe the issue is.
- Notes (and recommendations)
Observations that describe the issue. (Note both positive observations and issues that go against usability heuristics.) Recommendations for how to address the issue. (These should be high level recommendations rather than detailed solutions.)
If you’re starting out with heuristic evaluations, you might find it useful to start from a template — there are plenty of examples. It’s common to create a spreadsheet like the one below or you can make one to represent your own needs.
For example, we provide a screenshot of the area affected and use markup and annotation to be clear about which UI element is affected. For the severity rating system we use values ranging from positive feedback to catastrophic issue: 0 Positive, 1 Cosmetic Issue, 2 Minor Issue, 3 Major Issue. 4 Catastrophic Issue.
You can make a copy of the example Heuristic Evaluation template spreadsheet below if you’re signed in to a Google account.
When the evaluator identifies an issue that appears to contradict a heuristic, they make a note in the document about the corresponding screen and the interface element. They select the rule that is being violated, select a rating for the severity of the issue, and make a note about why they believe this interface element raises a usability issue.
If you’re doing a review of an existing web-based application or website, there is a Chrome Extension called UX Check that simplifies the review. As you view a web page, you click to highlight problem areas in the page, select an applicable principle that is being violated, write notes describing the problem, rate its severity and make recommendations.
3. Summary and next steps
When your heuristic evaluation is complete, it’s time to analyze and summarize your results. Organize your findings in a way that makes it presentable to the stakeholders and product team, summarizing the findings. Some evaluators like to organize by severity, and others by heuristic. Begin to write up an executive summary with a high level description of the types of issues encountered.
The final report should be a detailed analysis that describes the type of issues that could be addressed, and noting which issues most severely affect the usability of the feature. This document should provide enough insight into areas that will benefit from improvement.
Use this as a starting point to explain the core usability issues with your team, and provide recommendations for how these areas might allow the user to be more successful at accomplishing goals. These discussions should spark discussion and exploration of ideas for the next design iteration.
These usability inspection methods can provide a lot of value in the product design process, with very little investment. Furthermore, because you can employ these techniques early in your design process, they might help reduce risk and lost time with regard to usability issues.
As with all types of usability research, these activities can be best performed by a usability expert, but with some knowledge of usability principles and some time spent learning the methods, you should also be able to conduct these exercises in-house with members of your product team.
Keep an eye out for our next article in this series on usability research, in which we’ll discuss testing your wireframes with users.