People tend to draw conclusions about others from themselves. We outline how companies can avoid these often expensive false conclusions while developing new products.
Today, hardly any company that wants to launch a new product does so without profound market research. An essential part of this assessment of needs is user testing. By answering the following 7 questions, we provide detailed insights.
- What is User Testing?
- Why is User Testing important?
- How does User Testing work?
- User Testing and Usability-Tests: What’s the difference?
- What is Remote Testing?
- When is User Testing useful?
- What does User Testing cost?
User testing is equally interesting for design teams, product owners and start-up founders. To sum it up: for everyone who wants to develop user- and market-friendly products.
What is User Testing?
The users of a (digital) product stand at the centre of user testing. During a test, they get to know the product or its prototype. They interact with the functionalities of a website or app via an interface and are observed in the process.. Both their behaviour and their statements confirm or refute the assumptions previously made by the UX design team. The results of the user testing influence the further development of the product.
Why is User Testing important?
- With the help of user testing it is possible to evaluate concepts at a very early stage.
- UX design teams get to know the users of a product and can thus uncover potential problems in use at an early stage.
- The tests not only help to improve customer satisfaction, but also ensure significant cost savings by avoiding errors.
- In addition, there is a considerable time saving, because important insights are found much faster through the user-based tests.
How does User Testing work?
In addition to the actual execution of a user test, the preparation and follow-up are of crucial importance. The entire process is divided into these 6 steps:
Preparation:
- As soon as the prototype of a product is ready, initial assumptions are made.
- These assumptions must later be verified or refuted in A/B tests.
- The hypotheses can be made on different bases, such as statistics, user research, heuristics or following the intuition of the team conducting the test.
- For each test, an individual test design is created that outlines the structure of the process and determines the use of formats and templates.
- The final step of preparation is the user casting. Social media platforms (FB, LinkedIn, Xing, Twitter) and advertising portals are suitable for this, as well as sending out invitations via newsletters.
- The number of participants depends on the type of testing. In qualitative testing, for example, it is perfectly sufficient to work with 5 users.
| Quantitative Testing | Qualitative Testing | |
|---|---|---|
| Goal | - Collecting facts and figures - Obtaining statistically reliable data |
- Developing empathy for users, their needs and problems - Exploring narratives from real contexts |
| Execution | Efficient questionnaires or similar, answered by numerous users | Open questions in intensive conversations with few users |
| Examples for results | - 6/10 users rated the product as not reliable - 47% of users would buy the product (again) |
- Users feel unsure about using a product - They like the product but are very dissatisfied with the connected customer service |
| Participants | > 100 persons | 4-8 persons |
Execution:
At the centre of testing is the user's interaction with the product – this applies equally to the digital and the physical. Various tools help to structure this interaction in such a way that a later evaluation delivers results that are as meaningful and, above all, comparable as possible. Here are three examples:
- Card Sorting (open/closed):
This asks users to group navigation points and labels in the way that makes the most sense to them. - Paper Prototyping:
The design team creates manual drawings of the digital product/prototype in advance of the testing. This low-fidelity approach is particularly helpful when the product is still in its early stages and when it is important to find out which concept is best suited for further development. - Screen-Recording:
To secure every detail of a user interaction, a screen recording tool (e.g. Lookback) (both in real-time and as a recording) is recommended.
Follow-up:
A decisive factor for a successful follow-up is the immediate evaluation of user feedback. The “fresh” impressions from testing can be processed very well into user quotes, for example. The key findings – the central insights of the testing – should result in actionable outcomes, which find their way into the development of new product features.
Important: be fast with processing the results of the testing. “Pick up” all stakeholders as quickly as possible and thus bring about a general consensus and jointly made decisions as immediately as possible.
User Testing and Usability-Tests: What’s the difference?
Although both formats involve testing with users, they are intended to answer different questions.
User testing with a focus on desirability answers the fundamental question: do users need my app or product at all? The focus is on user research.
The right time for user testing with a focus on usability is then within the development process of a product. The question here is: Can the users operate the interface of my app or product? The focus is therefore more on usability and its optimisation. In our article Why is usability so important? we describe examples of such specific usability testing.
What is Remote Testing?
Testing conducted remotely, i.e. virtually, with users participating from their own computer/ device, can be practical or – regarding the pandemic situation – even a necessary alternative to an on-site process for a variety of reasons.
While the procedures are not significantly different from the process described above, a key difference is the method of delivery. Here the question arises: should the testing be moderated, should it be carried out by the user himself via tools or be completely automated?
At Boana, we clearly prefer the sessions moderated by our team, which provides us with the most meaningful results.
Ideally, the conducting team consists of several people, both remotely and on-site, each of whom is assigned for specific roles:
- … the organisation in the run-up to and during the testing,
- … the moderation of the interview
- and documentation of the results.
The remote testing participants use tools such as Google Meet, Zoom or Teams, as well as features such as waiting rooms, chat functions or recording options.
The decision on whether remote testing is feasible depends not least on the type of product or logistical challenges. Here are some aspects pro and against remote testing:
| Pro Remote Testing | Contra Remote Testing |
|---|---|
| Testing of simple processes, menu structures, visual design variants and wordings in user interfaces | Testing of user flows with many media breaks |
| Integration of the device and the user's settings increases the understanding of applicability beyond the prototype | Usability tests of hardware innovations |
| Participants are widely spread geographically, long journeys would be necessary | Testing prototypes that run analogue in sub-processes (e.g. scanning a document) |
| Low budget for testing | Testing prototypes that are subject to nondisclosure |
When is User Testing useful?
Involving future users in the process at an early stage is always a benefit for everyone involved in the process. This is because the feedback and the additional ideas that often arise from users can have an extremely positive influence on further development. The ideal time for user testing is therefore as soon as the first prototype is ready and before further development starts. This by no means excludes repetition or follow-up testing – on the contrary. With an iterative and customer-centred approach, the chances of a product's success increase significantly.
What does User Testing cost?
Let's answer this question with a thought experiment: Assuming a company were to implement the 6 steps of successful testing in-house, a team of 2 people would be fully occupied with this for about 1.5 to 2 weeks. This is a considerable use of resources – in addition, the "testing amateur" would be required to do a lot of things that the experienced professionals can do much more effectively.
Throughout this article we have emphasised how important the business values are that are delivered here. If you were to go directly from the prototype to the technical implementation without testing, you would save this expenditure, but without the user feedback, there’s a high risk that parts or even the entire product concept will be rejected by the market after implementation.
The costs for user testing therefore stand in contrast to the costs of the entire implementation, which would then have to be revised – or in a worst case scenario – completely redone.
So we answer the question correctly by putting it the other way round: it might cost a fortune not to do user testing.