Plan and carry out fair testing of a product and gather user feedback, and use the results as evidence to evaluate and improve the design

What this dot point is asking

SEAB wants you to plan and carry out fair testing of a product and gather user feedback, then use the results as evidence to evaluate and improve the design. Testing turns "I think it works" into "here is proof it works". You should be able to design a fair, repeatable test, gather feedback from real users, and use both as evidence. This underpins honest evaluation.

The answer

Why test and gather feedback

Evaluation needs evidence, and evidence comes from testing the product and asking the people who use it. Testing shows whether the product actually does what the specification requires; user feedback shows whether it works for real people. Without this evidence, an evaluation is just the designer's opinion, which may be biased or simply wrong. Good designers test thoroughly and listen to users.

Functional testing and the fair test

Functional testing checks whether the product performs its job: does it hold the load, stay leak-proof, switch on, fold flat? The key to trustworthy testing is the fair test: conditions are controlled and kept the same each time, and only one thing is changed at a time. Testing two materials for strength is only fair if both samples are the same size and tested the same way. A fair test should also be repeatable, so the same test gives the same result, confirming it is reliable.

User testing and gathering feedback

User testing puts the product in the hands of real users from the target group and watches what happens. Methods include:

• Observation. Watch users use the product and note where they struggle or hesitate; this often reveals problems they would not think to mention.
• Questioning. Ask users simple, neutral questions about what was easy or hard and why.
• Timing and counting. Measure how long a task takes or how often it goes wrong, giving numbers to compare.

Using several real users, and both observation and questions, gives reliable feedback. Real users matter because they may struggle with something the designer finds easy, exposing genuine usability issues.

Using results as evidence

The point of testing and feedback is to feed the evaluation and the next iteration. Results are recorded (measurements, observations, user comments) and used to judge each specification point and to identify improvements. "Three of five children could not open the catch in under 10 seconds" is strong evidence that the catch needs redesigning. This links testing to the iterative loop: test, learn, improve, re-test.

Examples in context

Example 1. Drop-testing a phone case. To test protection, a designer drops a weighted dummy phone in the case from a set height onto the same surface several times, keeping the height and surface constant so the test is fair and repeatable. The recorded results (no damage at 1 m, a crack at 1.5 m) are objective evidence of how much protection the case gives, far more reliable than claiming "it protects the phone well".

Example 2. User-testing a kitchen tool with elderly users. A designer gives a jar opener to several elderly users, watches them use it, times how long opening takes, and asks what felt difficult. Observation reveals that the grip is too small for arthritic hands, a problem the younger designer never felt. This genuine, evidenced usability issue leads to a larger grip, showing why real target users are essential.

Try this

• Cue. Define a fair test. Answer: a test in which conditions are controlled and kept the same each time, with only one thing changed at a time, so the result reflects what is being tested.

• Cue. Give two ways to gather user feedback on a product. Answer: any two of observing users using it, asking them neutral questions, timing tasks, and counting failures or errors.

• Cue. Explain why a result should be repeatable. Answer: repeating a test and getting the same result shows it is reliable and not a one-off fluke, so the evidence can be trusted.