Product thinking / DIJION
What decisions can a small usability study support?
A guide to reporting task observations, blockers and sample limits without turning a small study into a prevalence claim.
A small team’s study with a few people can reveal an important navigation barrier. It does not establish that every user will encounter it. In an illustrative registration study, examining the blocked step may be more useful than presenting one percentage as a success score. Preserve the finding’s value without overstating its reach.
Define what the study can answer
Observation can examine understanding and recovery from errors. Population prevalence or a small difference between versions may require a different study design. GOV.UK usability guidance discusses realistic tasks and appropriate participants. Include intended use context in recruitment and reporting, rather than treating every convenient participant as interchangeable.
Report conditions alongside counts
Scroll the table horizontally to see all columns.
| Record | What it shows | What it does not establish |
|---|---|---|
| Observed task blocker | Problem at a particular step | Population frequency |
| Completed with assistance | Progress after help | Independent completion |
| Positive comment | A stated opinion | Future use |
Do not rank severity by count alone
One instance of data loss may matter more than several minor visual confusions. Assess consequence, recoverability and task criticality. Explicitly record when moderator guidance changes the outcome. A technical failure and a misunderstood task instruction have different causes; combining them into one finding can obscure what needs to be changed.
Connect uncertainty to the next step
A proposed fix may be examined through another prototype round. If affected contexts are unclear, recruit from a different relevant group. If prevalence matters to the decision, design an appropriate measurement plan. Reusing the same participants can introduce learned behavior; distinguish new-participant sessions from return visits instead of blending their outcomes.
Complete the finding card
- Record task, participant context and starting state.
- Separate observed behavior from moderator assistance.
- Describe consequence, recovery and reproducibility.
- Include the inference boundary and the next investigation.
The outcome is richer than a success percentage: problem, context, impact and an open question. This makes a small study actionable without exaggerating what it establishes.