A summative usability study is the final validation that your medical device can be used safely and effectively by its intended users. It's the study FDA reviewers will scrutinize in your 510(k) or PMA submission. Getting it right requires careful planning across user group definition, participant recruitment, facility selection, study protocol design, and documentation. This guide walks through each step.
A summative study (also called a human factors validation study) is a formal evaluation conducted under realistic conditions with representative users. Unlike formative studies, which are iterative and exploratory, the summative study is designed to demonstrate that residual use-related risks are acceptable.
FDA's guidance document "Applying Human Factors and Usability Engineering to Medical Devices" outlines what reviewers expect to see. IEC 62366-1 provides the underlying process framework.
Identify every distinct user population that will interact with your device. This typically includes primary operators (e.g., nurses, physicians, surgical technologists), setup and maintenance personnel, patients or caregivers (for home-use devices), and reprocessing staff.
Each user group with meaningfully different interaction patterns or risk profiles needs to be tested separately. FDA generally expects a minimum of 15 participants per user group for summative evaluations.
Work from your use-related risk analysis (URRA) to identify hazard-related use scenarios: tasks where use error could lead to patient harm. These are the tasks your summative study must evaluate.
Prioritize based on severity and likelihood. Every critical task needs a corresponding test scenario with clear success and failure criteria.
Your screener defines the inclusion and exclusion criteria for each user group. For clinicians, this typically includes specialty, years of experience, clinical setting, and familiarity with similar devices. For patients, it includes diagnosis, treatment history, age, and health literacy level.
Recruitment should source participants who are genuinely representative of your intended users. Not company employees. Not professional research participants who cycle through studies for income.
If your study requires FDA self-selection blinding (where participants cannot know the study sponsor or device identity during recruitment), your recruitment process must be designed accordingly.
Allow 4-6 weeks for clinical recruitment. Tight screeners and specialty populations require more lead time.
Your study environment should simulate the intended use environment as closely as possible. For clinical devices, this means a controlled space with appropriate equipment, lighting, and workflow simulation. For home-use devices, this may mean a simulated home environment.
Essential facility requirements include one-way observation capability, HD video and audio recording for every session, a separate observation room for your team and client observers, configurable room layout, and on-site staff to manage participant logistics.
If your study includes multiple user groups, a facility with two labs and a shared observation suite allows you to run parallel sessions and compress your study timeline.
The protocol defines everything: training procedures, task scenarios, moderator script, data collection instruments, success/failure criteria for each task, and post-task interview questions.
Two areas deserve extra attention. First, if your device includes any training or onboarding, decide whether to include a learning decay period between training and testing. FDA wants to know that users can retain what they learned, not just perform immediately after instruction. A 24-hour or longer gap between training and evaluation sessions is common for devices with meaningful learning curves.
Second, write your success and failure criteria before the study, not after. Vague criteria like "participant completes the task" invite post-hoc interpretation. Define what counts as a use error, a close call, and a successful completion for each critical task. When a participant takes 4 attempts to insert a cartridge but eventually succeeds, is that a pass or a fail? That question should be answered in the protocol, not in the analysis meeting.
The protocol should be detailed enough that any trained moderator can run the study consistently across all sessions. Include provisions for root cause analysis when use errors are observed. Understanding why an error occurred is as important as documenting that it happened, and FDA reviewers expect to see that analysis in your validation report.
On study day, the facility should be fully configured before the first participant arrives. Participant check-in, informed consent, and any pre-session questionnaires should be handled by facility staff so your research team can focus on the study itself.
Record every session. Take detailed notes on task performance, use errors, close calls, and participant comments. Conduct post-task and post-session interviews to understand the participant's mental model and decision-making process.
Your Human Factors Validation Report should follow FDA's recommended structure: device description, user profiles, use environments, risk analysis summary, formative study summaries, validation study protocol, results by task and user group, use error analysis, and conclusions.
The narrative must clearly connect your risk analysis to your study design and demonstrate that residual use-related risks are acceptable. Include participant demographics, individual task performance data, and verbatim quotes where they illuminate use error root causes.
If you're preparing for a summative validation, the earlier you lock down your facility and recruitment plan, the fewer surprises you'll face on study day. We've supported enough submissions to know what works. Tell us where you are in the process and we'll figure out the rest.
Tell us your timeline and user groups. We'll tell you what's realistic.
Get in Touch