Evidence Record

Evidence summary

FDA human-factors guidance requires validation sensitive enough to capture use-related problems whether or not participants know they made an error. It directs analysis toward observed errors and close calls on critical tasks, their potential harm, and their root causes.

Exact claim supported or contradicted

Supports prospective identification of critical task steps and observer-coded error/close-call outcomes rather than relying on confidence, satisfaction, or participant recall.

Source provenance

Official U.S. regulator guidance. It is nonbinding and written for medical devices, but directly documents risk-based human-factors validation practice.

Relevant excerpt or data

Sections 8 and Appendix C distinguish observational, knowledge-task, and interview data and require investigation of critical use errors even when the participant did not notice them.

Interpretation

The synthetic pilot should predefine which wrong actions make the result invalid or would matter in a real analogue. Observer coding and a post-task probe should be retained as different data sources.

Limitations

The future pilot explicitly excludes medical and other high-stakes tasks. Regulatory sample-size recommendations and “no serious use error” acceptance logic do not transfer to a small feasibility pilot.

Counterevidence

Low-risk tasks may have no genuinely harmful step. In that case, “critical” should mean task-invalidating, not rhetorically inflated safety severity.

Reproduction or verification notes

Inspect the guidance’s sections 8.1.5–8.1.7 and Appendix C.