ULTRA-metrics: Delphi-derived Framework for Assessing Ultrasound Competency

Abstract

OBJECTIVES: Ultrasound competency is critical in modern healthcare, yet no standardized framework currently supports ultrasound skill monitoring across diverse clinical settings and user types. Existing frameworks often lack generalizability, overemphasize exam counts, and fail to assess key skills such as interpretation, limiting ultrasound's safe and effective integration into clinical practice. The objective of this study is to develop a consensus-based, universal framework for monitoring ultrasound competency across clinical applications and disciplines.

METHODS: A modified Delphi process was conducted with an international panel of Point-of-Care ultrasound experts. Panelists independently evaluated framework elements categorized by competency domains (experience, skills, autonomy), skill domains (indication, acquisition, interpretation, clinical integration), metrics (eg, exam counts, entrustability, interpretation accuracy, etc.), answer sets (score-based inputs used by assessors), and score criteria (requirements for each score). Consensus thresholds were defined as strong consensus at >84%, and weak consensus at 68-84%. Two Delphi rounds were completed.

RESULTS: Nineteen experts participated across 2 Delphi rounds. Strong consensus was reached to include 3 competency domains (experience, skills, autonomy) and 4 skill domains (indication, acquisition, interpretation, and clinical integration). Optional components, including the use of acquisition skill trees and varied answer set granularity, were favored by some participants to allow ultrasound programs to tailor the framework to specific examinations, assessment scenarios, and job roles.

CONCLUSION: The resulting modular framework provides a flexible, consensus-based approach to ultrasound competency assessment, enabling cross-program comparisons and evaluation of training methods. Validation across diverse settings is needed to support its use in global competency standards and ultrasound education expansion.