Reliable and valid assessment of clinical bronchoscopy performance

Background: There have been several attempts to systematically assess performance in bronchoscopy. Earlier validation studies have used bronchoscopy simulators, not real-life performance in patients. Objectives: The aim of this study was to explore the reliability and validity of an assessment tool aimed for the use in a clinical setting. Methods: Five junior residents, 5 senior residents and 9 consultants performed 3 bronchoscopies each. All 57 bronchoscopies were video-recorded and assessed blindly and independently by two bronchoscopy experts using the new assessment tool. Results: The interrater reliability was high, with Cronbach's α = 0.86. Assessment of 3 bronchoscopies by a single rater had a generalizability coefficient of 0.84. The correlation between experience and performance was good (Pearson correlation = 0.76). There were significant differences between the groups for all aspects of the assessment, but post hoc tests showed different discriminative abilities. Conclusions: This new tool for assessing clinical bronchoscopy performance has a high interrater reliability. One rater assessing performance of 3 bronchoscopies ensures sufficient reliability. The assessment tool demonstrated sufficient construct validity.