Validation of a Novel Assessment Tool Identifying Proficiency in Transurethral Bladder Tumor Resection: The OSATURBS assessment tool

Background: Competence in transurethral resection of bladder tumors (TURB) is critical in bladder cancer management and should be ensured before independent practice. Objective: To develop an assessment tool for TURB and explore validity evidence in a clinical context. Design, Setting, and Participants: From July 2019 to March 2021, a total of 33 volunteer doctors from three hospitals were included after exemption from the regional ethics committee (REG-008-2018). Participants performed two TURB procedures on patients with bladder tumors. A newly developed assessment tool (Objective Structured Assessment for Transurethral Resection of Bladder Tumors Skills, OSATURBS) was used for direct observation assessment (DOA), self-assessment (SA), and blinded video assessment (VA). Outcome Measurements and Statistical Analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t-test and independent samples t-test. Results and Limitations: The internal consistency reliability across items Cronbach's alpha was 0.94 ( n  = 260, p  < 0.001). Inter-rater reliability was 0.80 ( n  = 64, p  < 0.001). Test-retest correlation was high, r  = 0.71 ( n  = 32, p  < 0.001). Relationship with TURB experience was high, r  = 0.71 ( n  = 32, p  < 0.001). Pass/fail score was 19 points. DOAs were strongly correlated with video ratings ( r  = 0.85, p  < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances. Conclusions: OSATURBS tool for TURB can be used for assessment of surgical proficiency in the clinical setting. DOA and SA are biased, and blinded VA of TURB performances is advised. Clinical Trials NCT03864302.