Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study

Ellen Astrid Holm, Shaymaa Jaafar Lafta Al-Bayati, Toke Seierøe Barfod, Maurice A Lembeck, Hanne Pedersen, Emilie Ramberg, Åse Kathrine Klemmensen, Jette Led Sorensen

Publikation: Bidrag til tidsskriftArtikelForskningpeer review

Abstract

OBJECTIVES: To examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed.

DESIGN: Qualitative text analysis supplemented by quantitative descriptive analysis.

SETTING: Internal Medicine Departments in Zealand, Denmark.

PARTICIPANTS: 48 postgraduate trainees in internal medicine specialties, 1 clinical supervisor for each trainee and 376 feedback givers (respondents).

INTERVENTION: This study examines the use of an electronic, purely narrative MSF instrument. After the MSF process, the trainee and the supervisor answered a postquestionnaire concerning their perception of the process. The authors coded the comments in the MSF reports for valence (positive or negative), specificity, relation to behaviour and whether the comment suggested a strategy for improvement. Four of the authors independently classified the MSF reports as either 'no reasons for concern' or 'possibly some concern', thereby examining discriminating capacity. Through iterative readings, the authors furthermore tried to identify how many respondents were needed in order to get a reliable impression of a trainee.

RESULTS: Out of all comments coded for valence (n=1935), 89% were positive and 11% negative. Out of all coded comments (n=4684), 3.8% were suggesting ways to improve. 92% of trainees and supervisors preferred a narrative MSF to a numerical MSF, and 82% of the trainees discovered performance in need of development, but only 53% had made a specific plan for development. Kappa coefficients for inter-rater correlations between four authors were 0.7-1. There was a significant association (p<0.001) between the number of negative comments and the qualitative judgement by the four authors. It was not possible to define a specific number of respondents needed.

CONCLUSIONS: A purely narrative MSF contributes with educational value and experienced supervisors can discriminate between trainees' performances based on the MSF reports.

OriginalsprogEngelsk
Sider (fra-til)e047019
TidsskriftBMJ open
Vol/bind11
Udgave nummer7
DOI
StatusUdgivet - 28 jul. 2021

Bibliografisk note

© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Fingeraftryk

Udforsk hvilke forskningsemner 'Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study' indeholder.

Citationsformater