Inferior reliability of VAS scoring compared with International Society of the Knee reporting system for abstract assessment

Ole Rahbek, Steen L Jensen, Martin Lind, Jeannette Ø Penny, Thomas Kallemose, Thomas Jakobsen, Anders Troelsen

Publikation: Bidrag til tidsskriftArtikelForskningpeer review


INTRODUCTION: Knowledge of how abstracts may be se-lected for medical conferences in an efficient and reliable manner is sparse. To improve abstract selection, the Danish Orthopaedic Society implemented the International Society of the Knee (ISK) quality-of-reporting system and visual analogue scale (VAS) scoring for abstract evaluation at its 2014 Annual Congress. We sought to find out if a simple VAS score was more reliable than a multiple-question system for assessment of over-all abstract quality.

METHODS: A total of 214 abstracts were submitted for review. All abstracts were reviewed by 3 reviewers using a VAS score and the ISK score. Of the 214, 71 abstracts were reviewed again 6 months later to estimate intra-rater agreement.

RESULTS: The VAS and the ISK score were poorly correlated (r = 0.64), and the ISK score demonstrated a better intra- and interrater agreement (p < 0.001). The VAS scores of all abstracts were more widely distributed than the ISK scores, which clustered around values in the 50-70 range. Chronbach's alpha for the ISK score was 0.66 (95% confidence interval: 0.62-0.68).

CONCLUSIONS: The VAS score has a poorer intra- and interrater agreement than the ISK score, and the two scores do not correlate well. VAS scores were more widely distributed, which is beneficial when selecting a scientific programme, but the score is unreliable. We continue to use the ISK score, although its reliability may still be improved.

FUNDING: none.


TidsskriftDanish medical journal
Udgave nummer4
StatusUdgivet - apr. 2017
Udgivet eksterntJa

Bibliografisk note

Articles published in the Danish Medical Journal are “open access”. This means that the articles are distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits any non-commercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.


Udforsk hvilke forskningsemner 'Inferior reliability of VAS scoring compared with International Society of the Knee reporting system for abstract assessment' indeholder.