TY - JOUR
T1 - The thresholds for statistical and clinical significance - A five-step procedure for evaluation of intervention effects in randomised clinical trials
AU - Jakobsen, Janus Christian
AU - Gluud, Christian
AU - Winkel, Per
AU - Lange, Theis
AU - Wetterslev, Jørn
PY - 2014/3/4
Y1 - 2014/3/4
N2 - Background: Thresholds for statistical significance are insufficiently demonstrated by 95% confidence intervals or P-values when assessing results from randomised clinical trials. First, a P-value only shows the probability of getting a result assuming that the null hypothesis is true and does not reflect the probability of getting a result assuming an alternative hypothesis to the null hypothesis is true. Second, a confidence interval or a P-value showing significance may be caused by multiplicity. Third, statistical significance does not necessarily result in clinical significance. Therefore, assessment of intervention effects in randomised clinical trials deserves more rigour in order to become more valid. Methods. Several methodologies for assessing the statistical and clinical significance of intervention effects in randomised clinical trials were considered. Balancing simplicity and comprehensiveness, a simple five-step procedure was developed. Results: For a more valid assessment of results from a randomised clinical trial we propose the following five-steps: (1) report the confidence intervals and the exact P-values; (2) report Bayes factor for the primary outcome, being the ratio of the probability that a given trial result is compatible with a 'null' effect (corresponding to the P-value) divided by the probability that the trial result is compatible with the intervention effect hypothesised in the sample size calculation; (3) adjust the confidence intervals and the statistical significance threshold if the trial is stopped early or if interim analyses have been conducted; (4) adjust the confidence intervals and the P-values for multiplicity due to number of outcome comparisons; and (5) assess clinical significance of the trial results. Conclusions: If the proposed five-step procedure is followed, this may increase the validity of assessments of intervention effects in randomised clinical trials.
AB - Background: Thresholds for statistical significance are insufficiently demonstrated by 95% confidence intervals or P-values when assessing results from randomised clinical trials. First, a P-value only shows the probability of getting a result assuming that the null hypothesis is true and does not reflect the probability of getting a result assuming an alternative hypothesis to the null hypothesis is true. Second, a confidence interval or a P-value showing significance may be caused by multiplicity. Third, statistical significance does not necessarily result in clinical significance. Therefore, assessment of intervention effects in randomised clinical trials deserves more rigour in order to become more valid. Methods. Several methodologies for assessing the statistical and clinical significance of intervention effects in randomised clinical trials were considered. Balancing simplicity and comprehensiveness, a simple five-step procedure was developed. Results: For a more valid assessment of results from a randomised clinical trial we propose the following five-steps: (1) report the confidence intervals and the exact P-values; (2) report Bayes factor for the primary outcome, being the ratio of the probability that a given trial result is compatible with a 'null' effect (corresponding to the P-value) divided by the probability that the trial result is compatible with the intervention effect hypothesised in the sample size calculation; (3) adjust the confidence intervals and the statistical significance threshold if the trial is stopped early or if interim analyses have been conducted; (4) adjust the confidence intervals and the P-values for multiplicity due to number of outcome comparisons; and (5) assess clinical significance of the trial results. Conclusions: If the proposed five-step procedure is followed, this may increase the validity of assessments of intervention effects in randomised clinical trials.
KW - Bayes factor
KW - Confidence interval
KW - P-value
KW - Randomised clinical trial
KW - Threshold for significance
UR - http://www.scopus.com/inward/record.url?scp=84899481382&partnerID=8YFLogxK
U2 - 10.1186/1471-2288-14-34
DO - 10.1186/1471-2288-14-34
M3 - Article
C2 - 24588900
AN - SCOPUS:84899481382
SN - 1471-2288
VL - 14
JO - BMC Medical Research Methodology
JF - BMC Medical Research Methodology
IS - 1
M1 - 34
ER -