BiostatKit / Diagnostic tests / ROC curve and AUC
ROC Curve and AUC Calculator
Paste the test value and the true condition (1 or 0) of each person and get the ROC curve, the area under it (AUC) with its 95% confidence interval, the best cut-off by the Youden index with the sensitivity, specificity and likelihood ratios there, a sentence you can paste into your manuscript, and the R code that reproduces the numbers.
Your data never leaves your browser. No sign-up, no ads.Result
How well a numeric test separates people with and without the condition.
Sentence for your manuscript
Written in AMA style from your numbers. Check it against your own study design before you submit.
Reporting checklist
- STARD 2015 (diagnostic accuracy studies): define the cut-off and say whether it was set before the study or chosen from these data (item 12a), state how accuracy was estimated (item 14), and report estimates with their precision, such as 95% CIs (item 24).
- Report the AUC with its 95% CI and name the method used for the interval.
- Say how many people had and did not have the condition. A cut-off chosen on the same data is optimistic, so check it in another sample when you can.
Methods wording. The ROC curve and the area under it were calculated with BiostatKit [1]. The 95% CI of the AUC used the method of DeLong et al [2], and the optimal cut-off was the value with the highest Youden index.
[1] BiostatKit. ROC curve and AUC calculator. Accessed [date]. [URL]
[2] DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845.
Reproduce it in R
The script is filled in with your numbers. Paste it into R or RStudio and compare.
This calculator was checked against Python (SciPy Mann-Whitney U and scikit-learn roc_curve; the DeLong variance was computed by two different algorithms and the coverage of the CI checked by simulation) on 2 reference cases and matched to 4 significant digits. See the verification log.
How to read an ROC curve and the AUC
What it is
A ROC curve shows, for every possible cut-off of a test that gives a number, the sensitivity against 1 - specificity. The area under it (AUC) is the chance that a person picked at random from those with the condition has a higher test value than a person picked at random from those without it, with ties counted as half. 0.5 is no better than a coin toss, 1 is perfect separation.
How this page computes it
Every observed value is tried as a cut-off. The AUC is the Mann-Whitney statistic divided by n1 x n0, which equals the area under the empirical curve joined by straight lines (Hanley and McNeil, 1982).
AUC = mean over all pairs of [1 if X > Y, 1/2 if X = Y, 0 if X < Y]
The standard error and the 95% CI follow DeLong et al (1988): each person gets a placement value, the share of the other group they beat. Var(AUC) = S10 / n1 + S01 / n0, where S10 and S01 are the variances of the placement values in the two groups, and the CI is AUC ± 1.96 SE, cut to between 0 and 1. The Hanley-McNeil standard error, an older approximation, is shown for comparison.
The best cut-off has the highest Youden index (Youden, 1950), J = sensitivity + specificity - 1. If several cut-offs tie, the strictest one is shown. Sensitivity and specificity there come with Wilson score CIs, and the likelihood ratios with log-method CIs.
Assumptions
- Each row is a different person with one test value and a label from the reference standard: 1 for the condition present, 0 for absent.
- The reference standard was applied to everyone, and the people resemble those in whom the test will be used. Only clear cases and clear non-cases inflate the AUC.
- The test values can be put in order: a measurement, or a rating with several levels.
Common mistakes
- Judging the test by the AUC alone. It sums up all cut-offs, but in practice you use only one.
- Choosing the cut-off and reporting its sensitivity and specificity on the same data without saying so. They are optimistic.
- Setting the wrong direction. An AUC below 0.5 usually means the scale runs the other way.
- Quoting the AUC without its confidence interval, or comparing two AUCs by eye.
- Using a yes/no test. With two values the curve has a single corner; report sensitivity and specificity instead.
Frequently asked questions
What is a good AUC?
There is no fixed threshold. 0.5 means no better than a coin toss and 1 means perfect separation. Look at the 95% confidence interval and at what the test is for: an AUC of 0.75 can be useful for screening and too low to confirm a diagnosis.
Which cut-off should I choose?
The Youden cut-off treats a missed case and a false alarm as equally bad. If missing the condition costs more, pick a cut-off with higher sensitivity on clinical grounds and say so. Report whether the cut-off was set in advance (STARD item 12a).
How are tied values handled?
A tie between a person with and a person without the condition counts as half a win, as in the Mann-Whitney statistic. A rating scale with a few levels gives a curve with only a few points.
Can I compare two tests here?
No. Comparing two AUCs measured on the same people needs the paired DeLong test, which this page does not run.
Is my data stored or sent anywhere?
No. All calculations run in your browser tab. Nothing is uploaded, and the page works offline once loaded.
Related calculators: Sensitivity and specificity calculator · Cohen's kappa