perf_wilcox {diffuStats} | R Documentation |
Function perf_wilcox
compares all the columns of a matrix
through a wilcox.test
.
The columns are assumed to be performance measures (e.g. AUROC)
whereas the rows are instances.
perf_wilcox(perf_mat, adjust = function(p) stats::p.adjust(p, method = "fdr"), ci = 0.95, digits_ci = 2, digits_p = 3, ...)
perf_mat |
Numeric matrix whose columns contain performance metrics of different methods. |
adjust |
Function to adjust the p-values for multiple testing.
By default, |
ci |
Numeric, confidence interval (defaults to |
digits_ci |
Integer, digits to display in the confidence interval |
digits_p |
Integer, digits to display in the p-value |
... |
further arguments for |
The statistical comparison of the columns is intended to
ease comparisons between methods in a rigorous way.
Methods are compared pairwise and a p-value for difference in
performance.
The function perf_wilcox
returns a character matrix so that
(1) the upper triangular matrix contains confidence intervals
on the estimate of the difference between performances, and
(2) the lower triangular matrix contains the two-tailed p-value
that tests difference in performance, with multiple testing correction.
The comparison takes place between row and column in that precise order:
a positive difference favours the row and a negative one, the column.
Character matrix. The upper triangular matrix contains a confidence interval and the estimate of the pairwise difference in performance. The lower triangular matrix shows the associated two-tailed p-value, with multiple testing correction.
# Dummy data frame to test n <- 100 perf_mat <- cbind( good = runif(n = n, min = 0.5, max = 1), so_so = runif(n = n, min = 0.2, max = 0.7), bad = runif(n = n, min = 0, max = 0.5) ) wilcox_mat <- perf_wilcox(perf_mat) # See how the methods in the rows compare to those # in the columns, confidence interval # (upper) and p-value (lower) wilcox_mat