SAFFRONstar {onlineFDR} | R Documentation |
Implements the SAFFRON algorithm for asynchronous online testing, as presented by Zrnic et al. (2018).
SAFFRONstar(pval, alpha = 0.05, version, gammai, w0, lambda = 0.5, decision.times, lags, batch.sizes, discard = FALSE, tau.discard = 0.5)
pval |
A vector of p-values |
alpha |
Overall significance level of the procedure, the default is 0.05. |
version |
Takes values 'async', 'dep' or 'batch'. This specifies the version of LORDstar to use. |
gammai |
Optional vector of γ_i. A default is provided with γ_j proportional to 1/j^(1.6). |
w0 |
Initial ‘wealth’ of the procedure, defaults to α/10. |
lambda |
Optional threshold for a ‘candidate’ hypothesis, must be between 0 and 1. Defaults to 0.5. |
decision.times |
A vector of decision times for the hypothesis tests,
this is required for |
lags |
A vector of lags or the hypothesis tests, this is required for
|
batch.sizes |
A vector of batch sizes, this is required for
|
discard |
Logical. If |
tau.discard |
Optional threshold for hypotheses to be selected for
testing. Must be between 0 and 1, defaults to 0.5. This is required if
|
The function takes as its input a vector of p-values, as well as a vector describing the conflict sets for the hypotheses. This takes the form of a vector of decision times, lags or batch sizes (see below).
Zrnic et al. (2018) present explicit three versions of SAFFRONstar:
1) version='async'
is for an asynchoronous testing process, consisting
of tests that start and finish at (potentially) random times. The discretised
finish times of the test correspond to the decision times. These decision
times are given as the input decision.times
for this version of the
SAFFRONstar algorithm. For this version of SAFFRONstar, Tian and Ramdas
(2019) presented an algorithm that can improve the power of the procedure in
the presence of conservative nulls by adaptively ‘discarding’ these p-values.
This can be called by setting the option discard=TRUE
.
2) version='dep'
is for online testing under local dependence of the
p-values. More precisely, for any t>0 we allow the p-value p_t to
have arbitrary dependence on the previous L_t p-values. The fixed
sequence L_t is referred to as ‘lags’, and is given as the input
lags
for this version of the SAFFRONstar algorithm.
3) version='batch'
is for controlling the mFDR in mini-batch testing,
where a mini-batch represents a grouping of tests run asynchronously which
result in dependent p-values. Once a mini-batch of tests is fully completed,
a new one can start, testing hypotheses independent of the previous batch.
The batch sizes are given as the input batch.sizes
for this version of
the SAFFRONstar algorithm.
Given an overall significance level α, SAFFRONstar depends on constants w_0 and λ, where w_0 satisfies 0 ≤ w_0 ≤ (1 - λ)α and represents the intial ‘wealth’ of the procedure, and 0 < λ < 1 represents the threshold for a ‘candidate’ hypothesis. A ‘candidate’ refers to p-values smaller than λ, since SAFFRONstar will never reject a p-value larger than λ. The algorithms also require a sequence of non-negative non-increasing numbers γ_i that sum to 1.
Note that these SAFFRONstar algorithms control the modified FDR (mFDR). The ‘async’ version also controls the usual FDR if the p-values are assumed to be independent.
Further details of the SAFFRONstar algorithms can be found in Zrnic et al. (2018).
d.out |
A dataframe with the original p-values |
Zrnic, T., Ramdas, A. and Jordan, M.I. (2018). Asynchronous Online Testing of Multiple Hypotheses. arXiv preprint, https://arxiv.org/abs/1812.05068.
Tian, J. and Ramdas, A. (2019). ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls. arXiv preprint, https://arxiv.org/abs/1905.11465.
SAFFRON
presents versions of SAFFRON for synchronous
p-values, i.e. where each test can only start when the previous test has
finished.
If version='async'
and discard=TRUE
, then SAFFRONstar is
identical to ADDIS
with option async=TRUE
.
pval = c(2.90e-08, 0.06743, 3.51e-04, 0.0174, 0.04723, 3.60e-05, 0.79149, 0.27201, 0.28295, 7.59e-06, 0.69274, 0.30443, 0.00136, 0.82342, 0.54757) SAFFRONstar(pval, version='async', decision.times=seq_len(15)) # Synchronous SAFFRONstar(pval, version='async', decision.times=seq_len(15)+1) # Asynchronous SAFFRONstar(pval, version='dep', lags=rep(0,15)) # Synchronous SAFFRONstar(pval, version='dep', lags=rep(1,15)) # Locally dependent SAFFRONstar(pval, version='batch', batch.size=rep(1,15)) # Synchronous SAFFRONstar(pval, version='batch', batch.size=c(4,6,5)) # Batched