Package 'fbst'

Title: The Full Bayesian Evidence Test, Full Bayesian Significance Test and the e-Value
Description: Provides access to a range of functions for computing and visualizing the Full Bayesian Significance Test (FBST) and the e-value for testing a sharp hypothesis against its alternative, and the Full Bayesian Evidence Test (FBET) and the (generalized) Bayesian evidence value for testing a composite (or interval) hypothesis against its alternative. The methods are widely applicable as long as a posterior MCMC sample is available.
Authors: Riko Kelter
Maintainer: Riko Kelter <[email protected]>
License: GPL-3
Version: 2.2
Built: 2024-11-13 05:01:44 UTC
Source: https://github.com/cran/fbst

Help Index


The Full Bayesian Evidence Test, Full Bayesian Significance Test and the e-Value

Description

Provides access to a range of functions for computing and visualizing the Full Bayesian Significance Test (FBST) and the e-value for testing a sharp hypothesis against its alternative, and the Full Bayesian Evidence Test (FBET) and the (generalized) Bayesian evidence value for testing a composite (or interval) hypothesis against its alternative. The methods are widely applicable as long as a posterior MCMC sample is available.

Details

Package for conducting the Full Bayesian Evidence Test (FBET) and the Full Bayesian Significance Test (FBST). The FBST is a Bayesian hypothesis test for testing a sharp hypothesis against its alternative by calculating the e-value, the Bayesian evidence against the null hypothesis. The FBET is a generalization of the underlying Pereira-Stern theory of the FBST and allows for testing interval hypotheses. It provides the Bayesian evidence value, or generalized e-value, which includes the e-value of the FBST as a special case. The Bayesian evidence value is based on the relative surprise function to a reference function. In the FBST, the tangential set corresponding to a sharp null hypothesis serves for quantifying the Bayesian evidence. In the FBET, the Bayesian evidence interval serves for quantifying the Bayesian evidence, which has a strong analogy to the Bayes factor. Next to the core functions, helper functions and visualizations of the results of a FBST and FBET are provided in the package.

Package: fbst
Type: Package
Title: The Full Bayesian Evidence Test, Full Bayesian Significance Test and the e-Value
Version: 2.2
Date: 2024-02-14
Author: Riko Kelter
Maintainer: Riko Kelter <[email protected]>
Description: Provides access to a range of functions for computing and visualizing the Full Bayesian Significance Test (FBST) and the e-value for testing a sharp hypothesis against its alternative, and the Full Bayesian Evidence Test (FBET) and the (generalized) Bayesian evidence value for testing a composite (or interval) hypothesis against its alternative. The methods are widely applicable as long as a posterior MCMC sample is available.
Imports: bayestestR, methods
Depends: cubature, ks, viridis, rstanarm
Suggests: BayesFactor, knitr, rmarkdown
License: GPL-3
VignetteBuilder: knitr
NeedsCompilation: no
Packaged: 2024-02-14 14:01:39 UTC; riko
Date/Publication: 2024-02-15 08:00:02 UTC
Config/pak/sysreqs: cmake libglpk-dev make libicu-dev libxml2-dev zlib1g-dev
Repository: https://rikoke.r-universe.dev
RemoteUrl: https://github.com/cran/fbst
RemoteRef: HEAD
RemoteSha: dcad708279af325581f5024f445e917d1629e42f

Index of help topics:

$,fbet-method           Returns an object from an object of class
                        'fbet'.
$,fbst-method           Returns an object from an object of class
                        'fbst'.
bdm                     bdm
fbet                    fbet
fbet-class              Class '"fbet-class"'
fbst                    fbst
fbst-class              Class '"fbst-class"'
fbst-package            The Full Bayesian Evidence Test, Full Bayesian
                        Significance Test and the e-Value
names.fbet              names.fbet
names.fbst              names.fbst
plot.fbet               plot.fbet
plot.fbst               plot.fbst
show.fbet               show.fbet
show.fbst               show.fbst
summary.fbet            summary.fbet
summary.fbst            summary.fbst

Further information is available in the following vignettes:

fbet Bayesian hypothesis testing with the Full Bayesian Evidence Test (source, pdf)
fbst Precise Bayesian hypothesis testing with the Full Bayesian Significance Test (source, pdf)
twodimfbst Precise Bayesian hypothesis testing with the Full Bayesian Significance Test in multidimensional posteriors (source, pdf)

Author(s)

Riko Kelter

Maintainer: Riko Kelter <[email protected]>

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.


Returns an object from an object of class fbet.

Description

Returns an object from an object of class fbet

Details

-

Value

-

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)

# Return the Bayesian evidence value for the interval null hypothesis
res$eValueH0

Returns an object from an object of class fbst.

Description

Returns an object from an object of class fbst

Details

-

Value

-

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 3, dimensionNullset = 2)

# Return the e-value from an fbst object
res$eValue

bdm

Description

Calculates the Bayesian discrepancy measure for a precise null hypothesis.

Usage

bdm(posteriorDensityDraws, nullHypothesisValue=0)

Arguments

posteriorDensityDraws

Vector of (MCMC) posterior parameter draws.

nullHypothesisValue

Parameter value of the precise null hypothesis. Defaults to zero.

Details

The BDM is calculated as δH(x):=2P(θIH(x)x)\delta_H(x):=2\cdot P(\theta \in I_H(x)|x) where IH(x):=(m,θ0)I_H(x):=(m,\theta_0) if m<θ0m<\theta_0, IH(x):={m}I_H(x):=\{m\} if m=θ0m=\theta_0 and IH(x):=(θ0,m)I_H(x):=(\theta_0,m) if m>θ0m>\theta_0, where mm denotes the posterior median of the parameter θ\theta, and the null hypothesis specifies H0:θ=θ0H_0:\theta=\theta_0.

Value

Returns the value δH(x)\delta_H(x) of the BDM.

Author(s)

Riko Kelter

References

For details, see: https://arxiv.org/abs/2105.13716

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

bdm(p,0)

fbet

Description

The function computes the Full Bayesian Evidence Test (FBST) and the Bayesian evidence value (the generalized e-value which obtains the e-value of the FBST as a special case), which is the Bayesian evidence against an interval null hypothesis. The function assumes posterior MCMC draws and constructs a posterior density based on a kernel density estimator subsequently. The Bayesian evidence interval is computed using a linear search based on the evidence-threshold and the calculation of the Bayesian evidence value is performed using numerical integration.

Usage

fbet(posteriorDensityDraws=NULL, interval, nu=1, FUN=NULL, 
par=NULL, posterior=NULL, par_posterior=NULL)

Arguments

posteriorDensityDraws

Vector of MCMC posterior parameter draws

interval

Vector of two numerical values containing the boundaries of the interval null hypothesis to be tested

nu

Numerical value which provides the evidence-threshold based on which the Bayesian evidence interval is calculated

FUN

Reference function

par

Additional parameters of the reference function

posterior

Posterior density function

par_posterior

Additional parameters of the posterior density function

Details

If no reference function is specified, a flat reference function r(θ)=1r(\theta)=1 is used as default reference function when posteriorDensityDraws are provided.

Value

Returns an object of class fbet if posteriorDensityDraws are provided. When using the posterior argument to pass the posterior as a function, it provides the evidence value for the hypothesis specified in the interval argument.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.3,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function, nu = 0
res = fbet(p, interval = c(-0.1,0.1), nu=0, FUN=NULL, par=NULL)
summary(res)
plot(res)

# flat reference function, nu = 1
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
summary(res)
plot(res)

# medium Cauchy C(0,1) reference function, nu = 1
res_med = fbet(posteriorDensityDraws = p, interval = c(-0.1,0.1), nu = 1, 
FUN = dcauchy, par = list(location = 0, scale=sqrt(2)))
summary(res_med)
plot(res_med)

# posterior as function argument
fbet(posterior=dbeta, par_posterior = list(shape1 = 3, shape2 = 4), 
interval = c(0.2,1), nu = 1, FUN=dbeta, par = list(shape1 = 1, shape2 = 1))

Class "fbet-class"

Description

Class for modelling the results of a Full Bayesian Evidence Test

Objects from the Class

Store the results of a FBET

Slots

data:

Object of class "list" holding the results of the Full Bayesian Evidence Test. posteriorDensityDraws holds the posterior MCMC parameter draws, posteriorDensityDrawsSorted stores the sorted posterior MCMC parameter draws, postDensValues stores the posterior density values, indices stores the indices for deciding which values pass the evidence-threshold ν\nu, interval stores the boundaries of the interval null hypothesis, referenceFunction stores the name of the reference function used, nu specifies the evidence-threshold used for computation of the Bayesian evidence interval, evidenceInterval holds the endpoints of the resulting Bayesian evidence interval, eValueH0 holds the Bayesian evidence value in favour of the interval null hypothesis, eValueH1 holds the Bayesian evidence value in favour of the alternative hypothesis (or equivalently, against the interval null hypothesis)


fbst

Description

The function computes the Full Bayesian Significance Test (FBST) and the e-value, which is the Bayesian evidence against a precise null hypothesis. The function assumes posterior MCMC draws and constructs a posterior density based on a kernel density estimator subsequently.

Usage

fbst(posteriorDensityDraws, nullHypothesisValue, FUN, par, 
dimensionTheta, dimensionNullset, dim, gridSize)

Arguments

posteriorDensityDraws

Vector of (MCMC) posterior parameter draws.

nullHypothesisValue

Parameter value of the precise null hypothesis.

FUN

Reference function.

par

Additional parameters of the reference function.

dimensionTheta

Dimension of the parameter space, defaults to 1 and can be changed to 2. Dimensions larger than 2 are currently not supported.

dimensionNullset

Dimension of the null set corresponding to the null hypothesis.

dim

Dimension of the posterior subspace over which integration is required. Defaults to 1. Can be changed to 2 if required.

gridSize

Grid size for the multivariate two-dimensional kernel density estimation in case dimensionTheta=2. Defaults to 1000.

Details

If no reference function is specified, a flat reference function r(θ)=1r(\theta)=1 is used as default reference function. Note that the posterior dimension dim defaults to 1, and if dim=2, only flat reference functions are supported. Thus, specifying FUN or par has no effect when dim=2.

Value

Returns an object of class fbst.

Author(s)

Riko Kelter

References

For a details, see: https://link.springer.com/article/10.3758/s13428-021-01613-6.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
summary(res)
plot(res)

# medium Cauchy C(0,1) reference function
res_med = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, dimensionTheta = 2, 
dimensionNullset = 1, FUN = dcauchy, par = list(location = 0, scale = sqrt(2)/2))
summary(res_med)
plot(res_med)

Class "fbst-class"

Description

Class for modelling the results of a Full Bayesian Significance Test

Objects from the Class

Store the results of a FBST

Slots

data:

Object of class "list" holding the results of the Full Bayesian Significance Test. posteriorDensityDraws holds the posterior MCMC parameter draws, postEffSizeSorted stores the sorted posterior MCMC parameter draws, densZero stores the surprise function value at the sharp null hypothesis parameter value, postDensValues stores the posterior density values, indices stores the indices for deciding which values are located inside the tangential set, nullHypothesisValue stores the sharp null hypothesis parameter value, referenceFunction holds the name of the reference function used, dimensionTheta holds the dimension of the parameter space, dimensionNullset holds the dimension of the null set corresponding to the null hypothesis, eValue holds the Bayesian evidence against the sharp null hypothesis, the e-value, pValue holds the p-value associated with the Bayesian e-value in favour of the sharp null hypothesis, sev_H_0 holds the standardized e-value as a replacement of the frequentist p-value.


names.fbet

Description

Plots the names of the objects stored in the data object of a Full Bayesian Evidence Test.

Usage

## S3 method for class 'fbet'
names(x)

Arguments

x

An Object of class "fbet".

Details

Plots the names of the objects stored in the data object of a Full Bayesian Evidence Test.

Value

Returns a list of names.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
names(res)

names.fbst

Description

Plots the names of the objects stored in the data object of a Full Bayesian Significance Test.

Usage

## S3 method for class 'fbst'
names(x)

Arguments

x

An Object of class "fbst".

Details

Plots the names of the objects stored in the data object of a Full Bayesian Significance Test.

Value

Returns a list of names.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
names(res)

plot.fbet

Description

Plots the results of a Full Bayesian Evidence Test.

Usage

## S3 method for class 'fbet'
plot(x, ..., leftBoundary = -100, rightBoundary = 100, type = "posterior", 
legendposition = "topleft", main = "")

Arguments

x

An Object of class "fbet".

...

Additional parameters, see "plot(x, ...)".

leftBoundary

x-coordinate for the left boundary to which is used for visualising the results. Defaults to -100.

rightBoundary

x-coordinate for the right boundary to which is used for visualising the results. Defaults to 100.

type

Defaults to "posterior", which produces a posterior-density based plot. Can be changed to "surprise" to show the surprise function instead.

legendposition

Position of the legend. Defaults to "topleft". Must be one of the standard string values available for the legend function of base R.

main

Title string for the plot. Default to no title.

Details

Plots the resulting surprise function, the interval null hypothesis (dotted blue lines), the resulting Bayesian evidence interval (solid blue lines), the evidence-threshold ν\nu (dotted black line) and the resulting Bayesian evidence values. The Bayesian evidence value in favour of the interval null hypothesis is visualized as the blue area, and the Bayesian evidence value in favour of the alternative hypothesis is visualized as the red area.

Value

Returns a plot.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.3,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
summary(res)
plot(res)

plot.fbst

Description

Plots the results of a Full Bayesian Significance Test.

Usage

## S3 method for class 'fbst'
plot(x, ..., leftBoundary = -100, rightBoundary = 100, type = "contour", parNames = NULL, 
xlimleft = NULL, xlimright = NULL, xlabString = "Parameter", ylabString = NULL)

Arguments

x

An Object of class "fbst".

...

Additional parameters, see "plot(x, ...)".

leftBoundary

x-coordinate for the left boundary to which is used for visualising the evidence in support of the null hypothesis. Defaults to -100.

rightBoundary

x-coordinate for the right boundary to which is used for visualising the evidence in support of the null hypothesis. Defaults to 100.

type

Relevant only if dim=2. Defaults to "contour" which provides a contour plot of the posterior, with a magenta point that shows the supremum over the null set. Alternatively, "persp" provides a 3-dimensional perspective plot of the posterior.

parNames

Vector of two entries which specifies the names for the parameters. Only relevant if dimensionTheta=2.

xlimleft

The left value for the x-axis range for the plot. Defaults to the minimum value provided in the posterior draws stored in the FBST object.

xlimright

The right value for the x-axis range for the plot. Defaults to the maximum value provided in the posterior draws stored in the FBST object.

xlabString

String for the x-axis label. Defaults to "Parameter".

ylabString

String for the y-axis label. Default to "density".

Details

Plots the surprise function, the supremum of the surprise function restricted to the null set (blue point) and visualises the Bayesian e-value against the sharp null hypothesis as the blue shaded area under the surprise function. The Bayesian e-value in favour of the sharp null hypothesis is visualised as the red shaded area under the surprise function.

Value

Returns a plot.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
plot(res)
plot(res, xlimleft = -1.5, xlimright = 0.5)

show.fbet

Description

Prints the main results of a Full Bayesian Evidence Test to the console.

Usage

## S3 method for class 'fbet'
show(object)

Arguments

object

An Object of class "fbet".

Details

Shows the main results of a Full Bayesian Evidence Test stored in an object of class fbet.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
show(res)

show.fbst

Description

Prints the main results of a Full Bayesian Significance Test to the console.

Usage

## S3 method for class 'fbst'
show(object)

Arguments

object

An Object of class "fbst".

Details

Shows the main results of a Full Bayesian Significance Test stored in an object of class fbst.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
show(res)

summary.fbet

Description

Prints the results of a Full Bayesian Evidence Test.

Usage

## S3 method for class 'fbet'
summary(object, ...)

Arguments

object

An Object of class "fbet".

...

Additional parameters, see "summary(object, ...)".

Details

Summarises the results of a Full Bayesian Evidence Test.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.3,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
summary(res)

summary.fbst

Description

Prints the results of a Full Bayesian Significance Test.

Usage

## S3 method for class 'fbst'
summary(object, ...)

Arguments

object

An Object of class "fbst".

...

Additional parameters, see "summary(object, ...)".

Details

Summarises the results of a Full Bayesian Significance Test.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
summary(res)