The Horseshoe distribution

The horseshoe distribution (Carvalho et al., 2008) is a heavy-tailed continuous distribution defined as a scale mixture of normals. It is primarily used as a shrinkage prior in sparse Bayesian regression, where it concentrates mass near zero while retaining heavy tails that leave large signals unshrunk.

Usage

dist_horseshoe(lambda, tau)

Arguments

lambda

A positive numeric vector of local scale parameters \(\lambda > 0\) (one per observation).

tau

A positive scalar global scale parameter \(\tau > 0\).

Details

We recommend reading this documentation on pkgdown which renders math nicely. https://pkg.mitchelloharawild.com/distributional/reference/dist_horseshoe.html

In the following, let \(X\) be a horseshoe random variable with local scale parameter lambda = \(\lambda > 0\) and global scale parameter tau = \(\tau > 0\).

Support: \(x \in \mathbb{R}\), the set of all real numbers.

Mean: \(E(X)\) — not available in closed form.

Variance: \(\mathrm{Var}(X)\) — not available in closed form.

Probability density function (p.d.f):

The horseshoe density does not have a simple closed form but can be expressed as a scale mixture:

$$ X \mid \lambda, \tau \sim \mathcal{N}(0,\, \lambda^2 \tau^2) $$

where the half-Cauchy hyperprior \(\lambda \sim C^+(0, 1)\) induces the characteristic horseshoe shrinkage behaviour.

References

Carvalho, C.M., Polson, N.G., and Scott, J.G. (2008). "The Horseshoe Estimator for Sparse Signals". Discussion Paper 2008-31. Duke University Department of Statistical Science.

Carvalho, C.M., Polson, N.G., and Scott, J.G. (2009). "Handling Sparsity via the Horseshoe". Journal of Machine Learning Research, 5, p. 73–80.

See also

LaplacesDemon::dhs(), LaplacesDemon::rhs()

Examples

dist <- dist_horseshoe(lambda = c(0.5, 1, 2), tau = 1)
dist
#> <distribution[3]>
#> [1] HS(0.5, 1) HS(1, 1)   HS(2, 1)  

support(dist)
#> <support_region[3]>
#> [1] R R R
generate(dist, 10)
#> [[1]]
#>  [1]  0.07760715  0.06434405 -0.76653273  0.10118034 -0.35876933  0.18084738
#>  [7]  0.69950215  0.18634948 -0.78282215 -0.02584727
#> 
#> [[2]]
#>  [1]  0.5140821  0.5498995  0.8678169  0.6843601 -0.1626800 -1.7843647
#>  [7] -1.0371456  0.8301477  0.6073469 -0.1221864
#> 
#> [[3]]
#>  [1]  1.8662503 -1.9225534  0.5101634 -1.0908031  1.8607215 -1.0753011
#>  [7] -0.9048521 -0.8785819 -1.2324462  0.8832691
#> 

density(dist, 0)
#> [1] 0.7978846 0.3989423 0.1994711
density(dist, 0, log = TRUE)
#> [1] -0.2257914 -0.9189385 -1.6120857