L1 Linear Regression

Solve the linear system A x = b in an Lp sense, that is minimize the term sum |b - A x|^p. The case p=1 is also called “least absolute deviation” (LAD) regression.

L1linreg(A, b, p = 1, tol = 1e-07, maxiter = 200)

Arguments

A

matrix of independent variables.

b

independent variables.

p

the p in L^p norm, p<=1.

tol

relative tolerance.

maxiter

maximum number of iterations.

Details

L1/Lp regression is here solved applying the “iteratively reweighted least square” (IRLS) method in which each step involves a weighted least squares problem.

If an intercept term is required, add a unit column to A.

Value

Returns a list with components x the linear coefficients describing the solution, reltol the relative tolerance reached, and niter the number of iterations.

References

Dasgupta, M., and S.K. Mishra (2004). Least absolute deviation estimation of linear econometric models: A literature review. MPRA Paper No. 1781.

Note

In this case of p=1, the problem would be better approached by use of linear programming methods.

See also

lm, lsqnonlin, quantreg::rq

Examples

m <- 101; n <- 10       # no. of data points, degree of polynomial
x <- seq(-1, 1, len=m)
y <- runge(x)           # Runge's function
A <- outer(x, n:0, '^') # Vandermonde matrix
b <- y

( sol <- L1linreg(A, b) )
#> $x
#>  [1] -2.193242e+01 -3.463779e-13  6.291092e+01  6.559040e-13 -6.784854e+01
#>  [6] -3.931573e-13  3.414400e+01  8.367666e-14 -8.118989e+00 -4.699117e-15
#> [11]  8.453273e-01
#> 
#> $reltol
#> [1] 6.710152e-10
#> 
#> $niter
#> [1] 81
#> 
# $x
# [1] -21.93242   0.00000  62.91092   0.00000 -67.84854   0.00000
# [7]  34.14400   0.00000  -8.11899   0.00000   0.84533
# 
# $reltol
# [1] 6.712355e-10
# 
# $niter
# [1] 81

# minimum value of polynomial L1 regression
sum(abs(polyval(sol$x, x) - y))
#> [1] 3.061811
# [1] 3.061811