Bounded memory linear regression

biglm creates a linear model object that uses only p^2 memory for p variables. It can be updated with more data using update. This allows linear regression on data sets larger than memory.

biglm(formula, data, weights=NULL, sandwich=FALSE)
# S3 method for class 'biglm'
update(object, moredata,...)
# S3 method for class 'biglm'
vcov(object,...)
# S3 method for class 'biglm'
coef(object,...)
# S3 method for class 'biglm'
summary(object,...)
# S3 method for class 'biglm'
AIC(object,...,k=2)
# S3 method for class 'biglm'
deviance(object,...)

Arguments

formula

A model formula

weights

A one-sided, single term formula specifying weights

sandwich

TRUE to compute the Huber/White sandwich covariance matrix (uses p^4 memory rather than p^2)

object

A biglm object

data

Data frame that must contain all variables in formula and weights

moredata

Additional data to add to the model

...

Additional arguments for future expansion

k

penalty per parameter for AIC

Details

The model formula must not contain any data-dependent terms, as these will not be consistent when updated. Factors are permitted, but the levels of the factor must be the same across all data chunks (empty factor levels are ok). Offsets are allowed (since version 0.8).

Value

An object of class biglm

References

Algorithm AS274 Applied Statistics (1992) Vol.41, No. 2

See also

lm

Examples

data(trees)
ff<-log(Volume)~log(Girth)+log(Height)

chunk1<-trees[1:10,]
chunk2<-trees[11:20,]
chunk3<-trees[21:31,]

a <- biglm(ff,chunk1)
a <- update(a,chunk2)
a <- update(a,chunk3)

summary(a)
#> Large data regression model: biglm(ff, chunk1)
#> Sample size =  31 
#>                Coef    (95%     CI)     SE p
#> (Intercept) -6.6316 -8.2312 -5.0320 0.7998 0
#> log(Girth)   1.9826  1.8326  2.1327 0.0750 0
#> log(Height)  1.1171  0.7082  1.5260 0.2044 0
deviance(a)
#> [1] 0.1854634
AIC(a)
#> [1] 48.18546