The lmerTest package provides p-values in type I, II or III
anova and summary
tables for linear mixed models (lmer model fits cf. lme4)
via Satterthwaite's degrees of freedom method; a Kenward-Roger method is also
available via the pbkrtest package.
Model selection and assessment methods include step,
drop1, anova-like tables for random effects (ranova),
least-square means (LS-means; ls_means)
and tests of linear contrasts of fixed effects (contest).
overloads lme4::lmer and produced an object of class
lmerModLmerTest which inherits from lmerMod. In addition to
computing the model (using lme4::lmer), lmerTest::lmer
computes a couple of components needed for the evaluation of Satterthwaite's
denominator degrees of freedom.
anova method for lmer model fits produces
type I, II, and III anova tables for fixed-effect terms with
Satterthwaite and Kenward-Roger methods for denominator degrees of freedom
for F-tests.
summary method for lmer model fits adds
denominator degrees of freedom and p-values to the coefficient table.
anova-like table of random effects via likelihood ratio tests
with methods for both lmerMod and lmerModLmerTest objects.
ranova can either test reduction of random-effect terms to simpler
structures or it can test removal of entire random-effect terms.
F-tests of fixed-effect terms using Satterthwaite or
Kenward-Roger methods for denominator degrees of freedom. These 'single term
deletion' tables are useful for model selection and tests of marginal terms.
Compared to the likelihood ratio tests of lme4::drop1 the F-tests and
p-values of lmerTest::drop1 are more accurate and considerably faster
since no additional model fitting is required.
tests of contrasts, i.e. tests of linear functions of the
fixed-effect coefficients. A user-friendly interface for tests of contrasts
with outputs either as a summary-like table of t-tests or an anova-like table
of F-tests (or a list of either). Contrasts can optionally be tested for
estimability. Contrasts are allowed to be rank-deficient as the rank is
automatically detected and appropriate adjustments made. Methods for
lmerModLmerTest as well as lmerMod objects – the latter avoids
the Satterthwaite specific computations when the Kenward-Roger method is used.
a function which operates on anova tables and LS-means tables makes it possible to see exactly which functions of the coefficients are being tested. This is helpful when differences between type I, II and III anova tables are being considered and discussed.
computes the so-called least-squares means (classical Yates contrasts) as well as pairwise differences of these.
performs automatic backward model selection of fixed and random parts of the linear mixed model.
an explicit coerce function from class
lmerMod to lmerModLmerTest.
The computational approach is to let lmerTest::lmer compute the
Hessian and derivatives needed for evaluation of degrees of freedom and
t- and F-tests and to store these in the model object. The
Hessian and derivatives are therefore computed only once per model fit
and reused with each call to anova, summary, etc. Evaluation of
t and F-tests does not involve model re-fitting.
lmerTest::lmer roughly amounts to calling lme4::lmer followed by
lmerTest::as_lmerModLmerTest, so for computationally intensive model
fits it can make sense to use lme4::lmer rather than lmerTest::lmer
if computational time is an issue and summary tables and anova tables will
not be needed.
Alexandra Kuznetsova, Per B. Brockhoff and Rune H. B. Christensen (2017) lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software, 82(13), 1–26. doi:10.18637/jss.v082.i13
## load lmerTest package
library(lmerTest)
## Fit linear mixed model to the ham data:
fm <- lmer(Informed.liking ~ Gender + Information * Product + (1 | Consumer) +
(1 | Consumer:Product), data=ham)
## Summary including coefficient table with p-values for t-statistics using
## Satterthwaite's method for denominator degrees of freedom:
summary(fm)
#> Linear mixed model fit by REML. t-tests use Satterthwaite's method [
#> lmerModLmerTest]
#> Formula: Informed.liking ~ Gender + Information * Product + (1 | Consumer) +
#> (1 | Consumer:Product)
#> Data: ham
#>
#> REML criterion at convergence: 2705.5
#>
#> Scaled residuals:
#> Min 1Q Median 3Q Max
#> -3.1482 -0.4478 0.0207 0.4510 3.3626
#>
#> Random effects:
#> Groups Name Variance Std.Dev.
#> Consumer:Product (Intercept) 3.1622 1.7783
#> Consumer (Intercept) 0.3756 0.6129
#> Residual 1.6675 1.2913
#> Number of obs: 648, groups: Consumer:Product, 324; Consumer, 81
#>
#> Fixed effects:
#> Estimate Std. Error df t value Pr(>|t|)
#> (Intercept) 5.8490 0.2843 322.3364 20.574 <2e-16 ***
#> Gender2 -0.2443 0.2606 79.0000 -0.938 0.3514
#> Information2 0.1605 0.2029 320.0000 0.791 0.4296
#> Product2 -0.8272 0.3453 339.5123 -2.395 0.0171 *
#> Product3 0.1481 0.3453 339.5123 0.429 0.6682
#> Product4 0.2963 0.3453 339.5123 0.858 0.3915
#> Information2:Product2 0.2469 0.2870 320.0000 0.860 0.3902
#> Information2:Product3 0.2716 0.2870 320.0000 0.946 0.3446
#> Information2:Product4 -0.3580 0.2870 320.0000 -1.248 0.2131
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> Correlation of Fixed Effects:
#> (Intr) Gendr2 Infrm2 Prdct2 Prdct3 Prdct4 In2:P2 In2:P3
#> Gender2 -0.453
#> Informatin2 -0.357 0.000
#> Product2 -0.607 0.000 0.294
#> Product3 -0.607 0.000 0.294 0.500
#> Product4 -0.607 0.000 0.294 0.500 0.500
#> Infrmtn2:P2 0.252 0.000 -0.707 -0.415 -0.208 -0.208
#> Infrmtn2:P3 0.252 0.000 -0.707 -0.208 -0.415 -0.208 0.500
#> Infrmtn2:P4 0.252 0.000 -0.707 -0.208 -0.208 -0.415 0.500 0.500
## Type III anova table with p-values for F-tests based on Satterthwaite's
## method:
(aov <- anova(fm))
#> Type III Analysis of Variance Table with Satterthwaite's method
#> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
#> Gender 1.4656 1.4656 1 79 0.8789 0.35135
#> Information 6.5201 6.5201 1 320 3.9101 0.04885 *
#> Product 19.1551 6.3850 3 240 3.8291 0.01048 *
#> Information:Product 10.3873 3.4624 3 320 2.0765 0.10321
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
## Inspect the contrast matrix for the Type III test of Product:
show_tests(aov, fractions = TRUE)$Product
#> (Intercept) Gender2 Information2 Product2 Product3 Product4
#> Product2 0 0 0 1 0 0
#> Product3 0 0 0 0 1 0
#> Product4 0 0 0 0 0 1
#> Information2:Product2 Information2:Product3 Information2:Product4
#> Product2 1/2 0 0
#> Product3 0 1/2 0
#> Product4 0 0 1/2
## Choose type II anova table with Kenward-Roger method for the F-test:
if (FALSE) { # \dontrun{
if(requireNamespace("pbkrtest", quietly = TRUE))
anova(fm, type=2, ddf="Kenward-Roger")
} # }
## Anova-like table of random-effect terms using likelihood ratio tests:
ranova(fm)
#> ANOVA-like table for random-effects: Single term deletions
#>
#> Model:
#> Informed.liking ~ Gender + Information + Product + (1 | Consumer) + (1 | Consumer:Product) + Information:Product
#> npar logLik AIC LRT Df Pr(>Chisq)
#> <none> 12 -1352.8 2729.5
#> (1 | Consumer) 11 -1354.3 2730.7 3.186 1 0.07425 .
#> (1 | Consumer:Product) 11 -1435.0 2891.9 164.443 1 < 2e-16 ***
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
## F-tests of 'single term deletions' for all marginal terms:
drop1(fm)
#> Single term deletions using Satterthwaite's method:
#>
#> Model:
#> Informed.liking ~ Gender + Information * Product + (1 | Consumer) + (1 | Consumer:Product)
#> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
#> Gender 1.4656 1.4656 1 79 0.8789 0.3514
#> Information:Product 10.3873 3.4624 3 320 2.0765 0.1032
## Least-Square means and pairwise differences:
(lsm <- ls_means(fm))
#> Least Squares Means table:
#>
#> Estimate Std. Error df t value lower upper
#> Gender1 5.85366 0.18311 79.0 31.969 5.48920 6.21812
#> Gender2 5.60938 0.18538 79.0 30.259 5.24038 5.97837
#> Information1 5.63121 0.13981 104.2 40.278 5.35397 5.90845
#> Information2 5.83183 0.13981 104.2 41.713 5.55458 6.10907
#> Product1 5.80713 0.23232 311.5 24.996 5.35002 6.26425
#> Product2 5.10343 0.23232 311.5 21.967 4.64631 5.56055
#> Product3 6.09108 0.23232 311.5 26.218 5.63397 6.54820
#> Product4 5.92442 0.23232 311.5 25.501 5.46730 6.38154
#> Information1:Product1 5.72689 0.25351 426.6 22.591 5.22861 6.22517
#> Information2:Product1 5.88738 0.25351 426.6 23.224 5.38910 6.38566
#> Information1:Product2 4.89973 0.25351 426.6 19.328 4.40145 5.39801
#> Information2:Product2 5.30713 0.25351 426.6 20.935 4.80885 5.80541
#> Information1:Product3 5.87504 0.25351 426.6 23.175 5.37676 6.37332
#> Information2:Product3 6.30713 0.25351 426.6 24.879 5.80885 6.80541
#> Information1:Product4 6.02318 0.25351 426.6 23.759 5.52490 6.52146
#> Information2:Product4 5.82565 0.25351 426.6 22.980 5.32737 6.32393
#> Pr(>|t|)
#> Gender1 < 2.2e-16 ***
#> Gender2 < 2.2e-16 ***
#> Information1 < 2.2e-16 ***
#> Information2 < 2.2e-16 ***
#> Product1 < 2.2e-16 ***
#> Product2 < 2.2e-16 ***
#> Product3 < 2.2e-16 ***
#> Product4 < 2.2e-16 ***
#> Information1:Product1 < 2.2e-16 ***
#> Information2:Product1 < 2.2e-16 ***
#> Information1:Product2 < 2.2e-16 ***
#> Information2:Product2 < 2.2e-16 ***
#> Information1:Product3 < 2.2e-16 ***
#> Information2:Product3 < 2.2e-16 ***
#> Information1:Product4 < 2.2e-16 ***
#> Information2:Product4 < 2.2e-16 ***
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> Confidence level: 95%
#> Degrees of freedom method: Satterthwaite
ls_means(fm, which = "Product", pairwise = TRUE)
#> Least Squares Means table:
#>
#> Estimate Std. Error df t value lower upper
#> Product1 - Product2 0.703704 0.314112 240 2.2403 0.084936 1.322471
#> Product1 - Product3 -0.283951 0.314112 240 -0.9040 -0.902718 0.334817
#> Product1 - Product4 -0.117284 0.314112 240 -0.3734 -0.736051 0.501484
#> Product2 - Product3 -0.987654 0.314112 240 -3.1443 -1.606422 -0.368887
#> Product2 - Product4 -0.820988 0.314112 240 -2.6137 -1.439755 -0.202220
#> Product3 - Product4 0.166667 0.314112 240 0.5306 -0.452101 0.785434
#> Pr(>|t|)
#> Product1 - Product2 0.025987 *
#> Product1 - Product3 0.366912
#> Product1 - Product4 0.709193
#> Product2 - Product3 0.001875 **
#> Product2 - Product4 0.009523 **
#> Product3 - Product4 0.596189
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> Confidence level: 95%
#> Degrees of freedom method: Satterthwaite
## ls_means also have plot and as.data.frame methods:
if (FALSE) { # \dontrun{
plot(lsm, which=c("Product", "Information"))
as.data.frame(lsm)
## Inspect the LS-means contrasts:
show_tests(lsm, fractions=TRUE)$Product
} # }
## Contrast test (contest) using a custom contrast:
## Here we make the 2-df joint test of the main effects of Gender and Information
(L <- diag(length(fixef(fm)))[2:3, ])
#> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
#> [1,] 0 1 0 0 0 0 0 0 0
#> [2,] 0 0 1 0 0 0 0 0 0
contest(fm, L = L)
#> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
#> 1 2.508819 1.254409 2 125.9797 0.7522799 0.4733959
## backward elimination of non-significant effects:
step_result <- step(fm)
## Elimination tables for random- and fixed-effect terms:
step_result
#> Backward reduced random-effect table:
#>
#> Eliminated npar logLik AIC LRT Df Pr(>Chisq)
#> <none> 12 -1352.8 2729.5
#> (1 | Consumer) 0 11 -1354.3 2730.7 3.186 1 0.07425 .
#> (1 | Consumer:Product) 0 11 -1435.0 2891.9 164.443 1 < 2e-16 ***
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> Backward reduced fixed-effect table:
#> Degrees of freedom method: Satterthwaite
#>
#> Eliminated Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
#> Gender 1 1.4656 1.4656 1 79 0.8789 0.35135
#> Information:Product 2 10.3873 3.4624 3 320 2.0765 0.10321
#> Information 0 6.5201 6.5201 1 323 3.8714 0.04997 *
#> Product 0 19.3466 6.4489 3 240 3.8291 0.01048 *
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> Model found:
#> Informed.liking ~ Information + Product + (1 | Consumer) + (1 | Consumer:Product)
# Extract the model that step found:
final_model <- get_model(step_result)