A method for multcomp::cld() is provided for users desiring to produce
compact-letter displays (CLDs).
This method uses the Piepho (2004) algorithm (as implemented in the
multcompView package) to generate a compact letter display of all
pairwise comparisons of estimated marginal means. The function obtains (possibly
adjusted) P values for all pairwise comparisons of means, using the
contrast function with method = "pairwise". When a P
value exceeds alpha, then the two means have at least one letter in
common.
# S3 method for class 'emmGrid'
cld(object, details = FALSE, sort = TRUE, by,
alpha = 0.05, Letters = c("1234567890", LETTERS, letters),
reversed = decreasing, decreasing = FALSE, signif.sets = FALSE,
delta = 0, ...)
# S3 method for class 'emm_list'
cld(object, ..., which = 1)An object of class emmGrid
Logical value determining whether detailed information on tests of pairwise comparisons is displayed
Logical value determining whether the EMMs are sorted before the comparisons
are produced. When TRUE, the results are displayed according to
reversed.
Character value giving the name or names of variables by which separate
families of comparisons are tested. If NULL, all means are compared.
If missing, the object's by.vars setting, if any, is used.
Numeric value giving the significance level for the comparisons
Character vector of letters to use in the display. Any strings of length greater than 1 are expanded into individual characters
Logical value (passed to multcompView::multcompLetters.)
If TRUE, the order of use of the letters is reversed.
Either reversed or decreasing may be specified, thus providing
compatibility with both multcompView::multcompLetters(..., reversed, ...) and
multcomp::cld(..., decreasing, ...).
In addition, if both sort and reversed are TRUE, the sort
order of results is reversed.
Logical value. If FALSE (and delta = 0), a
‘traditional’
compact-letter display is constructed with groupings representing sets of
estimates that are not statistically different. If TRUE, the criteria
are reversed so that two estimates sharing the same symbol test as significantly
different. See also delta.
Numeric value passed to test.emmGrid. If this
is positive, it is used as an equivalence threshold in the TOST procedure for
two-sided equivalence testing. In the resulting compact letter display,
two estimates share the same grouping letter only if they are found to be
statistically equivalent – that is, groupings reflect actual findings
of equivalence rather than failure to find a significant difference.
When delta is nonzero, signif.sets is ignored.
Arguments passed to contrast (for example,
an adjust method)
Which element of the emm_list object to process
(If length exceeds one, only the first one is used)
A summary_emm object showing the estimated marginal means
plus an additional column labeled .group (when signif.sets = FALSE),
.signif.set (when signif.sets = TRUE), or .equiv.set
(when delta > 0).
We warn that the default display encourages a poor
practice in interpreting significance tests. Such CLDs are misleading because they
visually group means with comparisons P > alpha as though they
are equal, when in fact we have only failed to prove that they differ.
A better alternative if one wants to show groupings is to specify an equivalence
threshold delta; then groupings will be based on actual findings of
equivalence. Another way to display actual findings is to set
signif.sets = TRUE, so that estimates in the same group are those
found to be statistically different. Obviously, these different options
require different interpretations of the results; the annotations and the label
given the final column help guide how to assess the results.
As further alternatives, consider pwpp (graphical display of P
values) or pwpm (matrix display).
Piepho, Hans-Peter (2004) An algorithm for a letter-based representation of all pairwise comparisons, Journal of Computational and Graphical Statistics, 13(2), 456-466.
if(requireNamespace("multcomp"))
emm_example("cld-multcomp")
#>
#> --- Running code from 'system.file("extexamples", "cld-multcomp.R", package = "emmeans")'
#>
#> > pigs.lm <- lm(log(conc) ~ source + factor(percent),
#> + data = pigs)
#>
#> > pigs.emm <- emmeans(pigs.lm, "percent", type = "response")
#>
#> > multcomp::cld(pigs.emm, alpha = 0.1, Letters = LETTERS)
#> percent response SE df lower.CL upper.CL .group
#> 9 31.4 1.28 23 28.8 34.1 A
#> 12 37.5 1.44 23 34.7 40.6 B
#> 15 39.0 1.70 23 35.6 42.7 B
#> 18 42.3 2.24 23 37.9 47.2 B
#>
#> Results are averaged over the levels of: source
#> Confidence level used: 0.95
#> Intervals are back-transformed from the log scale
#> P value adjustment: tukey method for comparing a family of 4 estimates
#> Tests are performed on the log scale
#> significance level used: alpha = 0.1
#> NOTE: If two or more means share the same grouping symbol,
#> then we cannot show them to be different.
#> But we also did not show them to be the same.
#>
#> > multcomp::cld(pigs.emm, alpha = 0.1, signif.sets = TRUE)
#> percent response SE df lower.CL upper.CL .signif.set
#> 9 31.4 1.28 23 28.8 34.1 123
#> 12 37.5 1.44 23 34.7 40.6 1
#> 15 39.0 1.70 23 35.6 42.7 2
#> 18 42.3 2.24 23 37.9 47.2 3
#>
#> Results are averaged over the levels of: source
#> Confidence level used: 0.95
#> Intervals are back-transformed from the log scale
#> P value adjustment: tukey method for comparing a family of 4 estimates
#> Tests are performed on the log scale
#> significance level used: alpha = 0.1
#> Estimates sharing the same symbol are significantly different
#>
#> > multcomp::cld(pigs.emm, delta = log(1.25), adjust = "sidak")
#> percent response SE df lower.CL upper.CL .equiv.set
#> 9 31.4 1.28 23 28.1 35.0 1
#> 12 37.5 1.44 23 33.8 41.6 2
#> 15 39.0 1.70 23 34.6 43.9 2
#> 18 42.3 2.24 23 36.7 48.8 3
#>
#> Results are averaged over the levels of: source
#> Confidence level used: 0.95
#> Conf-level adjustment: sidak method for 4 estimates
#> Intervals are back-transformed from the log scale
#> P value adjustment: sidak method for 6 tests
#> Statistics are tests of equivalence with a threshold of 0.22314
#> P values are left-tailed
#> Tests are performed on the log scale
#> significance level used: alpha = 0.05
#> Estimates sharing the same symbol test as equivalent
#>
# Use emm_example("cld-multcomp", list = TRUE) # to just list the code