Kernel Density Estimation with Mixed Data Types

npudens computes kernel unconditional density estimates on evaluation data, given a set of training data and a bandwidth specification (a bandwidth object or a bandwidth vector, bandwidth type, and kernel type) using the method of Li and Racine (2003).

Usage

npudens(bws, ...)

# S3 method for class 'formula'
npudens(bws, data = NULL, newdata = NULL, ...)

# S3 method for class 'bandwidth'
npudens(bws,
        tdat = stop("invoked without training data 'tdat'"),
        edat,
        ...,
        se = FALSE)


# Default S3 method
npudens(bws, tdat, ..., se = FALSE)

Arguments

se

Logical; compute and retain asymptotic standard errors when TRUE. The default FALSE skips uncertainty-only computation without changing the point estimates.

Data, Bandwidth Inputs And Formula Interface

These arguments identify the bandwidth specification, formula/data interface, and training data.

bws

a bandwidth specification. This can be set as a bandwidth object returned from an invocation of npudensbw, or as a \(p\)-vector of bandwidths, with an element for each variable in the training data. If specified as a vector, then additional arguments will need to be supplied as necessary to change them from the defaults to specify the bandwidth type, kernel types, training data, and so on.

data

This argument belongs to the formula interface (including supported formula-bearing bandwidth-object refits). Native calls must supply their vectors, matrices or data frames directly, or use with(data, ...). Supplying data, including explicit data = NULL, to a native route that does not consume it is an error; see np-data-interfaces.

an optional data frame, list or environment (or object coercible to a data frame by as.data.frame) containing the variables in the model. If not found in data, the variables are taken from environment(bws), typically the environment from which npudensbw was called.

tdat

a \(p\)-variate data frame of sample realizations (training data) used to estimate the density. Defaults to the corresponding training data recoverable from the bandwidth object in the current session.

Evaluation Data

These arguments control where the fitted density is evaluated.

edat

a \(p\)-variate data frame of density evaluation points. By default, evaluation takes place on the data provided by tdat.

newdata

An optional data frame in which to look for evaluation data. If omitted, the training data are used.

Additional Arguments

Further arguments are passed to npudensbw when bandwidths are computed internally, or used to interpret a numeric bws vector.

...

additional arguments supplied to npudensbw when npudens computes bandwidths internally, or arguments needed to interpret a numeric bws vector. This is where bandwidth selection controls such as bwmethod, bwtype, bwscaling, kernel/support controls such as ckertype, ckerorder, and ckerbound, categorical kernel controls such as ukertype and okertype, and search controls such as nmulti and scale.factor.search.lower are supplied. See npudensbw for the complete bandwidth-selection argument surface.

Details

With generalized nearest-neighbor bandwidths, query identity matters when selecting a radius: a training observation excludes its own occurrence, whereas explicit evaluation data remain external queries even if their values equal training observations. Consequently, predict(fit, newdata = training.data) can differ from fitted(fit). This does not omit the training contribution from the fitted estimator. See np.kernels for the radius convention.

For native (non-formula) objects, named newdata columns are matched to the retained variable names and reordered as needed. Extra columns are ignored; missing, duplicate or ambiguous names are errors. An unnamed matrix or vector remains positional for a single evaluation role. Explicit native evaluation arguments retain their positional contract and take precedence over newdata.

For a native bandwidth object, newdata supplies edat; explicitly supplied native evaluation arguments take precedence. This evaluates at the requested rows without repeating bandwidth selection.

With bwtype="adaptive_nn" and se=TRUE, standard errors include the same-sample influence of each donor-centred nearest-neighbour radius. A two-sided rank-spacing pilot estimates the marginal density sum at the radius endpoints. Cross-coordinate and categorical-kernel covariance are retained in the joint observation influence. The selected NN ranks are treated as fixed; bandwidth-selection uncertainty is not included. This calculation supports Gaussian and Epanechnikov kernels of orders 2, 4, 6 and 8, and uniform kernels, without finite kernel bounds. Uniform density errors additionally include moving-support boundary influences with joint other-coordinate and categorical kernel marks. These use type-7 sample-quantile endpoints and deterministic two-sided rank-spacing pilots whose interior window grows as the two-thirds power of sample size. A strictly absent support face contributes zero; an exact probability endpoint or undefined finite-face pilot is unavailable. Finite kernel bounds (including beta kernels), extended or saturated NN ranks, and degenerate pilot spacings are not currently qualified: point estimates are retained and standard errors are NA, with one availability notice. No additional uncertainty computation is performed for se=FALSE. Requested adaptive-NN standard errors can take several times as long as the point fit because they include the donor-radius influence calculation; this cost is incurred only when standard errors are requested.

Documentation guide: see npudensbw for bandwidth selection and search controls, np.kernels for kernels, np.options for global options, and plot, plot.np for plotting options.

When bws is omitted, the formula and default methods call npudensbw first and pass bandwidth-selection arguments from ... to that call. When bws is already a bandwidth object, npudens estimates with the stored bandwidth metadata in that object.

The beta associated density described in np.kernels is available for continuous coordinates at orders 2, 4, 6, and 8 with manual or cross-validated metric bandwidths. Such coordinates may be combined with unordered and ordered categorical coordinates. Fixed, generalized-NN, and adaptive-NN bandwidths are supported. Use ckertype="beta" and either ckerbound="fixed" with finite lower and upper bounds or ckerbound="range" for beta outer half-spacing sample bounds as defined in np.kernels. Every continuous training and evaluation value must lie within the resulting support. For example:

    fhat <- npudens(tdat=data.frame(x), edat=data.frame(x=x.eval),
                    bws=0.12, ckertype="beta", ckerorder=2,
                    ckerbound="fixed", ckerlb=0, ckerub=1)
  

The beta contribution already includes its complete original-scale density normalization. No additional division by the bandwidth is performed. Beta density routes require at least one continuous coordinate and reject normal-reference selection. Cross-validated cv.ml and bounded target-coordinate cv.ls are available; the latter integrates the actual estimator and does not substitute the public beta-overlap operator. Orders 4, 6, and 8 are signed and can produce negative density estimates; no positivity repair is applied.

Because an associated beta kernel has target-dependent shape and roughness, its reported standard error does not substitute a translation-invariant \(R(K)\) constant. If \(K_i(x)\) denotes the full multivariate beta contribution, the implementation reports the ordinary standard error of their sample mean, $$\widehat{\mathrm{se}}\{\hat f(x)\}= \sqrt{\frac{1}{n}\,\frac{1}{n-1} \sum_{i=1}^n\{K_i(x)-\hat f(x)\}^2},$$ with zero reported when \(n=1\). This contribution-based expression automatically respects the changing boundary shape.

When all variables are categorical, uncertainty uses the complete joint-category contributions \(Z_i(x)\) produced by the selected kernel computation. With \(\bar Z(x)=n^{-1}\sum_i Z_i(x)\), the empirical-population convention is $$\widehat{\mathrm{se}}\{\hat f(x)\}= \frac{1}{n}\sqrt{\sum_{i=1}^n\{Z_i(x)-\bar Z(x)\}^2}.$$ Thus zero smoothing retains \(\sqrt{\hat p(x)(1-\hat p(x))/n}\), while exactly constant contributions have zero standard error. Category-profile compression includes the original observation counts; it does not treat distinct profiles as equally frequent observations.

For ordinary kernels with mixed continuous and categorical data, fixed and generalized-nearest-neighbour standard errors retain the leading continuous- bandwidth approximation while using the squared joint categorical kernel product. Thus a categorical factor that multiplies the density by a constant also multiplies its standard error by that constant. This is not the sample variance of finite-bandwidth continuous kernel contributions. Generalized-NN errors use the existing frozen-query-radius approximation. The separate adaptive-NN influence calculation is described above.

Argument groups for bandwidth selection are documented on npudensbw. The most common workflow is to choose data and bandwidth inputs first, then bandwidth criterion and representation, then kernel/support controls and numerical search controls if defaults need to be changed.

For S3 plotting help, see plot.np. You can list available plot methods with methods("plot").

Typical usages are (see below for a complete list of options and also the examples at the end of this help file)

    Usage 1: first compute the bandwidth object via npudensbw and then
    compute the density:

    bw <- npudensbw(~y)
    fhat <- npudens(bw)

    Usage 2: alternatively, compute the bandwidth object indirectly:

    fhat <- npudens(~y)

    Usage 3: modify the default kernel and order:

    fhat <- npudens(~y, ckertype="epanechnikov", ckerorder=4)

    Usage 4: use the data frame interface rather than the formula
    interface:

    fhat <- npudens(tdat = y, ckertype="epanechnikov", ckerorder=4)
  

npudens implements a variety of methods for estimating multivariate density functions (\(p\)-variate) defined over a set of possibly continuous and/or discrete (unordered, ordered) data. The approach is based on Li and Racine (2003) who employ ‘generalized product kernels’ that admit a mix of continuous and discrete data types.

Three classes of kernel estimators for the continuous data types are available: fixed, adaptive nearest-neighbor, and generalized nearest-neighbor. Adaptive nearest-neighbor bandwidths change with each sample realization in the set, \(x_i\), when estimating the density at the point \(x\). Generalized nearest-neighbor bandwidths change with the point at which the density is estimated, \(x\). Fixed bandwidths are constant over the support of \(x\).

Data contained in the data frame tdat (and also edat) may be a mix of continuous (default), unordered discrete (to be specified in the data frame tdat using the factor command), and ordered discrete (to be specified in the data frame tdat using the ordered command). Data can be entered in an arbitrary order and data types will be detected automatically by the routine (see np for details).

A variety of kernels may be specified by the user. Kernels implemented for continuous data types include the second, fourth, sixth, and eighth order Gaussian and Epanechnikov kernels, and the uniform kernel. Unordered discrete data types use a variation on Aitchison and Aitken's (1976) kernel, while ordered data types use a variation of the Wang and van Ryzin (1981) kernel.

Value

Standard errors are opt-in. To add them to an existing fit without repeating bandwidth selection, use fit <- npudens(bws=fit$bws, se=TRUE), supplying the original training data and desired evaluation data again when these cannot be recovered from the bandwidth object's call. The se() extractor reads retained errors only and gives a helpful refit message when they were not requested. Use predict(fit, se.fit=TRUE, ...) for prediction with standard errors.

npudens returns a npdensity object. The generic accessor functions fitted, and se, extract estimated values and asymptotic standard errors on estimates, respectively, from the returned object. Furthermore, the functions predict, summary and plot support objects of both classes. The returned objects have the following components:

eval

the evaluation points.

dens

estimation of the density at the evaluation points

derr

Standard errors of the density estimates when se=TRUE, otherwise an empty numeric vector.

se

Whether standard-error computation was requested.

log_likelihood

aggregate fitted-density log likelihood. Every represented strictly positive density estimate, including a positive subnormal value, contributes its literal logarithm. An exact zero or negative density estimate has no mathematical logarithm and is represented in this aggregate by the established computational placeholder log(DBL_MIN). Consequently this component is not a literal log likelihood if any fitted density is nonpositive.

Book And Method Pointers

The unconditional density target is \(f(x)\). In the continuous fixed-bandwidth case the estimator has the usual kernel form \(\hat f(x)=n^{-1}\sum_i K_h(X_i,x)\); with mixed data, npudens uses generalized product kernels so that the continuous, unordered, and ordered components each contribute their appropriate kernel factor. Nearest-neighbor bandwidth types change how the continuous bandwidth is indexed, not the statistical target.

For book-length derivations, see Li and Racine (2007), Chapter 1 Density Estimation, especially Sections 1.1, 1.3, 1.6, 1.8, and 1.9, and Chapter 4 Kernel Estimation with Mixed Data. The later workflow treatment is Racine (2019), Chapter 2 Continuous Density and Cumulative Distribution Functions and Chapter 3 Mixed-Data Probability Density and Cumulative Distribution Functions.

References

Aitchison, J. and C.G.G. Aitken (1976), “ Multivariate binary discrimination by the kernel method,” Biometrika, 63, 413-420.

Li, Q. and J.S. Racine (2007), Nonparametric Econometrics: Theory and Practice, Princeton University Press.

Li, Q. and J.S. Racine (2003), “Nonparametric estimation of distributions with categorical and continuous data,” Journal of Multivariate Analysis, 86, 266-292.

Ouyang, D. and Q. Li and J.S. Racine (2006), “Cross-validation and the estimation of probability distributions with categorical data,” Journal of Nonparametric Statistics, 18, 69-100.

Pagan, A. and A. Ullah (1999), Nonparametric Econometrics, Cambridge University Press.

Scott, D.W. (1992), Multivariate Density Estimation: Theory, Practice and Visualization, New York: Wiley.

Silverman, B.W. (1986), Density Estimation, London: Chapman and Hall.

Wang, M.C. and J. van Ryzin (1981), “A class of smooth estimators for discrete distributions,” Biometrika, 68, 301-309.

Chen, S.X. (1999), “Beta kernel estimators for density functions,” Computational Statistics & Data Analysis, 31(2), 131–145. doi:10.1016/S0167-9473(99)00010-9.

Igarashi, G. and Kakizawa, Y. (2020), “Higher-order bias corrections for asymmetric kernel density estimators,” Journal of Nonparametric Statistics, 32(3), 716–752. doi:10.1080/10485252.2020.1770754.

Author

Tristen Hayfield tristen.hayfield@gmail.com, Jeffrey S. Racine racinej@mcmaster.ca

Usage Issues

If you are using data of mixed types, then it is advisable to use the data.frame function to construct your input data and not cbind, since cbind will typically not work as intended on mixed data types and will coerce the data to the same type.

For formula fits, character-valued missing-data policies are resolved through the stats model-frame owner; pass a caller-local policy as a function object rather than its name. With replacement training data, an explicit na.action, including NULL, takes precedence over the saved policy. Native fitting still requires complete rows, so a permissive policy does not guarantee the same retained sample as stats::model.frame.

When reusing a bandwidth object without explicit training or evaluation data, newly constructed bandwidth objects use retained training values. A fit with explicit replacement training data retains that sample in its returned bandwidth object; evaluation grids do not replace training data. Older objects without retained samples use their stored call environment when available and may require explicit training inputs after saveRDS() and readRDS(). Custom formula transformations used on new evaluation data must still be available.

See also

np.kernels, np.options, plot, plot.np npudensbw , density

Examples

if (FALSE) { # \dontrun{
# EXAMPLE 1 (INTERFACE=FORMULA): For this example, we load Giovanni
# Baiocchi's Italian GDP panel (see Italy for details), then create a
# data frame in which year is an ordered factor, GDP is continuous,
# compute bandwidths using likelihood cross-validation, then create a
# grid of data on which the density will be evaluated for plotting
# purposes.

data("Italy")
with(Italy, {

# Compute bandwidths using likelihood cross-validation (default).

bw <- npudensbw(formula=~ordered(year)+gdp)

# At this stage you could use npudens() to do a variety of
# things. Here we compute the npudens() object and place it in fhat.

fhat <- npudens(bws=bw)

# Note that simply typing the name of the object returns some useful
# information. For more info, one can call summary:

summary(fhat)

# Next, we illustrate how to create a grid of `evaluation data' and feed
# it to the perspective plotting routines in R, among others.

# Create an evaluation data matrix

year.seq <- sort(unique(year))
gdp.seq <- seq(1,36,length=50)
data.eval <- expand.grid(year=year.seq,gdp=gdp.seq)

# Generate the estimated density computed for the evaluation data

fhat <- fitted(npudens(bws=bw, newdata=data.eval))

# Coerce the data into a matrix for plotting with persp()

f <- matrix(fhat, length(unique(year)), 50)

# Next, create a 3D perspective plot of the PDF f, and a 2D
# contour plot.

persp(as.integer(levels(year.seq)), gdp.seq, f, col="lightblue",
      ticktype="detailed", ylab="GDP", xlab="Year", zlab="Density",
      theta=300, phi=50)

# Sleep for 5 seconds so that we can examine the output...

if (interactive()) Sys.sleep(5)

contour(as.integer(levels(year.seq)),
        gdp.seq,
        f,
        xlab="Year",
        ylab="GDP",
        main = "Density Contour Plot",
        col=topo.colors(100))

# Sleep for 5 seconds so that we can examine the output...

if (interactive()) Sys.sleep(5)

# Alternatively, you could use the plot() command (<ctrl>-C will
# interrupt on *NIX systems, <esc> will interrupt on MS Windows
# systems).

if (interactive()) plot(bw)

})

# EXAMPLE 1 (INTERFACE=DATA FRAME): For this example, we load Giovanni
# Baiocchi's Italian GDP panel (see Italy for details), then create a
# data frame in which year is an ordered factor, GDP is continuous,
# compute bandwidths using likelihood cross-validation, then create a
# grid of data on which the density will be evaluated for plotting
# purposes.

data("Italy")
with(Italy, {

data <- data.frame(year=ordered(year), gdp)

# Compute bandwidths using likelihood cross-validation (default).

bw <- npudensbw(dat=data)

# At this stage you could use npudens() to do a variety of
# things. Here we compute the npudens() object and place it in fhat.

fhat <- npudens(bws=bw)

# Note that simply typing the name of the object returns some useful
# information. For more info, one can call summary:

summary(fhat)

# Next, we illustrate how to create a grid of `evaluation data' and feed
# it to the perspective plotting routines in R, among others.

# Create an evaluation data matrix

year.seq <- sort(unique(year))
gdp.seq <- seq(1,36,length=50)
data.eval <- expand.grid(year=year.seq,gdp=gdp.seq)

# Generate the estimated density computed for the evaluation data

fhat <- fitted(npudens(edat = data.eval, bws=bw))

# Coerce the data into a matrix for plotting with persp()

f <- matrix(fhat, length(unique(year)), 50)

# Next, create a 3D perspective plot of the PDF f, and a 2D
# contour plot.

persp(as.integer(levels(year.seq)), gdp.seq, f, col="lightblue",
      ticktype="detailed", ylab="GDP", xlab="Year", zlab="Density",
      theta=300, phi=50)

# Sleep for 5 seconds so that we can examine the output...

if (interactive()) Sys.sleep(5)

contour(as.integer(levels(year.seq)),
        gdp.seq,
        f,
        xlab="Year",
        ylab="GDP",
        main = "Density Contour Plot",
        col=topo.colors(100))

# Sleep for 5 seconds so that we can examine the output...

if (interactive()) Sys.sleep(5)

# Alternatively, you could use the plot() command (<ctrl>-C will
# interrupt on *NIX systems, <esc> will interrupt on MS Windows
# systems).

if (interactive()) plot(bw)

})

# EXAMPLE 2 (INTERFACE=FORMULA): For this example, we load the old
# faithful geyser data and compute the density and distribution
# functions.

library("datasets")
data("faithful")
with(faithful, {

# Note - this may take a few minutes depending on the speed of your
# computer...

bw <- npudensbw(formula=~eruptions+waiting)

summary(bw)

# Plot the density function (<ctrl>-C will interrupt on *NIX systems,
# <esc> will interrupt on MS Windows systems). Note that we use xtrim =
# -0.2 to extend the plot outside the support of the data (i.e., extend
# the tails of the estimate to meet the horizontal axis).

if (interactive()) plot(bw, xtrim=-0.2)

})

# EXAMPLE 2 (INTERFACE=DATA FRAME): For this example, we load the old
# faithful geyser data and compute the density and distribution
# functions.

library("datasets")
data("faithful")
with(faithful, {

# Note - this may take a few minutes depending on the speed of your
# computer...

bw <- npudensbw(dat=faithful)

summary(bw)

# Plot the density function (<ctrl>-C will interrupt on *NIX systems,
# <esc> will interrupt on MS Windows systems). Note that we use xtrim =
# -0.2 to extend the plot outside the support of the data (i.e., extend
# the tails of the estimate to meet the horizontal axis).

if (interactive()) plot(bw, xtrim=-0.2)

})
} # }