Hyperparameter estimation for the Gaussian Radial Basis kernel

Given a range of values for the "sigma" inverse width parameter in the Gaussian Radial Basis kernel for use with Support Vector Machines. The estimation is based on the data to be used.

# S4 method for class 'formula'
sigest(x, data=NULL, frac = 0.5, na.action = na.omit, scaled = TRUE)
# S4 method for class 'matrix'
sigest(x, frac = 0.5, scaled = TRUE, na.action = na.omit)

Arguments

x: a symbolic description of the model upon the estimation is based. When not using a formula x is a matrix or vector containing the data
data: an optional data frame containing the variables in the model. By default the variables are taken from the environment which `ksvm' is called from.
frac: Fraction of data to use for estimation. By default a quarter of the data is used to estimate the range of the sigma hyperparameter.
scaled: A logical vector indicating the variables to be scaled. If scaled is of length 1, the value is recycled as many times as needed and all non-binary variables are scaled. Per default, data are scaled internally to zero mean and unit variance (since this the default action in ksvm as well). The center and scale values are returned and used for later predictions.
na.action: A function to specify the action to be taken if NAs are found. The default action is na.omit, which leads to rejection of cases with missing values on any required variable. An alternative is na.fail, which causes an error if NA cases are found. (NOTE: If given, this argument must be named.)

Details

sigest estimates the range of values for the sigma parameter which would return good results when used with a Support Vector Machine (ksvm). The estimation is based upon the 0.1 and 0.9 quantile of \(\|x -x'\|^2\). Basically any value in between those two bounds will produce good results.

Value

Returns a vector of length 3 defining the range (0.1 quantile, median and 0.9 quantile) of the sigma hyperparameter.

References

B. Caputo, K. Sim, F. Furesjo, A. Smola,
Appearance-based object recognition using SVMs: which kernel should I use?
Proc of NIPS workshop on Statitsical methods for computational experiments in visual processing and computer vision, Whistler, 2002.

Author

Alexandros Karatzoglou
alexandros.karatzoglou@ci.tuwien.ac.at

Examples


## estimate good sigma values for promotergene
data(promotergene)
srange <- sigest(Class~.,data = promotergene)
srange
#>        90%        50%        10% 
#> 0.01449275 0.01639344 0.01845018 

s <- srange[2]
s
#>        50% 
#> 0.01639344 
## create test and training set
ind <- sample(1:dim(promotergene)[1],20)
genetrain <- promotergene[-ind, ]
genetest <- promotergene[ind, ]

## train a support vector machine
gene <- ksvm(Class~.,data=genetrain,kernel="rbfdot",
             kpar=list(sigma = s),C=50,cross=3)
gene
#> Support Vector Machine object of class "ksvm" 
#> 
#> SV type: C-svc  (classification) 
#>  parameter : cost C = 50 
#> 
#> Gaussian Radial Basis kernel function. 
#>  Hyperparameter : sigma =  0.0163934426229508 
#> 
#> Number of Support Vectors : 81 
#> 
#> Objective Function Value : -44.3314 
#> Training error : 0.05814 
#> Cross validation error : 0.127668 

## predict gene type on the test set
promoter <- predict(gene,genetest[,-1])

## Check results
table(promoter,genetest[,1])
#>         
#> promoter  +  -
#>        + 12  1
#>        -  0  7