Nelder-Mead Optimization of Parameters, Possibly (Box) Constrained

Nelder-Mead optimization of parameters, allowing optimization subject to box constraints (contrary to the default, method = "Nelder-Mead", in R's optim()), and using reverse communications.

Nelder_Mead(fn, par, lower = rep.int(-Inf, n), upper = rep.int(Inf, n),
            control = list())

Arguments

fn

a function of a single numeric vector argument returning a numeric scalar.

par

numeric vector of starting values for the parameters.

lower

numeric vector of lower bounds (elements may be -Inf).

upper

numeric vector of upper bounds (elements may be Inf).

control

a named list of control settings. Possible settings are

iprint

numeric scalar - frequency of printing evaluation information. Defaults to 0 indicating no printing.

maxfun

numeric scalar - maximum number of function evaluations allowed (default:10000).

FtolAbs

numeric scalar - absolute tolerance on change in function values (default: 1e-5)

FtolRel

numeric scalar - relative tolerance on change in function values (default:1e-15)

XtolRel

numeric scalar - relative tolerance on change in parameter values (default: 1e-7)

MinfMax

numeric scalar - maximum value of the minimum (default: .Machine$double.xmin)

xst

numeric vector of initial step sizes to establish the simplex - all elements must be non-zero (default: rep(0.02,length(par)))

xt

numeric vector of tolerances on the parameters (default: xst*5e-4)

verbose

numeric value: 0=no printing, 1=print every 20 evaluations, 2=print every 10 evalutions, 3=print every evaluation. Sets ‘iprint’, if specified, but does not override it.

warnOnly

a logical indicating if non-convergence (codes -1,-2,-3) should not stop(.), but rather only call warning and return a result which might inspected. Defaults to FALSE, i.e., stop on non-convergence.

Value

a list with components

fval

numeric scalar - the minimum function value achieved

par

numeric vector - the value of x providing the minimum

convergence

integer valued scalar, if not 0, an error code:

-4

nm_evals: maximum evaluations reached

-3

nm_forced: ?

-2

nm_nofeasible: cannot generate a feasible simplex

-1

nm_x0notfeasible: initial x is not feasible (?)

0

successful convergence

message

a string specifying the kind of convergence.

control

the list of control settings after substituting for defaults.

feval

the number of function evaluations.

See also

The NelderMead class definition and generator function.

Examples

fr <- function(x) {   ## Rosenbrock Banana function
    x1 <- x[1]
    x2 <- x[2]
    100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}
p0 <- c(-1.2, 1)

oo  <- optim(p0, fr) ## also uses Nelder-Mead by default
o.  <- Nelder_Mead(fr, p0)
o.1 <- Nelder_Mead(fr, p0, control=list(verbose=1))# -> some iteration output
#> (NM) 20: f = 4.09883 at -0.998242   1.02902
#> (NM) 40: f = 3.85065 at -0.946969  0.921237
#> (NM) 60: f = 2.12558 at -0.43529 0.215075
#> (NM) 80: f = 1.052 at -0.00264012  -0.0216052
#> (NM) 100: f = 0.351113 at 0.431272 0.169364
#> (NM) 120: f = 0.0662375 at  0.77641 0.590067
#> (NM) 140: f = 0.00120832 at 0.976014  0.95512
#> (NM) 160: f = 8.04753e-06 at 1.00132  1.0029
#> (NM) 180: f = 4.64058e-08 at 0.999793 0.999593
#> (NM) 200: f = 1.94299e-10 at 0.999989 0.999979
stopifnot(identical(o.[1:4], o.1[1:4]),
          all.equal(o.$par, oo$par, tolerance=1e-3))# diff: 0.0003865


o.2 <- Nelder_Mead(fr, p0, control=list(verbose=3, XtolRel=1e-15, FtolAbs= 1e-14))
#> (NM) 1: f = inf at -1.2    1
#> (NM) init_pos <= d_n
#> (NM) 2: f = 24.2 at -1.2    1
#> (NM) init_pos <= d_n
#> (NM) 3: f = 20.1502 at -1.18     1
#> (NM) init_pos <= d_n
#> (NM) 4: f = 20.1502 at -1.18     1
#> (NM) 5: f = 18.6206 at -1.18  1.02
#> (NM) 6: f = 16.1942 at -1.17  1.03
#> (NM) 7: f = 14.3881 at -1.15  1.01
#> (NM) 8: f = 11.3082 at -1.125  1.005
#> (NM) 9: f = 8.80899 at -1.115  1.035
#> (NM) 10: f = 5.7602 at -1.0825  1.0525
#> (NM) 11: f = 4.39059 at -1.0375  1.0275
#> (NM) 12: f = 4.39059 at -1.0375  1.0275
#> (NM) 13: f = 4.39059 at -1.0375  1.0275
#> (NM) 14: f = 4.39059 at -1.0375  1.0275
#> (NM) 15: f = 4.39059 at -1.0375  1.0275
#> (NM) 16: f = 4.39059 at -1.0375  1.0275
#> (NM) 17: f = 4.12674 at -1.02852  1.04695
#> (NM) 18: f = 4.12674 at -1.02852  1.04695
#> (NM) 19: f = 4.12674 at -1.02852  1.04695
#> (NM) 20: f = 4.09883 at -0.998242   1.02902
#> (NM) 21: f = 4.09883 at -0.998242   1.02902
#> (NM) 22: f = 4.07791 at -1.01767  1.02731
#> (NM) 23: f = 4.06835 at -0.987393   1.00938
#> (NM) 24: f = 4.06835 at -0.987393   1.00938
#> (NM) 25: f = 4.03092 at -1.00682  1.00768
#> (NM) 26: f = 4.03092 at -1.00682  1.00768
#> (NM) 27: f = 4.03092 at -1.00682  1.00768
#> (NM) 28: f = 3.98541 at -0.995967  0.988037
#> (NM) 29: f = 3.98541 at -0.995967  0.988037
#> (NM) 30: f = 3.98541 at -0.995967  0.988037
#> (NM) 31: f = 3.98541 at -0.995967  0.988037
#> (NM) 32: f = 3.98541 at -0.995967  0.988037
#> (NM) 33: f = 3.98078 at -0.988967  0.993801
#> (NM) 34: f = 3.94339 at -0.978118  0.974163
#> (NM) 35: f = 3.93793 at -0.963768  0.957406
#> (NM) 36: f = 3.93793 at -0.963768  0.957406
#> (NM) 37: f = 3.93793 at -0.963768  0.957406
#> (NM) 38: f = 3.89364 at -0.960968  0.945425
#> (NM) 39: f = 3.85065 at -0.946969  0.921237
#> (NM) 40: f = 3.85065 at -0.946969  0.921237
#> (NM) 41: f = 3.77359 at -0.90777 0.860654
#> (NM) 42: f = 3.68007 at -0.879771  0.812278
#> (NM) 43: f = 3.67015 at -0.90217 0.836692
#> (NM) 44: f = 3.59215 at -0.890971  0.806627
#> (NM) 45: f = 3.36249 at -0.823773  0.697667
#> (NM) 46: f = 3.10773 at -0.762175  0.585882
#> (NM) 47: f = 3.10773 at -0.762175  0.585882
#> (NM) 48: f = 3.0182 at -0.64458 0.359486
#> (NM) 49: f = 3.0182 at -0.64458 0.359486
#> (NM) 50: f = 2.79777 at -0.63338 0.365137
#> (NM) 51: f = 2.79777 at -0.63338 0.365137
#> (NM) 52: f = 2.79777 at -0.63338 0.365137
#> (NM) 53: f = 2.79777 at -0.63338 0.365137
#> (NM) 54: f = 2.59475 at -0.601882  0.345315
#> (NM) 55: f = 2.49823 at -0.580533  0.338229
#> (NM) 56: f = 2.49823 at -0.580533  0.338229
#> (NM) 57: f = 2.18451 at -0.477988  0.229247
#> (NM) 58: f = 2.12558 at -0.43529 0.215075
#> (NM) 59: f = 2.12558 at -0.43529 0.215075
#> (NM) 60: f = 2.12558 at -0.43529 0.215075
#> (NM) 61: f = 1.85847 at -0.339132 0.0894769
#> (NM) 62: f = 1.85847 at -0.339132 0.0894769
#> (NM) 63: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 64: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 65: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 66: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 67: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 68: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 69: f = 1.74109 at  -0.169346 -0.0324544
#> (NM) 70: f = 1.6557 at -0.269968 0.0521755
#> (NM) 71: f = 1.55396 at    -0.17894 -0.00848461
#> (NM) 72: f = 1.50655 at  -0.138222 -0.0268297
#> (NM) 73: f = 1.50655 at  -0.138222 -0.0268297
#> (NM) 74: f = 1.33243 at -0.107098 -0.021205
#> (NM) 75: f = 1.33243 at -0.107098 -0.021205
#> (NM) 76: f = 1.33243 at -0.107098 -0.021205
#> (NM) 77: f = 1.33243 at -0.107098 -0.021205
#> (NM) 78: f = 1.16367 at -0.0478342 -0.0233467
#> (NM) 79: f = 1.052 at -0.00264012  -0.0216052
#> (NM) 80: f = 1.052 at -0.00264012  -0.0216052
#> (NM) 81: f = 0.980011 at 0.0203829 0.0146849
#> (NM) 82: f = 0.794206 at  0.110771 0.0181679
#> (NM) 83: f = 0.794206 at  0.110771 0.0181679
#> (NM) 84: f = 0.794206 at  0.110771 0.0181679
#> (NM) 85: f = 0.794206 at  0.110771 0.0181679
#> (NM) 86: f = 0.794206 at  0.110771 0.0181679
#> (NM) 87: f = 0.607797 at  0.224182 0.0579409
#> (NM) 88: f = 0.607797 at  0.224182 0.0579409
#> (NM) 89: f = 0.607797 at  0.224182 0.0579409
#> (NM) 90: f = 0.607797 at  0.224182 0.0579409
#> (NM) 91: f = 0.44247 at 0.375246 0.117973
#> (NM) 92: f = 0.44247 at 0.375246 0.117973
#> (NM) 93: f = 0.44247 at 0.375246 0.117973
#> (NM) 94: f = 0.44247 at 0.375246 0.117973
#> (NM) 95: f = 0.351113 at 0.431272 0.169364
#> (NM) 96: f = 0.351113 at 0.431272 0.169364
#> (NM) 97: f = 0.351113 at 0.431272 0.169364
#> (NM) 98: f = 0.351113 at 0.431272 0.169364
#> (NM) 99: f = 0.351113 at 0.431272 0.169364
#> (NM) 100: f = 0.351113 at 0.431272 0.169364
#> (NM) 101: f = 0.308628 at 0.445597 0.194999
#> (NM) 102: f = 0.270158 at 0.480773 0.233512
#> (NM) 103: f = 0.270158 at 0.480773 0.233512
#> (NM) 104: f = 0.197707 at 0.593757 0.334471
#> (NM) 105: f = 0.197707 at 0.593757 0.334471
#> (NM) 106: f = 0.197707 at 0.593757 0.334471
#> (NM) 107: f = 0.197707 at 0.593757 0.334471
#> (NM) 108: f = 0.197707 at 0.593757 0.334471
#> (NM) 109: f = 0.126096 at 0.645994 0.420093
#> (NM) 110: f = 0.126096 at 0.645994 0.420093
#> (NM) 111: f = 0.126096 at 0.645994 0.420093
#> (NM) 112: f = 0.106835 at 0.690501 0.487301
#> (NM) 113: f = 0.106835 at 0.690501 0.487301
#> (NM) 114: f = 0.106835 at 0.690501 0.487301
#> (NM) 115: f = 0.106835 at 0.690501 0.487301
#> (NM) 116: f = 0.106835 at 0.690501 0.487301
#> (NM) 117: f = 0.10306 at 0.688431 0.466201
#> (NM) 118: f = 0.0727579 at 0.732938 0.533409
#> (NM) 119: f = 0.0662375 at  0.77641 0.590067
#> (NM) 120: f = 0.0662375 at  0.77641 0.590067
#> (NM) 121: f = 0.0662375 at  0.77641 0.590067
#> (NM) 122: f = 0.0458798 at 0.799439 0.631583
#> (NM) 123: f = 0.0458798 at 0.799439 0.631583
#> (NM) 124: f = 0.0458798 at 0.799439 0.631583
#> (NM) 125: f = 0.0458798 at 0.799439 0.631583
#> (NM) 126: f = 0.0274164 at 0.842102 0.704151
#> (NM) 127: f = 0.0175222 at 0.874948 0.761194
#> (NM) 128: f = 0.0175222 at 0.874948 0.761194
#> (NM) 129: f = 0.0175222 at 0.874948 0.761194
#> (NM) 130: f = 0.0175222 at 0.874948 0.761194
#> (NM) 131: f = 0.0175222 at 0.874948 0.761194
#> (NM) 132: f = 0.0175222 at 0.874948 0.761194
#> (NM) 133: f = 0.00997536 at 0.900123 0.810242
#> (NM) 134: f = 0.00440675 at 0.943057 0.885944
#> (NM) 135: f = 0.00440675 at 0.943057 0.885944
#> (NM) 136: f = 0.00440675 at 0.943057 0.885944
#> (NM) 137: f = 0.00395144 at 0.942326 0.890478
#> (NM) 138: f = 0.00120832 at 0.976014  0.95512
#> (NM) 139: f = 0.00120832 at 0.976014  0.95512
#> (NM) 140: f = 0.00120832 at 0.976014  0.95512
#> (NM) 141: f = 0.00120832 at 0.976014  0.95512
#> (NM) 142: f = 0.00120832 at 0.976014  0.95512
#> (NM) 143: f = 0.00120832 at 0.976014  0.95512
#> (NM) 144: f = 0.00120832 at 0.976014  0.95512
#> (NM) 145: f = 0.00101227 at 1.00728 1.01771
#> (NM) 146: f = 0.00101227 at 1.00728 1.01771
#> (NM) 147: f = 0.000184224 at  1.0134 1.02676
#> (NM) 148: f = 0.000184224 at  1.0134 1.02676
#> (NM) 149: f = 0.000184224 at  1.0134 1.02676
#> (NM) 150: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 151: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 152: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 153: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 154: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 155: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 156: f = 7.50713e-05 at 0.999298 0.997733
#> (NM) 157: f = 4.19678e-05 at 1.00646 1.01293
#> (NM) 158: f = 4.19678e-05 at 1.00646 1.01293
#> (NM) 159: f = 8.04753e-06 at 1.00132  1.0029
#> (NM) 160: f = 8.04753e-06 at 1.00132  1.0029
#> (NM) 161: f = 8.04753e-06 at 1.00132  1.0029
#> (NM) 162: f = 8.04753e-06 at 1.00132  1.0029
#> (NM) 163: f = 8.04753e-06 at 1.00132  1.0029
#> (NM) 164: f = 2.30719e-06 at 1.00024 1.00033
#> (NM) 165: f = 2.30719e-06 at 1.00024 1.00033
#> (NM) 166: f = 2.02763e-06 at 0.998617 0.997203
#> (NM) 167: f = 2.02763e-06 at 0.998617 0.997203
#> (NM) 168: f = 7.9281e-07 at 1.00038 1.00083
#> (NM) 169: f = 7.9281e-07 at 1.00038 1.00083
#> (NM) 170: f = 4.11309e-07 at 0.999869 0.999675
#> (NM) 171: f = 4.11309e-07 at 0.999869 0.999675
#> (NM) 172: f = 4.11309e-07 at 0.999869 0.999675
#> (NM) 173: f = 4.11309e-07 at 0.999869 0.999675
#> (NM) 174: f = 4.77849e-08 at 0.999998  1.00002
#> (NM) 175: f = 4.77849e-08 at 0.999998  1.00002
#> (NM) 176: f = 4.77849e-08 at 0.999998  1.00002
#> (NM) 177: f = 4.77849e-08 at 0.999998  1.00002
#> (NM) 178: f = 4.77849e-08 at 0.999998  1.00002
#> (NM) 179: f = 4.77849e-08 at 0.999998  1.00002
#> (NM) 180: f = 4.64058e-08 at 0.999793 0.999593
#> (NM) 181: f = 4.64058e-08 at 0.999793 0.999593
#> (NM) 182: f = 9.81791e-09 at 0.999977 0.999944
#> (NM) 183: f = 9.81791e-09 at 0.999977 0.999944
#> (NM) 184: f = 9.81791e-09 at 0.999977 0.999944
#> (NM) 185: f = 9.81791e-09 at 0.999977 0.999944
#> (NM) 186: f = 9.81791e-09 at 0.999977 0.999944
#> (NM) 187: f = 2.51267e-09 at 1.00004 1.00008
#> (NM) 188: f = 2.51267e-09 at 1.00004 1.00008
#> (NM) 189: f = 9.81397e-10 at 0.999975 0.999953
#> (NM) 190: f = 9.81397e-10 at 0.999975 0.999953
#> (NM) 191: f = 9.81397e-10 at 0.999975 0.999953
#> (NM) 192: f = 9.81397e-10 at 0.999975 0.999953
#> (NM) 193: f = 9.81397e-10 at 0.999975 0.999953
#> (NM) 194: f = 4.39883e-10 at 1.00002 1.00004
#> (NM) 195: f = 4.39883e-10 at 1.00002 1.00004
#> (NM) 196: f = 4.39883e-10 at 1.00002 1.00004
#> (NM) 197: f = 2.55886e-10 at 0.999985 0.999969
#> (NM) 198: f = 2.55886e-10 at 0.999985 0.999969
#> (NM) 199: f = 1.94299e-10 at 0.999989 0.999979
#> (NM) 200: f = 1.94299e-10 at 0.999989 0.999979
#> (NM) 201: f = 1.84303e-11 at       1 1.00001
#> (NM) 202: f = 1.84303e-11 at       1 1.00001
#> (NM) 203: f = 1.84303e-11 at       1 1.00001
#> (NM) 204: f = 1.84303e-11 at       1 1.00001
#> (NM) 205: f = 1.84303e-11 at       1 1.00001
all.equal(o.2[-5],o.1[-5], tolerance=1e-15)# TRUE, unexpectedly
#> [1] "Component “control”: Component “iprint”: Mean relative difference: 19"    
#> [2] "Component “control”: Component “FtolAbs”: Mean relative difference: 1e+09"
#> [3] "Component “control”: Component “XtolRel”: Mean absolute difference: 1e-07"