Nonparametric Additive Modeling by Smoothing Splines: Robust Unbiased-Risk-Estimate Selector and a Nonisotropic-Smoothing Improvement
Requires a Wolfram Notebook System
Interact on desktop, mobile and cloud with the free Wolfram Player or other Wolfram Language products.
This Demonstration is dedicated to the memory of François de Crécy.
[more]
Contributed by: Didier A. Girard (November 2017)
(CNRS-LJK and Université Grenoble Alpes)
Open content licensed under CC BY-NC-SA
Snapshots
Details
Refer to Chapter 9.1.1 of [4] for a recent detailed presentation of the additive modeling of multivariate regression functions and the backfitting algorithm to implement it for a given value of .
The first-step estimates use a robustified selector of proposed in [5] (and evaluated in [3]). Possibly, the methods discussed in [6] could have been used as well.
In brief, the idea behind our proposed tuning formula (3) for is that the two smoothness energies occurring in should be roughly equal at the optimum, in the same way that the two terms of the risk (namely the squared bias and the variance) have a similar order when is well chosen. Precisely, the suggestion here is simply, in a second step, to choose so that:
where and , are the first-step estimate, that is, solution of where is given, as mentioned above, by the robust method. It is easy to see that the selected is then simply given by the formula (3).
It was François de Crécy, a dear friend and colleague over the past 25 years while he was a researcher at CEA (Commissariat à l'énergie atomique et aux énergies alternatives), who had suggested to me that for a more general multivariate additive model (i.e. the natural extension with explanatory variables, whereas in this Demonstration ), the dilation factors (those of ) could be very simply tuned by harmonizing, in a second step, all the smoothness energies in the penalization term (the one that generalizes the penalization term of ), each computed on the first-step estimate . Of course, the procedure could be iterated (i.e. a third step would determine new dilation factors by harmonizing the smoothness energies computed on the second-step estimate etc.), but such a surface estimator has not yet been evaluated.
References
[1] D. A. Girard, "Nonparametric Curve Estimation by Kernel Smoothers: Efficiency of Unbiased Risk Estimate and GCV Selectors," from the Wolfram Demonstrations Project—A Wolfram Web Resource. demonstrations.wolfram.com/NonparametricCurveEstimationByKernelSmoothersEfficiencyOfUnb.
[2] D. A. Girard, "Nonparametric Regression and Kernel Smoothing: Confidence Regions for the L2-Optimal Curve Estimate," from the Wolfram Demonstrations Project—A Wolfram Web Resource. demonstrations.wolfram.com/NonparametricRegressionAndKernelSmoothingConfidenceRegionsFo.
[3] D. A. Girard, "Nonparametric Curve Estimation by Smoothing Splines: Unbiased-Risk-Estimate Selector and Its Robust Version via Randomized Choices," from the Wolfram Demonstrations Project—A Wolfram Web Resource. demonstrations.wolfram.com/NonparametricCurveEstimationBySmoothingSplinesUnbiasedRiskEs.
[4] T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., New York: Springer, 2009.
[5] D. A. Girard, "Estimating the Accuracy of (Local) Cross-Validation via Randomised GCV Choices in Kernel or Smoothing Spline Regression," Journal of Nonparametric Statistics, 22(1), 2010 pp. 41–64. doi:10.1080/10485250903095820.
[6] M. A. Lukas, F. R. de Hoog and R. S. Anderssen, "Practical Use of Robust GCV and Modified GCV for Spline Smoothing," Computational Statistics, 31(1), 2016 pp. 269–289. doi:10.1007/s00180-015-0577-7.
Permanent Citation