We investigate the estimation of the derivatives of a regression function in the nonparametric regression model with random design. New wavelet estimators are developed. Their performances are evaluated via the mean integrated squared error. Fast rates of convergence are obtained for a wide class of unknown functions. 1. Introduction We consider the nonparametric regression model with random design described as follows. Let be random variables defined on a probability space , where are i.i.d. random variables such that and , are i.i.d. random variables with common density , and is an unknown regression function. It is assumed that and are independent for any . We aim to estimate , that is, the th derivative of , for any integer , from . In the literature, various estimation methods have been proposed and studied. The main ones are the kernel methods (see, e.g., [1–5]), the smoothing splines, and local polynomial methods (see, e.g., [6–9]). The object of this note is to introduce new efficient estimators based on wavelet methods. Contrary to the others, they have the benefit of enjoying local adaptivity against discontinuities thanks to the use of a multiresolution analysis. Reviews on wavelet methods can be found in, for example, Antoniadis [10], H？rdle et al. [11], and Vidakovic [12]. To the best of our knowledge, only Cai [13] and Petsa and Sapatinas [14] have proposed wavelet estimators for from (1) but defined with a deterministic equidistant design; that is, . The consideration of a random design complicates significantly the problem and no wavelet estimators exist in this case. This motivates our study. In the first part, assuming that is known, we propose two wavelet estimators: the first one is linear nonadaptive and the second one nonlinear adaptive. Both use the approach of Prakasa Rao [15] initially developed in the context of the density estimation problem. Then we determine their rates of convergence by considering the mean integrated squared error (MISE) and assuming that belongs to Besov balls. In a second part, we develop a linear wavelet estimator in the case where is unknown. It is derived from the one introduced by Pensky and Vidakovic [16] considering the estimation of from (1). We evaluate its rate of convergence again under the MISE over Besov balls. The obtained rates of convergence are similar to those attained by wavelet estimators for the derivatives of a density (see, e.g., [15, 17, 18]). The organization of this note is as follows. The next section describes some basics on wavelets and Besov balls. Our estimators and their
References
[1]
T. Gasser and H.-G. Müller, “Estimating regression functions and their derivatives by the kernel method,” Scandinavian Journal of Statistics, Theory and Applications, vol. 11, no. 3, pp. 171–185, 1984.
[2]
W. H？rdle and T. Gasser, “On robust kernel estimation of derivatives of regression functions,” Scandinavian Journal of Statistics, vol. 12, no. 3, pp. 233–240, 1985.
[3]
Y. P. Mack and H.-G. Müller, “Derivative estimation in nonparametric regression with random predictor variable,” Sankhya: The Indian Journal of Statistics, Series A, vol. 51, no. 1, pp. 59–72, 1989.
[4]
D. Ruppert and M. P. Wand, “Multivariate locally weighted least squares regression,” The Annals of Statistics, vol. 22, no. 3, pp. 1346–1370, 1994.
[5]
M. P. Wand and M. C. Jones, Kernel Smoothing, Chapman and Hall, London, UK, 1995.
[6]
C. Stone, “Additive regression and other nonparametric models,” The Annals of Statistics, vol. 13, no. 2, pp. 689–705, 1985.
[7]
G. Wahba and Y. H. Wang, “When is the optimal regularization parameter insensitive to the choice of the loss function?” Communications in Statistics: Theory and Methods, vol. 19, no. 5, pp. 1685–1700, 1990.
[8]
S. Zhou and D. A. Wolfe, “On derivative estimation in spline regression,” Statistica Sinica, vol. 10, no. 1, pp. 93–108, 2000.
[9]
R. Jarrow, D. Ruppert, and Y. Yu, “Estimating the interest rate term structure of corporate debt with a semiparametric penalized spline model,” Journal of the American Statistical Association, vol. 99, no. 465, pp. 57–66, 2004.
[10]
A. Antoniadis, “Wavelets in statistics: a review,” Journal of the Italian Statistical Society Series B, vol. 6, no. 2, pp. 97–130, 1997.
[11]
W. H？rdle, G. Kerkyacharian, D. Picard, and A. Tsybakov, Wavelets, Approximation, and Statistical Applications, vol. 129 of Lecture Notes in Statistics, Springer, New York, NY, USA, 1998.
[12]
B. Vidakovic, Statistical Modeling by Wavelets, John Wiley & Sons, New York, NY, USA, 1999.
[13]
T. Cai, “On adaptive wavelet estimation of a derivative and other related linear inverse problems,” Journal of Statistical Planning and Inference, vol. 108, no. 1-2, pp. 329–349, 2002.
[14]
A. Petsa and T. Sapatinas, “On the estimation of the function and its derivatives in nonparametric regression: a bayesian testimation approach,” Sankhya, Series A, vol. 73, no. 2, pp. 231–244, 2011.
[15]
B. L. S. Prakasa Rao, “Nonparametric estimation of the derivatives of a density by the method of wavelets,” Bulletin of Informatics and Cybernetics, vol. 28, no. 1, pp. 91–100, 1996.
[16]
M. Pensky and B. Vidakovic, “On non-equally spaced wavelet regression,” Annals of the Institute of Statistical Mathematics, vol. 53, no. 4, pp. 681–690, 2001.
[17]
Y. P. Chaubey, H. Doosti, and B. L. S. P. Rao, “Wavelet based estimation of the derivatives of a density with associated variables,” International Journal of Pure and Applied Mathematics, vol. 27, no. 1, pp. 97–106, 2006.
[18]
Y. P. Chaubey, H. Doosti, and B. L. S. P. Rao, “Wavelet based estimation of the derivatives of a density for a negatively associated process,” Journal of Statistical Theory and Practice, vol. 2, no. 3, pp. 453–463, 2008.
[19]
A. Cohen, I. Daubechies, and P. Vial, “Wavelets on the interval and fast wavelet transforms,” Applied and Computational Harmonic Analysis, vol. 1, no. 1, pp. 54–81, 1993.
[20]
I. Daubechies, Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics, 1992.
[21]
S. Mallat, A Wavelet Tour of Signal Processing, The Sparse Way, with Contributions from Gabriel Peyré, Elsevier, Academic Press, Amsterdam, The Netherlands, 3rd edition, 2009.
[22]
Y. Meyer, Wavelets and Operators, Cambridge University Press, Cambridge, UK, 1992.
[23]
C. Chesneau, “Regression with random design: a minimax study,” Statistics and Probability Letters, vol. 77, no. 1, pp. 40–53, 2007.
[24]
B. Delyon and A. Juditsky, “On minimax wavelet estimators,” Applied and Computational Harmonic Analysis, vol. 3, no. 3, pp. 215–228, 1996.
[25]
Y. P. Chaubey, C. Chesneau, and H. Doosti, “Adaptive wavelet estimation of a density from mixtures under multiplicative censoring,” http://hal.archives-ouvertes.fr/hal-00918069.
[26]
D. L. Donoho and J. M. Johnstone, “Ideal spatial adaptation by wavelet shrinkage,” Biometrika, vol. 81, no. 3, pp. 425–455, 1994.
[27]
D. L. Donoho and I. M. Johnstone, “Adapting to unknown smoothness via wavelet shrinkage,” Journal of the American Statistical Association, vol. 90, no. 432, pp. 1200–1224, 1995.
[28]
D. L. Donoho, I. M. Johnstone, G. Kerkyacharian, and D. Picard, “Wavelet shrinkage: asymptopia?” Journal of the Royal Statistical Society, Series B, vol. 57, no. 2, pp. 301–369, 1995.
[29]
D. L. Donoho, I. M. Johnstone, G. Kerkyacharian, and D. Picard, “Density estimation by wavelet thresholding,” Annals of Statistics, vol. 24, no. 2, pp. 508–539, 1996.
[30]
G. Kerkyacharian and D. Picard, “Regression in random design and warped wavelets,” Bernoulli, vol. 10, no. 6, pp. 1053–1105, 2004.
[31]
S. Ga？ffas, “Sharp estimation in sup norm with random design,” Statistics and Probability Letters, vol. 77, no. 8, pp. 782–794, 2007.
[32]
A. Antoniadis, M. Pensky, and T. Sapatinas, “Nonparametric regression estimation based on spatially inhomogeneous data: minimax global convergence rates and adaptivity,” ESAIM: Probability and Statistics, vol. 18, pp. 1–41, 2014.
