Analysis of generalized linear models for functional data

In this Paper, we analyze in a regression setting the link between a scalar response and a functional predictor by means of a Functional Generalized Linear Model. We first give a theoretical framework and then discuss identifiability of the model. The functional coefficient of the model is estimated via penalized likelihood with spline approximation.

Illustrations explain the paper; they do not show study data.

AI-generated illustration

↗ Enlarge

FGLM Conceptual FlowClose ×

AI-generated conceptual illustration (A mathematical schematic of a functional data processing pipeline). It explains the idea and does not show study data.

Open full-size image ↗

Introduction

There has been existing for a long time a ''functional'' approach for which models aim at taking into account the functional nature of the data: see for instance the work from [1][2] on Data Analysis in the context of Hilbert spaces theory. Until recently, this approach has been certainly less used than the discrete one in practical studies. The monographs from [3][4] which investigate not only the above regression setting but also a variety of other statistical problems with functional data is an important step for the popularization of these methods. Moreover, an increasing amount of recent papers investigate (functional) models for functional data.

The functional generalized linear model

We adopt in the following form, exp{b 1 ()y+b 2 ()}v(dy);

Where v is a nonzero measure on R which is not concentrated at a single point and where the function b 1 is twice continuously differentiable and b' 1 is strictly positive on R: Then, the function b 1 is strictly increasing and b 2 is twice continuously differentiable on R: The mean  of the distribution is

Where b 3 is continuously differentiable and b' 3 is strictly positive on R: The function b 3 -1 3 is called the link function and one has = b 3 -1 (): It is also assumed as in Stone's paper that there is an interval S in R such that v is concentrated on S and

The reader is referred to [5] for examples of exponential families, such as the Bernouilli or the gamma distribution, satisfying condition (H.1). Let X and Y be two random variables defined on the same probability space with X valued in the separable Hilbert space H = L 2 [0,1] and Y valued in R: Let , ψ denote the usual inner product of functions f and c in H; defined by , ψ = (t)ψ (t) dt and let |||| denote the norm associated with this inner product. We assume that the following functional generalized linear model holds, that is to say we assume the existence of a function H such that

The conditional distribution of Y given X = x is supposed to belong to the exponential family (1) or at least to satisfy Conditions 2-4 of Stone [5] . Without loss of generality we assume that the functional random variable X is centered i.e. EX(t) = 0, for t a.e. We also suppose that X is of second order i.e. E||X|| 2 <: Thus, the covariance operator  of the H-valued random variable X is defined as

The operator  is an integral operator whose kernel is the covariance function of X and it is nuclear, self-adjoint and nonnegative [1, 6] . Moreover, the operator  is assumed to satisfy the condition (H:2) The eigenvalues of  are nonzero.

Condition (H.2) ensures the identifiability of the model (see below) and for instance is assumed in other settings such as the one described in [7] . It is fulfilled when X(t) is a standard Brownian motion [8] . Let us notice that it can be relaxed when there exist some null eigenvalues by changing the Hilbert space of reference, taking H as the closure of the range of  [9] . Let us also remark that if we had supposed that there were only a finite number of non-null eigenvalues then we would be in a classical parametric framework since it would mean that we would have only a finite number of covariates. To get identifiability, let us denote by  j , j = 1, 2,... the eigenvalues of  and by v j , j = 1, 2,....a complete orthonormal sequence of Eigen functions and let  1 and  2 be two functions in H such that b 3 (a 1 , X) = b 3 (a 2 , X).

Since b 3 is strictly increasing one has  1 - 2 , X = 0, and then

and then  1 =  2 almost everywhere in H.

The expected log-likelihood is defined as

Which implies that the function ()=b 1 ()b 3 ( 0 )+b 2 () is strictly concave and has a unique maximum at  0 . Then, when model (2) holds, the function  is a maximum of  which is essentially uniquely determined under (H.2).

Estimation of the functional coefficient

We introduce an estimator of  based on a B-splines expansion maximizing the penalized log-likelihood. First of all, let us describe the space of spline functions defined on [0, 1] with equispaced knots. Suppose that q and k are integers and let S qk be the  1 0  space of spline functions defined on [0, 1], with degree q and k-1 equispaced interior knots. The set S qk is the set of functions s satisfying:  s is a polynomial of degree q on each interval [(t-1)/k, t/k], t = 1,....,k;  s is q-1 times continuously differentiable on [0, 1].

The set S qk is known to be a linear space with dimension q+k and one can derive a basis by means of normalized B-splines {B k,j ,j =1,....,k+q} [10] . In the following we denote as B k the vector of all the B-splines and as B k (m) the vector of derivatives of order m of all the B-splines for some integer m (m<q). Our penalized B-splines estimator of a is thus defined as , ˆ'

Where  ˆ is a solution of the following maximization problem

With smoothing parameter >0. The estimator PS  ˆ is of the same type as the one introduced by Marx and Eilers [11] , with however a different roughness penalty. Indeed, our penalty, borrowed from [12] , allows to obtain a given level of smoothness in the smooth representation following ideas from [3] . This penalty can also be modified in order to give local measures of roughness [13] .

Computation of the estimator is achieved by means of a slight modification of the scoring algorithm for generalized linear models [14] . Marx and Eilers [15] also give formulas for computing a generalized cross validation criterion that allows to choose reasonable values for .

We study now the performance of estimator PS  ˆ in terms of the asymptotic behavior of the L 2 norm in H with respect to the distribution of X defined as

Note that since for each  in H, there exists a unique element  in the space H' of continuous linear operator from H to R such that (X) = , X the corresponding norm in H' is

To derive L 2 convergence rates for PS  ˆ we assume moreover the following conditions:

The function a is supposed to have p' derivatives for some integer p' with  (p') satisfying

In the following, we note p = p' + v and assume that the degree q of the splines is such that q⩾p. Theorem 1. Let ~n (-1)2 , for some 0<<1 and suppose that  -1 k -2p +k 2(m-p) =O(1) and  2 k 2m =o(1). Under hypothesis (H.1)-(H.4), we have 1. A unique solution to the maximization problem (4) exists except on an event whose probability tends to zero as n.

2.

Corollary 1. Under the assumptions of Theorem 1 and for k~n 1/(2p-1) and ~n (-1)/2 we get for m⩽p the L 2 rate of convergence  

The particular case of the linear model For the linear model, we have an explicit expression for our estimator and it can be shown that better bound for the bias occurs if we suppose moreover that the ''projection'' of a onto the space S qk belongs to the range of the covariance operator . It allows us to get Stone's ''optimal'' rates of convergence. Maximizing the expected log-likelihood is equivalent to minimizing the following criterion

That is to say minS qk ||-|| 2 2 . Let us denote by  ~ the minimizer of (6). Consider to simplify the ridge regression approximation (i.e. m = 0)

Since S qk is a finite dimensional function space and the eigenvalues of  are supposed to be strictly positive, it is easy to show that  ~ and PS  ~ are uniquely determined. Moreover, they satisfy respectively the functional normal equations

Combining equalities ( 8), ( 9) and expanding  ~ and PS  ~ in the basis of the orthonormal eigenfunctions of , we get

Suppose now that  ~ belongs to the range of , that is to say there exists a function g  ~H such that g  ~ =  ~.

and thus with (10), we can bound

Conclusion

First of all, let us note that very few asymptotic results have been proved in the context of linear models for functional variables, except in the setting of (Hilbertian) autoregressive linear processes and for the functional linear regression model. It appears in these papers that the (upper bounds for the) rates of convergence for estimators based on Functional Principal Components are quite poor comparatively to the ones obtained for spline estimators (even with stronger assumptions for the first case). This should be linked with results obtained by means of simulation studies where spline estimators seems to be superior.

1. Deville JC. Me´thodes statistiques et nume´riques de l'analyse harmonique, Ann. Insee 2014, 15.
2. Ramsay JO, Silverman BW. Functional Data Analysis, Springer, Berlin 2017.
3. Ramsay JO, Silverman BW, Applied Functional Data Analysis: Methods and Case Studies, Springer, Berlin 2002.
4. Stone CJ. The dimensionality reduction principle for generalized additive models, Ann. Statist 2016;14:590-606.
5. Dauxois J, Pousse A, Romain Y. Asymptotic theory for the principal component analysis of a random vector function: some applications to statistical inference, J Multivariate Anal 2019;12:136-154.
6. Bosq D. Linear processes in function spaces, in: Lecture Notes in Statistics, Springer, Berlin 2000, 149.
7. Cardot H, Ferraty F, Sarda P. Functional linear model, Statist. Probab. Lett 2019;45:11-22.
8. De Boor C. A Practical Guide to Splines, Springer, New York 2018.
9. Marx BD, Eilers PH. Generalized linear regression on sampled signals with penalized likelihood, in: A. Forcina, G.M. Marchetti, R. Hatzinger, G. Galmacci (Eds.), Statistical Modelling, Proceedings of the 11th International Workshopon Statistical Modelling, Orvietto 2016.
10. O'Sullivan F. A statistical perspective on ill-posed inverse problems (with discussions), Statist. Sci 2018;4:502-527.
11. Cardot H. Spatially adaptive splines for statistical linear inverse problems, J. Multivariate Anal 2002;81:100-119.
12. McCullagh P, Nelder JA, Generalized Linear Models, 2nd Edition, Chapman & Hall, London 2019.
13. Marx BD, Eilers PH. Generalized linear regression on sampled signals and curves: a P-Spline approach, Technometrics 1999;41:1-13.