Introduction
With the release of the R package varycoef on CRAN (see) we enable you to analyze your spatial data and in a simple, yet versatile way. The underlying idea are spatially varying coefficients, short SVC, which have been studied over the last decades. These models are based on the linear regression model and are therefore very easy to interpret. Due to their design, SVC models offer a high flexibility. These two properties allow for better understanding of the data, gaining new insights and, ultimately, better predictive performance.
In this blog post, we will show what SVC models are and how to define them. Afterwards, we give a short and illustrative example on how to apply these tools in varycoef using the well known data set meuse from the package sp.
Disclaimer This analysis is meant to be a show case of what is possible using the package varycoef, not to write the most rigorous statistical analysis of a data set.
But first, let us start with the set up and let me show you where to find help:
There is also a version of varycoef on Github from which you can get the latest devel version.
Preliminaries
Before we start, we want to give some prerequisites in order to understand the analysis below. Beside a classical linear regression model, we require the knowledge of geostatistics and tools thereof, like:
- what is the advantage of geostatistics?
- two-dimensional Gaussian processes, i.e. Gaussian random fields
- covariance functions and how the range and variance parameter influence them
- maximum likelihood estimation
What are Spatially Varying Coefficient Models?
SVC models are a generalization of the classical linear regression model. In matrix notation with a response vector \(\mathbf y\), a data matrix \(\mathbf X\), and a vector \(\boldsymbol \varepsilon\) containing the errors, a linear regression model is defined as
\[\mathbf y = \mathbf X \boldsymbol \beta + \boldsymbol \varepsilon.\]
In geostatistics, the error term usually is called the nugget. SVC models are introduced through a random effect \(\boldsymbol \eta(s)\) that is depending on the location \(s\). The underlying model takes the form
\[\mathbf y = \mathbf X \boldsymbol \beta + \mathbf W \boldsymbol \eta (s) + \boldsymbol \varepsilon\] where \(\mathbf W\) parses the covariates to the random effect \(\boldsymbol \eta(s)\). Specifically, each SVC is defined by a Gaussian process, more specifically a Gaussian random field (GRF), with mean zero and some stationary covariance function. This function usually depends on some marginal variance (sill) and a range. An example of 3 SVC, i.e. 3 GRFs, and the nugget \(\boldsymbol \varepsilon\) on a regular grid is given in the figure below.
## var scale
## 1 0.1 0.3
## 2 0.2 0.1
## 3 0.3 0.2
Just by looking at the figure, one can see that
- SVC_1, SVC_2 and SVC_3 have some spatial clustering…
- …, whereas the nugget does not.
- The largest variance can be observed in SVC_2 and SVC_3.
- The largest spatial clusters are in SVC_1.
- They all vary around 0, which is expected as they are all zero mean GRF.
Goal of SVC Modelling
Given the response \(\mathbf y\), the data matrices \(\mathbf X\) and \(\mathbf W\), we want to estimate the coefficients \(\boldsymbol \beta\) as we would with a linear regression model, and further, try to understand the properties of \(\boldsymbol \eta(s)\). Another important aspect is that we are able to predict \(\boldsymbol \eta(s')\) for some new locations \(s'\). This is what we are about to do in the following sections.
First Models
Given the covariates, we continue to fit the data with different models and corresponding methods. Hence, we work our way up starting with a linear regression and geostatistical model.
Linear Regression Model
We start with a linear regression model (LM) and ordinary least squares (OLS).
##
## Call:
## lm(formula = l_cad ~ 1 + dist + lime + elev, data = meuse)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.5731 -0.4291 0.1752 0.5395 1.4191
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.75085 0.56767 8.369 3.63e-14 ***
## dist -2.04497 0.39846 -5.132 8.69e-07 ***
## lime1 0.57632 0.16454 3.503 0.000606 ***
## elev -0.47304 0.07087 -6.675 4.42e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.7685 on 151 degrees of freedom
## Multiple R-squared: 0.6141, Adjusted R-squared: 0.6064
## F-statistic: 80.09 on 3 and 151 DF, p-value: < 2.2e-16
The residual analysis shows:
The spatial distribution of the residuals is the following:
## cadmium copper lead zinc elev dist om ffreq soil lime landuse dist.m
## 1 11.7 85 299 1022 7.909 0.00135803 13.6 1 1 1 Ah 50
## 2 8.6 81 277 1141 6.983 0.01222430 14.0 1 1 1 Ah 30
## 3 6.5 68 199 640 7.800 0.10302900 13.0 1 1 1 Ah 150
## 4 2.6 81 116 257 7.655 0.19009400 8.0 1 2 0 Ga 270
## 5 2.8 48 117 269 7.480 0.27709000 8.7 1 2 0 Ah 380
## 6 3.0 61 137 281 7.791 0.36406700 7.8 1 2 0 Ga 470
## l_cad LM_res
## 1 2.4595888 0.8764648
## 2 2.1517622 0.1528248
## 3 1.8718022 0.4450312
## 4 0.9555114 0.2145201
## 5 1.0296194 0.3837506
## 6 1.0986123 0.7777244
One can observe that there is a spatial clustering of the residuals. This motivates us to use geostatistics.
Geostatistical Model
The classical geostatistical model is a good reference and starting point for SVC models. Geostatistical models using a GRF are a special case of SVC models. More specifically, we want to model a single random effect and the data matrix \(\mathbf W\) only contains the intercept, such that the SVC model simplifies to the known geostatistical model
\[\mathbf y = \mathbf X \boldsymbol \beta + \boldsymbol \eta (s) + \boldsymbol \varepsilon.\]
Usually, we start by computing the empirical (semi-) variogramm and estimating a suitable model. Trying the exponential covariance function, we get the following:
## model psill range
## 1 Nug 0.2343484 0.000
## 2 Exp 0.3845267 247.618
The fit looks reasonable and the estimated values are (literally) good starting points for our methodology coming up in the next section. So we can do ordinary kriging on the residuals.
## [using ordinary kriging]
This seems to fit the residuals pretty good.
The SVC Model
Now we want to see how the SVC model performs. We will use the same covariates. However, as for the locations, we will transform the Easting and Northing given in meters to kilometers. This is due to computational stability.
Our SVC model describes the data as follows: The fixed effects are the same as in the LM. For the random effects, we consider an intercept (as a geostatistical model would) as well as SVCs for the covariates dist and lime. The reason is to show that our methodology also applies to SVC models where we assume that some coefficients are spatially constant, i.e. we do not associate an SVC. In our case, this is the case for the covariate elev.
The methodology we use is maximum likelihood estimation (MLE). It is implemented as a numerical optimization of the likelihood function. More specifically, we optimize over the covariance parameters associated to the SVC as well as the error variance. The optimization is therefore performed on the profile likelihood. As the MLE is a numerical optimization, we need starting values. Here, the parameters estimated in the geostatistical model come in handy and we will use them across all SVC.
## [1] 0.2476180 0.3845267 0.2476180 0.3845267 0.2476180 0.3845267 0.2343484
Now, we can start the MLE:
##
## Call:
## SVC_mle with 4 fixed effect(s) and 3 SVC(s)
## using 155 observations at 155 different locations.
##
## Residuals:
## Min. 1st Qu. Median 3rd Qu. Max.
## -1.78800 -0.23599 0.01779 0.32350 0.92382
##
## Residual standard error (nugget effect): 0.5639
## Multiple R-squared: 0.8488
##
##
## Coefficients of fixed effects:
## (Intercept) dist lime1 elev
## 5.6530 -2.3149 0.5731 -0.5712
##
##
## Covariance parameters of the SVC(s):
## range variance
## (Intercept) 0.5862 0.1623
## dist 0.2752 1.2984
## lime1 0.4796 0.0723
##
## The covariance parameters were estimated for the GRFs using
## exp. covariance functions. No covariance tapering applied.
##
##
## MLE:
## The MLE terminated after 39 function evaluations with convergence code 0
## (0 meaning that the optimization was succesful).
## The final profile likelihood value for 7 parameters is -165.3.
If we want to make some kind of predictions, we will have to transform the locations, as we did with the initial meuse data.
## SVC_1 SVC_2 SVC_3 loc_x loc_y
## 1 0.2006837 0.09851442 0.04253028 181.18 333.74
## 2 0.2170734 0.11421993 0.04772871 181.14 333.70
## 3 0.2134029 0.11441976 0.04517714 181.18 333.70
## 4 0.2083308 0.11341102 0.04229306 181.22 333.70
## 5 0.2340341 0.13136961 0.05339666 181.10 333.66
## 6 0.2310743 0.13331480 0.05066926 181.14 333.66
For now, please ignore the black triangles and squares.
In the upper left panel, one can see the GRF estimated and predicted by the classical geostatistical model. This can be compared to the upper right panel, i.e. the SVC associated with the intercept. In the bottom row we have the other two SVCs for dist and lime. I leave the comparison and interpretation of the results to the reader. What I want to point out is that the GRFs representing the SVCs are all inherently different. For instance, the ranges and variances differ substantially across all 3, as one can observe in the summary output. Further, as they all are zero mean Gaussian processes, the SVC are to be interpreted as deviations from the estimated means, which are given in the summary output above, or can be retrieved by:
## (Intercept) dist lime1 elev
## 5.6530002 -2.3149459 0.5731168 -0.5711995
So in the example above, this translates to the following. We have, say, two points of interest (POI) where we would like to receive the exact prediction equation for cadmium. These POIs are indicated with a black triangles and squares. We can look up the SVC estimates at thods locations to get the following two regression equations.
## Intercept dist lime
## 235 0.005430712 -0.04569925 0.01549149
## (Intercept) dist lime1 elev
## 5.6584310 -2.3606451 0.5886083 -0.5711995
## Intercept dist lime
## 2016 -0.4944464 -0.5462941 -0.1554102
## (Intercept) dist lime1 elev
## 5.1585538 -2.8612399 0.4177066 -0.5711995
So, our two equations would be:
\[ y(s_{POI1}) = 5.658 + (-2.361) \cdot dist + 0.589 \cdot lime1 + (-0.571) \cdot elev\]
\[ y(s_{POI2}) = 5.159 + (-2.861) \cdot dist + 0.418 \cdot lime1 + (-0.571) \cdot elev\]
The coefficients are indeed different except for elev, as we did not associate an SVC in the model to it. For all other coefficients, we see that the estimated marginal effects of the covariates change over space. This is a very nice example why SVC models are easy to interpret. For a fixed location, they are linear regression models. As a reference, we give the regression equation as estimated with OLS:
\[ y = 4.751 + (-2.045) \cdot dist + 0.576 \cdot lime1 + (-0.473) \cdot elev.\]
