---
title: "Analysis of the Pied Flycatcher Data: Random Slopes"
author: "Simon Bonner"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Pied Flycatchers -- Part II}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r,echo=FALSE}
runModels <- FALSE # If true rerun models. OTW, reload previous output.
```

This vignette continues the example of the pied flycatchers discussed in the vignette "pied-flycatchers-1" by adding a random slope to the fixed effects model. I highly recommend that you work through the other vignette first.

# Library

First you have to load the package:
```{r}
## Load package
library(dalmatian)
```

# Raw data

The raw data for this example is provided in the package and can be accessed with:
```{r}
## Load pied flycatcher data
data(pied_flycatchers_1)
```

As before we will consider load as a response variable rounded to the nearest whole number and define the lower and upper bounds:
```{r}
## Create variables bounding the true load
pfdata$lower=ifelse(pfdata$load==0,log(.001),log(pfdata$load-.049))
pfdata$upper=log(pfdata$load+.05)
```

# Model

The model we consider will be the similar to the random effects model in "pied-flycatchers-1". We will modify this model by including a random, individual slope for the effect of $log(IVI)$ as well as the random intercept in the mean model. The new model for the mean will be:
$$\mu=\beta_0 + \beta_1 \mathrm{log(IVI)_{ij}} + \beta_2 \mathrm{broodsize}_i + \beta_3 \mathrm{sex}_i + \epsilon_{1i} + \epsilon_{2i} \mathrm{log(IVI)_{ij}}$$
For convenience, we will also simplify the model of the variance by assuming that the variance is constant across all observations. This can be done by setting:
$$\log(\sigma)=\psi_0$$.

The objects defining the new mean and variance components are specified as:

```{r}
# Random component of mean
mymean=list(fixed=list(name="alpha",
                formula=~ log(IVI) + broodsize + sex,
                priors=list(c("dnorm",0,.001))),
    random=list(name="epsilon",formula=~-1 + indidx + indidx:log(IVI)))


# Random component of variance
myvar=list(fixed=list(name="psi",
               link="log",
               formula=~1,
               priors=list(c("dnorm",0,.001))))

```

## Running the Model with dalmatian

The new model can now be run using `dalmatian` in exactly the same way as above. However, we will make one change. In order to shorten the convergence time, we will base initial values on the results of the fixed effects model and then provide these as input. In particular, we will define initial values for the fixed effects of both the mean and variance components by taking the values from the final iterations of each of the chains run for the fixed effects model. For convenience, we set the random effects variances equal to 1 in all three chains. Note that the chains have been run for a relatively small number of iterations and heavily subsampled to satisfy the size requirements for packages on \texttt{CRAN}. Neither of these is recommended in practice.


```{r}
## Set working directory
## By default uses a system temp directory. You probably want to change this.
workingDir <- tempdir()

## Define list of arguments for jags.model()
jm.args <- list(file=file.path(workingDir,"pied_flycatcher_3_jags.R"),n.adapt=1000)

## Define list of arguments for coda.samples()
cs.args <- list(n.iter=1000)

## Run the model using dalmatian
if(runModels){
  pfresults3 <- dalmatian(df=pfdata,
                          mean.model=mymean,
                          variance.model=myvar,
                          jags.model.args=jm.args,
                          coda.samples.args=cs.args,
                          rounding=TRUE,
                          lower="lower",
                          upper="upper",
                          debug=FALSE)
  
  save(pfresults2,"pfresults3.RData")
}
if(!runModels){
  ## Load output from previously run chain
  load(system.file("Pied_Flycatchers_2","pfresults3.RData",package="dalmatian"))
}

```

## Results
Once again, we can use the wrapper functions provided with `dalmatian` to conduct convergence diagnostics and compute summary statistics for the posterior distribution from which we can make inference. 

1. Convergence diagnostics
```{r}
## Compute convergence diagnostics
pfconvergence2 <- convergence(pfresults3,raftery=list(r=.01))

## Gelman-Rubin diagnostics
pfconvergence2$gelman

## Raftery diagnostics
pfconvergence2$raftery

## Effective sample size
pfconvergence2$effectiveSize
```

2. Traceplots
```{r,fig.width=6,fig.align="center"}
## Generate traceplots
pftraceplots2 <- traceplots(pfresults3,plot = FALSE)

## Fixed effects for mean
pftraceplots2$meanFixed

## Fixed effects for variance
pftraceplots2$varianceFixed

## Random effects variances for mean
pftraceplots2$meanRandom

## Random effects variances for variances
pftraceplots2$varianceRandom

```

3. Numerical summaries
```{r}
## Compute numerical summaries
summary(pfresults3)
```

4. Graphical summaries
```{r,fig.width=6,fig.align="center"}
## Generate caterpillar
pfcaterpillar2 <- caterpillar(pfresults3,plot = FALSE)

## Fixed effects for mean
pfcaterpillar2$meanFixed

## Fixed effects for variance
pfcaterpillar2$varianceFixed
```

The convergence diagnostics and traceplots suggest that the chains should be run for longer. In particular, the effective sample size for the variance of the random slopes is very small, but we will proceed with inference for illustration. In the numerical summaries we see that the estimated variance of the random slopes is very small, .02 with 95% credible interval (.00,.04). This suggests that there is very little inter-individual variation in the effect of `log(IVI)`.

Estimates of the individual random effects (predictions for frequentists) can be obtained from the function `ranef()`. This function computes posterior summary statistics for the individual random effects. In this case, the random effects for the mean include both the intercept and the effect of `log(IVI)`. 

The following code computes the summary statistics and plots the predicted values of the random slopes with HPD 95% confidence intervals. 
```{r,fig.width=6,fig.align="center"}
## Compute summary statistics for random effects
ranef2 <- ranef(pfresults3)

## Load ggplot2
library(ggplot2)

## Identify number of individuals
nind <- nlevels(pfdata$indidx)

## Plot predicted random slopes
ggplot(data=as.data.frame(ranef2$mean[nind+(1:nind),]),aes(x=1:nind,y=Mean)) + 
  geom_point() + 
  geom_errorbar(aes(ymin=`Lower 95%`,ymax=`Upper 95%`)) +
  geom_abline(intercept=0,slope=0)
```
Note that the 95% crebile intervals for all of the values include 0. This is further evidence that the effect of `log(IVI)` does not vary significantly between the individuals.  

