<!--
%\VignetteEngine{knitr::knitr}
%\VignetteIndexEntry{LambertW Overview}
-->
```{r setup, include=FALSE}
library(knitr)
opts_chunk$set(cache = TRUE)
options(digits = 3)

```


```{r load-packages}
library(LambertW)
library(ggplot2)
library(reshape2)
library(data.table)
```


# Gigantic kurtosis

[Post here](http://stats.stackexchange.com/questions/34565/gigantic-kurtosis)

Have a look at [heavy-tail Lambert W x F](http://www.hindawi.com/journals/tswj/aa/909231/)
 or [skewed Lambert W x F ](http://projecteuclid.org/euclid.aoas/1318514301) distributions a try.

Related posts:

  * http://stats.stackexchange.com/questions/33115/whats-the-distribution-of-these-data/47917#47917
  * http://stats.stackexchange.com/questions/20445/how-to-transform-data-to-normality/189991#189991
  
One advantage over Cauchy or student-t distribution with fixed degrees of freedom is that the tail parameters can be estimated from the data -- so you can let the data decide what moments exist.  Moreover the Lambert W x F framework allows you to transform your data and remove skewness / heavy-tails.   Itt is important to note though that OLS does not require Normality of $y$ or $X$.  However, for your EDA it might be worthwhile.

Here is an example of Lambert W x Gaussian estimates applied to stock market data.

```{r eu-stock-markets}
ret <- diff(log(EuStockMarkets)) * 100
plot(ret)
```


The summary metrics of the returns are similar (not as extreme) as in OP's post.
```{r pairs-metrics}
data_metrics <- function(x) {
  c(mean = mean(x), sd = sd(x), min = min(x), max = max(x), 
    skewness = skewness(x), kurtosis = kurtosis(x))
}
ret.metrics <- t(apply(ret, 2, data_metrics))
ret.metrics
```

Most series show clearly non-Normal characteristics (strong skewness and/or large kurtosis).  Let's Gaussianize each series using a heavy tailed 
Lambert W x Gaussian distribution (= Tukey's h) using a methods of moments
estimator (`IGMM`).
```{r gaussianize-returns}
library(LambertW)
ret.gauss <- Gaussianize(ret, type = "h", method = "IGMM")
colnames(ret.gauss) <- gsub("\\.X", "", colnames(ret.gauss))

plot(ret.gauss)
```

The time series plots show much fewer tails and also more stable variation over time (not constant though).  Computing the metrics again on the Gaussianized time series yields:

```{r metrics-gaussianized}
ret.gauss.metrics <- t(apply(ret.gauss, 2, data_metrics))
ret.gauss.metrics
```

The `IGMM` algorithm achieved exactly what it was set forth to do: transform the data to have kurtosis equal to $3$. Interestingly, all time series now have negative skewness, which is in line with most financial time series literature.  Important to point out here that `Gaussianize()` operates only marginally, not jointly (analogously to `scale()`).


## Simple bivariate regression

To consider the effect of Gaussianization on OLS consider predicting "FTSE" return from "DAX" returns and vice versa.

```{r DAX-FTSE, fig.width = 16, fig.height = 8}
layout(matrix(1:2, ncol = 2, byrow = TRUE))
plot(ret[, "FTSE"], ret[, "DAX"])
# grid()
plot(ret.gauss[, "DAX"], ret.gauss[, "FTSE"])
# grid()

```
The left scatterplot of the original series shows that the strong outliers did not occur at the same days, but at different times in India and Europe; other than that it is not clear if the data cloud in the center supports no correlation or negative/positive dependency.  Since outliers strongly affect variance and correlation estimates, it is worthwhile to look at the dependency with heavy tails removed (right scatterplot).  Here the patterns are much more clear and the positive relation between India and Eastern Europe market becomes apparent.


```{r fit-models, include = TRUE, eval = FALSE}
# try these models on your own
mod <- lm(FTSE ~ DAX + SMI + CAC, data = ret)
mod.robust <- rlm(FTSE ~ DAX + SMI + CAC, data = ret)
mod.gauss <- lm(FTSE ~ DAX + SMI + CAC, data = ret.gauss)

summary(mod)
summary(mod.robust)
summary(mod.gauss)
```

## Granger causality

A Granger causality test based on a  $VAR(5)$ model (I use $p = 5$ to capture the week effect of daily trades) for "DAX" and "CAX" rejects "no Granger causality" for DAX --> CAC direction.
```{r vars-ret}
library(vars)
mod.vars <- vars::VAR(ret[, c("DAX", "CAC")], p = 6)
causality(mod.vars, "DAX")$Granger
causality(mod.vars, "CAC")$Granger

```

However, for the Gaussianized data the answer is different!  Here the test can *not* reject H0 that "INDIA does *not* Granger-cause EASTEU", but still rejects that "EASTEU does not Granger-cause INDIA".  So the Gaussianized data supports the hypothesis that European markets drive markets in India the following day.


```{r vars-ret-gauss}
mod.vars.gauss <- vars::VAR(ret.gauss[, c("DAX", "CAC")], p = 6)
causality(mod.vars.gauss, "DAX")$Granger
causality(mod.vars.gauss, "CAC")$Granger
```


It is not clear to me which one is the *right* answer (if any), but it's an interesting observation to make.  Needless to say that this entire Causality testing is contingent on the $VAR(6)$ being the correct model -- which it is most likely not; but I think it serves well for illustratiton.
