maxlogL estimates parameters of probability functions tby Maximum Likelihood. The variance-covariance matrix is computed from Fisher information matrix, which is obtained by means of the Inverse Hessian matrix of estimators:
\[\begin{equation} Var(\hat{\boldsymbol{\theta}}) = I^{-1}(\hat{\boldsymbol{\theta}}) = C(\hat{\boldsymbol{\theta}}), \end{equation}\]
where \(I(\hat{\boldsymbol{\theta}})\) is the Fisher Information Matrix. Hence, the standard errors can be calculated as the square root of the diagonal elements of matrix \(C\), as follows:
\[\begin{equation} SE(\hat{\boldsymbol{\theta}}) = \sqrt{C_{jj}(\hat{\boldsymbol{\theta}})}, \end{equation}\]
To install the package, type the following commands:
if (!require('devtools')) install.packages('devtools')
devtools::install_github('Jaimemosg/EstimationTools', force = TRUE)The Hessian matrix is computed in the following way:
optim, with option hessian = TRUE in maxlogL function.hessian function from numDeriv package.Additionally, EstimationTools allows implementation of bootstrap for standard error estimation, even if the Hessian computation does not fail. The following chunks illustrates this feature:
library(EstimationTools)
x <- rnorm(n = 10000, mean = 160, sd = 6)
theta_1 <- maxlogL(x = x, dist = 'dnorm', control = list(trace = 1),
link = list(over = "sd", fun = "log_link"),
fixed = list(mean = 160))
#> 0: 43659.276: 1.00000
#> 1: 32501.025: 2.00000
#> 2: 32276.138: 1.92023
#> 3: 32138.153: 1.76473
#> 4: 32129.558: 1.79781
#> 5: 32129.413: 1.79411
#> 6: 32129.413: 1.79400
#> 7: 32129.413: 1.79400
summary(theta_1)
#> ---------------------------------------------------------------
#> Optimization routine: nlminb
#> Standard Error calculation: Hessian from optim
#> ---------------------------------------------------------------
#> AIC BIC
#> 64258.83 64258.83
#> ---------------------------------------------------------------
#> Estimate Std. Error
#> sd 6.0135 0.0425
#> -----
## Hessian
print(theta_1$fit$hessian)
#> [,1]
#> [1,] 553.0687
## Standard errors
print(theta_1$outputs$StdE)
#> [1] 0.0425
print(theta_1$outputs$StdE_Method)
#> [1] "Hessian from optim"Note that Hessian matrix was computed with no issues. However, the user can implement bootstrap alogorithm available in summary function:
summary(theta_1, Boot_Std_Err = TRUE, R = 1000)
#>
#> ...Bootstrap computation of Standard Error. Please, wait a few minutes...
#>
#> ---------------------------------------------------------------
#> Optimization routine: nlminb
#> Standard Error calculation: Bootstrap
#> ---------------------------------------------------------------
#> AIC BIC
#> 64258.83 64258.83
#> ---------------------------------------------------------------
#> Estimate Std. Error
#> sd 6.0135 0.0436
#> -----
## Hessian
print(theta_1$fit$hessian)
#> [,1]
#> [1,] 553.0687
## Standard errors
print(theta_1$outputs$StdE)
#> [1] 0.04357992
print(theta_1$outputs$StdE_Method)
#> [1] "Bootstrap"Notice that Standard Errors calculated with optim (\(0.0425\)) and those calculated with bootstrap implementation (\(0.04358\)) are approximately equals, but no identical.