---
title: "Updating USGDPpresidents"
author: "Spencer Graves"
date: "4/6/2019"
output: html_document
vignette: >
  %\VignetteIndexEntry{Updating USGDPpresidents}
  %\VignetteKeyword{USGDPpresidents}
  %\VignetteEngine{knitr::rmarkdown}
  %\SweaveUTF8
  \usepackage[utf8](inputenc)
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

## Purpose of this document 

This document describes the process for updating [USGDPpresidents {Ecdat}](https://www.rdocumentation.org/packages/Ecdat/versions/0.3-1/topics/USGDPpresidents).  

## Is there new data?  

Start by checking the span of years in USGDPpresidents:  

```{r yrSpan}
library(Ecdat)
(rngYrs <- range(USGDPpresidents$Year))
```

Next download "GDP - US" and "CPI - US" from [Measuring Worth](https://www.measuringworth.com/).  On 2019-04-06 this produced two csv files, which I downloaded and copied into the working directory:   

```{r csv}
getwd()
(csv2 <- dir(pattern='\\.csv$'))
if(length(csv2)==2){
  Update0 <- TRUE
} else Update0 <- FALSE
```

The following assumes there are only 2 csv files in the working directory at this time, with the CPI being the first (in lexicographical order).  If this is not correct, something needs to change for the following to work properly.

Read these two files:  

```{r read.csv}
Update <- FALSE
if(Update0){
  str(USCPI <- read.csv(csv2[1], skip=2))
  str(USGDP. <- read.csv(csv2[2], skip=1))
  library(Ecfun)
  USGDP <- asNumericDF(USGDP.)
  print(rngCPIyrs <- range(USCPI$Year) )
  print(rngGDPyrs <- range(USGDP$Year) )
  endYr <- max(rngCPIyrs, rngGDPyrs)
  if(endYr>rngYrs[2]) print(Update <- TRUE)
}
```

## Update 

If Update, create a local copy of USGDPpresidents with 
the additional rows required to hold the new data:  

```{r cy}
if(Update){
  rowsNeeded <- (endYr - rngYrs[2])
  Nold <- nrow(USGDPpresidents)
  iRep <- c(1:Nold, rep(Nold, rowsNeeded))
  USGDPp2 <- USGDPpresidents[iRep,]
}
```

Fix the Year and insert NAs for all other columns for the new rows:   

```{r Year}
if(Update){
  iNew <- (Nold+(1:rowsNeeded))
  USGDPp2$Year[iNew] <- ((rngYrs[2]+1):endYr)
  rownames(USGDPp2) <- USGDPp2$Year
#
  USGDPp2[iNew, -1] <- NA
}
```

Now replace CPI by the new numbers:  

```{r CPI}
if(Update){
  selCPI <- (USGDPp2$Year %in% USCPI$Year)
  if(any(!is.na(USGDPp2[!selCPI, 2]))){
    stop('ERROR:  There are CPI numbers ', 
         'in the current USGDPpresidents ', 
         'that are not in the new.  ', 
         'Manual review required.')
  }
  USGDPp2$CPI[selCPI] <- USCPI[,2]
}
```

Does USGDPpresidents.Rd needs to be updated
to reflect the proper reference years for the 
CPI?  

```{r CPIref}
if(Update){
  readLines(csv2[1], n=4)
}
```

If this says "Average 1982-84 = 100", it should be good.  Otherwise that (and this) should be updated.  

Now let's update GDPdeflator:  

```{r GDPdeflator}
if(Update){
  selGDP <- (USGDPp2$Year %in% USGDP$Year)
#
  if(any(!is.na(USGDPp2[!selGDP, 'GDPdeflator']))){
    stop('ERROR:  There are GDPdeflator numbers ', 
         'in the current USGDPpresidents ', 
         'that are not in the new.  ', 
         'Manual review required.')
  }
  selDefl <- grep('Deflator', names(USGDP))
  USGDPp2$GDPdeflator[selGDP] <- USGDP[,selDefl]
  print(names(USGDP)[selDefl])
}
```  

Compare the index year of "GDP.Deflator" with that in USGDPpresidents.Rd:  If they are different, fix 
USGDPpresidents.Rd.  

Now update population:  

```{r pop}
if(Update){
  selPop <- grep('Population', names(USGDP))
  USGDPp2$population.K[selGDP] <- USGDP[,selPop]
  print(names(USGDP)[selPop])
}
```  

Now realGDPperCapita.  This also has a reference year, so we need to make sure we get them all:  

```{r GDPperCap}
if(Update){
  if(any(!is.na(USGDPp2[!selGDP, 'readGDPperCapita']))){
    stop('ERROR:  There are realGDPperCapita numbers ', 
         'in the current USGDPpresidents ', 
         'that are not in the new.  ', 
         'Manual review required.')
  }
  selGDPperC <- grep('Real.GDP.per.c', names(USGDP))
  USGDPp2$realGDPperCapita[selGDP] <- USGDP[,selGDPperC]
  print(names(USGDP)[selGDPperC])
}
```  

Compare the index year of 'Real.GDP.per.capita' with 
that in USGDPpresidents.Rd:  If they are different, 
fix USGDPpresidents.Rd.  

Next:  executive:  

```{r executive}
if(Update){
  exec <- as.character(USGDPp2$executive)
  exec[is.na(exec)] <- c('Obama', 'Trump', 'Trump')
  lvlexec <- c(levels(USGDPp2$executive), 
               'Trump')
  USGDPp2$executive <- ordered(exec, lvlexec)
}
```  

Similarly:  war

```{r war}
if(Update){
  war <- as.character(USGDPp2$war)
  war[is.na(war)] <- ''
  lvlwar <- levels(USGDPp2$war)
  USGDPp2$war <- ordered(war, lvlwar)
}
```  

Next:  `battleDeaths` and `battleDeathsPMP`:  

```{r battleDeaths}
if(Update){
  USGDPp2$battleDeaths[iNew] <- 0 
#
  USGDPp2$battleDeathsPMP <- with(USGDPp2, 
          1000*battleDeaths/population.K) 
}
```

Keynes?

```{r Keynes}
if(Update){
  USGDPp2$Keynes[iNew] <- 0 
}
```

## Unemployment?  

Unemployment figures came from different 
sources for different years.  Since 1940 
the source has been the Bureau of Labor 
Statistics (BLS), series LNS14000000 from 
the Current Population Survey.  These data 
are available as a monthly series from 
the [Current Population Survey of the Bureau 
of Labor Statistics](https://www.bls.gov/cps/).  
Download the most recent years as an Excel 
file, compute row averages, and manually 
transfer the numbers for the most recent years 
here:  

```{r unemp}
if(Update){
  USGDPp2$unemployment[iNew] <- c(4.875, 
                    4.35, 3.89166666666667)
  USGDPp2$unempSource[iNew] <- USGDPp2$unempSource[
    iNew[1]-1]
  tail(USGDPp2)
}
```

## Done:  Save

```{r save}
if(Update){
  USGDPpresidents <- USGDPp2
  save(USGDPpresidents, file='USGDPpresidents.rda')
}
```

Now copy this file from the current working directory 
to ~Ecdat\data, overwriting the previous version.  
    