---
title: "The tidyMicro Pipeline"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{The tidyMicro Pipeline}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# Installation

```{r, eval = FALSE}
install.packages("devtools")
devtools::install_github("CharlieCarpenter/tidyMicro")
library(tidyMicro); library(magrittr)
```

```{r, include = FALSE}
library(tidyMicro); library(magrittr)
```

# tidyMicro

## Reading in OTU Counts
  
  The first step is to read in your table(s) of operational taxonomic unit (OTU) counts and the clinical data for your subjects. The `tidyMicro` package contains 4 OTU tables from premature infants who required mechanical ventilation and had tracheal aspirates samples collected at 7, 14, and 21 days of age (+/- 48 hours). Each infant had bronchopulmonary dysplasia severity classified as mild, moderate, or severe. The OTU tables come from the phylum, class, order, and family level classifications. The corresponding clinical data is also included. The data are published [here](https://doi.org/10.1371/journal.pone.0170120).

```{r data}
## Loading OTU tables
data(phy, package = "tidyMicro")
data(cla, package = "tidyMicro")
data(ord, package = "tidyMicro")
data(fam, package = "tidyMicro")

## Loading meta data to merge with OTU table
data(clin, package = "tidyMicro")
```
  
  The **clinical data** must have a column of *unique* sequencing libraries named "Lib" that match the column names of your OTU table(s).
  
  The **OTU tables** should be formatted as we see below: 
1. The first column contains the OTU names with taxonomic rank delineated by "/"
2. The following column names are the *unique* sequencing libraries 
3. The rest of the table is populated with sequencing counts.

```{r, echo = F}
cla[1:6, 1:6] %>% knitr::kable()
```

## tidy_micro {#TM}
  
  Once the OTU table is formatted in this way the `tidy_micro` function is fairly simple to use. There are three possible ways to merge your OTU table(s) and clinical data into a "*micro_set*".

1. Provide a single OTU table and the name separately 
2. Provide an unnamed list of your OTU tables and provide the OTU names separately
    i) Names provided will be the "Table" column in your final *micro_set*  
        a) Order of names and OTU tables should match up
3. Provide a named list of your OTU tables
    i) Names of OTU tables will be the "Table" column in your final *micro_set*

```{r, eval=FALSE}

## 1. Single OTU table
micro.set <- tidy_micro(otu_tabs = cla,        ## OTU Table
                        tab_names = "Class",   ## OTU Names (Ranks)
                        clinical = clin)       ## Clinical Data

## 2. Unnamed List
otu_tabs <- list(phy, cla, ord, fam)
tab_names <- c("Phylum", "Class", "Order", "Family")

micro.set <- tidy_micro(otu_tabs = otu_tabs,   ## OTU Table
                        tab_names = tab_names, ## OTU Names (Ranks)
                        clinical = clin)       ## Clinical Data

## 3. Named List
otu_tabs <- list(Phylum = phy, Class = cla, Order = ord, Family = fam)

micro.set <- tidy_micro(otu_tabs = otu_tabs,  ## OTU Table
                        clinical = clin)      ## Clinical Data
```

```{r, include = F}
otu_tabs <- list(Phylum = phy, Class = cla, Order = ord, Family = fam)

micro.set <- tidy_micro(otu_tabs = otu_tabs,  ## OTU Table
                        clinical = clin)      ## Clinical Data
```
  
  The default for this function is to keep only the libraries found in each of the OTU tables' column names and the sequencing library column of your clinical data. You will receive a warning if some libraries are excluded during the merge. This option can be turned off by setting the option `complete_clinical = FALSE`.  

  Below are the first 6 rows and 12 columns of the data frame we created with `tidy_micro`. The first 4 columns are the OTU table names, the subject/library names, the taxa names, and the sequencing depths (Total). Columns 4-8 are calculated from the given OTU table: `bin` is a binary variable for presence or absence of taxa in that subject/library, `cts` is the original count given, `clr` is centered log-ratio transformed counts, and `ra` is the relative abundance, $RA = \frac{Taxa\ Count}{Seq\ Depth}$, of the taxa. A small amount (1/sequencing depth) is added to each library's taxa count before the clr transformation in order to avoid $log(0) = -\infty$. The following columns contain the supplied clinical data. 

```{r, echo = F}
micro.set[1:6, 1:12] %>% knitr::kable()
```
  
  All of the following functions rely on this format of the data set, so this must be the first step for any use of this pipeline.
  
# Data Exploration

  For simplicity, we will continue using only the sequencing information from day 7.

```{r}
micro.set %<>% filter(day == 7)
```

  Several standard plots useful for data exploration are available within tidy.micro.

## Taxa Summaries

  The function `taxa_summary` will supply a table of useful descriptive statistics. It will output a table containing group counts, the percent of subjects with $RA=0$, the RA mean, RA median, RA standard deviation, RA IQR, and several RA percentiles. You can control the taxa information summarized using the `obj` argument. You can stratify by categorical variables using the `...` argument. If the `table` argument is left to `NULL`, `taxa_summary` will summarize all tables within your *micro_set*.
  
```{r taxaSummary}
taxa_summary(micro.set, table = "Phylum") %>% 
  knitr::kable()
```

## PCA and PCoA Plots
  
### Principal Components

  `micro_pca` will calculate principle components on the centered log ratio transformation of the taxa counts using the `prcomp` function from the `stats` package. Scaling the taxa counts to a unit variance is the default option, and recommended, but this can be changed using `scaled = F`. The components are then plotted using the `ggbiplot` function.
  
  We need to specify the OTU table we'd like to work with using the `table` argument. This is consistent throughout the pipeline. Different tables can contain different the taxonomic ranks for analysis as it does in this walk through (e.g. phylum, family, or genus level data), or the different tables could also reflect other important differences such as sampling site (e.g. skin, nasal, gut, etc).

```{r PCA, fig.cap = "PCA Plot. Principle components of CLR transformed Family level OTU table.", message=F, warning=F, fig.width=6, fig.height=4, fig.width=6, fig.height=4}
micro.set %>% micro_pca(table = "Family",   ## Taxonomic table of interest
                       grp_var = bpd1, ## A factor variable for colors
                       legend_title = "BPD Severity")      
```

### Principal Coordinates

  Principle components calculated from a dissimilarity matrix are called "principle coordinates". If we supply a dissimilarity or distance matrix, e.g. a [beta diversity](#Beta_Div), `micro_pca` will output a principal coordinate plot. Principle coordinate analysis (PCoA) and principle component analysis (PCA) are both excellent exploratory plots, and are the same process mathematically. However, PCoA is more appropriate than PCA when data are missing or when there are fewer subjects than there are dimensions of the feature space, as is often the case in microbiome (or any omics) analysis.
  
```{r PCoA, fig.cap = "PCoA Plot. Principle coordinates of Family level OTU table Bray-Curtis dissimilarity.", fig.width=6, fig.height=4}
bray_beta <- micro.set %>%  ## Calculating dissimilarity
  beta_div(table = "Family")
  
micro.set %>% 
  micro_pca(dist = bray_beta, ## Beta diversity
            grp_var = bpd1, ## A factor variable for colors
            legend_title = "BPD Severity")      
```

### Longitudinal PCA/PCoA
  
  The following methods were established in microbiome data by K. Williamson and is based off of the Tucker3 loss function. These are tools for dealing with repeated measures within the ordination framework.

```{r}
long_micro <- tidy_micro(otu_tabs = otu_tabs,  ## OTU tables
                         clinical = clin)      ## clinical Data
```

#### Three Mode PCA
  
  Three Mode PCA is a way of creating Principle component plots while controlling for correlations between observations. We can interpret them the same as we would any PCA plot. Unfortunately we aren't seeing much separation between our groups.  
  
  The function requires your *micro_set* to have a column that indicates your subject names and a column that indicates your time points. The function also requires a grouping variable.

```{r ThreeMode, fig.cap="Three Mode PCA Plot. Principle components controlled for repeaated measures.", fig.width=6, fig.height=4}
long_micro %>% 
  three_mode(table = "Family", group = bpd1, subject = study_id, 
             time_var = day, main = "ThreeMode PCA", 
             subtitle = "3 Time Points", legend_title = "BPD Severity")
```
  
  Notice the warning message that pops up. The function will automatically pull out the subjects that are in every time point and only make plots with their information.

#### 3D PCA
  
  These 3D plots collapse over a chosen dimension of variation (e.g. time, subject, or taxa) and perform principle component / coordinate analyses on the remaining two dimensions.
  
  In order to make the function work we need to supply a subject, time, and the number of time points as in the `three_mode` function. 
  
```{r pca3dTime, fig.cap="3D Time PCoA Plot. Plotting principle coordinants collapsing over time axis.", fig.width=6, fig.height=4}
long_micro %>%
  pca_3d(table = "Family", time_var = day, subject = study_id, 
         modes = "AC", type = "PCoA")
```

You can change which axis to collapse over by changing the "modes" to either "BA" or "CB".

```{r pca3DSubj, fig.cap="3D Subject PCoA Plot. Plotting principle coordinants collapsing over subject axis.", fig.width=6, fig.height=4}
long_micro %>%
  pca_3d(table = "Family", time_var = day, subject = study_id, 
         modes = "CB", type = "PCoA")
```


## Bar Charts {#RA}

   Stacked bar charts of taxa RA are very standard visualizations for microbiome research. We can create bar charts of the raw RA stratified by categorical variable(s) of interest. As before, we need to specify the OTU table we want to plot. 
  
```{r raBars, fig.cap = "Stack Bar Chart. Stacked bar charts of taxa relative abundance by BPD severity.", fig.width=6, fig.height=4}
ra_bars(micro.set,         ## Dataset
        table = "Phylum",  ## Table we want
        bpd1,              ## Variable of interest
        ylab = "% RA", 
        xlab = "BPD", 
        main = "Stacked Bar Charts")
```
  
  As you can see, several of these phylum are very low abundance within the cohort, not to mention how quickly our plot legend would overflow if we tried to plot all of our taxa orders or families. Several of our functions include options to aggregate low taxa counts together in order to clean up these plots. 
1. Specify how many taxa will be named and used in the stacked bar charts using `top_taxa`. This option will take $X$ taxa with the highest average RA and aggregate all others into an "Other" category. 
2. Use the `RA` option to aggregate all taxa with RA below your cutoff into an "Other" category. 

  Only one of these can be used at a time. If a particular taxa is of interest you can use `specific_taxa` to pull any taxa out of the "Other" category if it doesn't meet the bar set by either `top_taxa` or `RA`.

```{r raBarsFilter, fig.cap = "'Taxa-Filtered' Stack Bar Chart. Stacked bar charts of taxa relative abundance using aggregated counts.", fig.width=6, fig.height=4}
ra_bars(micro.set,          ## Dataset
         table = "Phylum",  ## Table we want
         bpd1,              ## Variable of interest
         top_taxa = 3,     
         RA = 0,
         specific_taxa = c("Actinobacteria", "Bacteroidetes"),
         ylab = "% RA", xlab = "BPD", main = "Stacked Bar Charts")
```

`ra_bars` is flexible enough to include multiple factors in the `...` argument.

```{r raBarsFactors, fig.cap = "Multiple Factor Stack Bar Chart. Stacked bar charts of taxa relative abundance by multiple factors.", fig.width=6, fig.height=4}

ra_bars(micro.set,           ## Dataset
        table = "Phylum",    ## Table we want
        bpd1, gender,        ## Variables of interest
        top_taxa = 6,
        ylab = "% RA", xlab = "BPD by Sex", 
        main = "Stacked Bar Charts") +
   theme(axis.text.x = element_text(angle = 45, hjust = 1))
```

  This function can also create subject level bar charts by using "Lib" as your grouping variable.
  
```{r raBarsSubject, fig.cap = "Subject Stack Bar Chart. Subject level stacked bar charts of taxa relative abundance.", fig.width=6, fig.height=4}
ra_bars(micro.set,          ## Dataset
        table = "Phylum",   ## Table we want
        Lib,                ## Variable of interest
        top_taxa = 6,
        ylab = "% RA", xlab = "Library", main = "Stacked Bar Charts") +
   theme(axis.text.x = element_text(angle = 90, hjust = 1))
```

  Or just create a single stacked bar chart for the entire cohort by ignoring the `...` argument.
  
```{r raBarsCohort, fig.cap = "Cohort Stack Bar Chart. Overall microbiome community taxa relative abundance.", fig.width=6, fig.height=4}
ra_bars(micro.set,            ## Dataset
        table = "Phylum",     ## Table we want
        top_taxa = 6, 
        ylab = "% RA", main = "Stacked Bar Charts")
```

Note that you can manipulate the output plots by adding on `geom`s as we did in `ra_bars`. Most plotting functions in this pipeline will return a `ggplot` that you can manipulate through additional `geom` just like any other [ggplot](https://ggplot2.tidyverse.org/).

```{r raBarsManipulated, fig.cap = "Manipulated Stack Bar Chart. Stacked bar charts of taxa relative abundance by multiple factors.", fig.width=6, fig.height=4}
ra_bars(micro.set,            ## Dataset
        table = "Phylum",     ## Table we want
        top_taxa = 6,
        main = "Manipulated Stacked Bar Charts") +
  
  ## Additional geoms
  theme_dark() + 
  coord_flip() + 
  theme(legend.title = element_text(color = "blue", size = 20),
        legend.text = element_text(color = "red"))
```

## Box Plots

  We can make box plots of taxa relative abundance stratified by some categorical variable using the function `taxa_boxplot`.

```{r taxaBoxplot, fig.cap="Single Variable Box Plots. Box plots of Staphylococcaceae relative abundance by BPD severity.", fig.width=6, fig.height=4}
staph <- "Firmicutes/Bacilli/Bacillales/Staphylococcaceae"

taxa_boxplot(micro.set,      ## Our dataset
             taxa = staph,   ## Taxa we are interested in
             bpd1,           ## Variable of interest
             xlab = "BPD", 
             ylab = "Relative Abundance", 
             main = "Box Plot") 
```
  
  We can plot other taxa information besides the relative abundance (such as the raw counts) by specifying the `y` argument. This function also allows for multiple variables of stratification, and will give you the interaction of all categorical variables given. There is an apparent trend here, but be aware that raw counts are rarely the information of interest for analysis. Relative abundance or the centered log ratio is almost always more meaningful.
 
```{r taxaBoxplotMultipleFactor, fig.cap="Multi-Variable Box Plots. Box plots of Staphylococcaceae relative abundance by BPD severity and sex.", fig.width=6, fig.height=4}
taxa_boxplot(micro.set,        ## Our dataset
             taxa = staph,     ## Taxa we are interested in
             y = clr,          ## Making Boxplot of CLR
             bpd1, gender,     ## Variables of interest
             ylab = "Staphylococcaceae CLR",
             main = "Box plot", subtitle = "Subtitle") +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))
```
  
## Correlation Plots
  
  Correlations between taxa RA and continuous variables can be useful descriptives during data exploration. The functions `cor_heatmap` and `cor_rocky_mtn` will calculate correlation between the selected variable(s) and taxa information. The taxa information (counts, CLR, or RA) can be controlled through the `y` argument, and the type of correlation can be controlled through the `cor_type` argument. We recommend using either CLR transformed counts with Spearman's rank based correlation or taxa RA with Kendall's rank based correlation. Since the large number of 0 counts often present will create many ties Kendall's correlation more appropriate for taxa RA.

### Heat Maps
  
  Correlations for each taxa within the selected table are plotted in a heat map for one or several continuous variables. This style of non-symmetric heat map is sometimes called a "lasagna plot."
  
```{r corHeatmap, message=F, warning=F, fig.cap= "Lasagna Plot. Spearman orrelations between CLR taxa counts infant weight, gestational age (weeks).", fig.width=6, fig.height=4}
micro.set %>% cor_heatmap(table = "Class", weight, gestational_age)
```
  
### Rocky Mountain Plots
  
  The rocky mountain plot shows the correlation for each taxa within the selected table displayed along the horizontal axis. They will be color coded by the phylum they belong to (characters before the first "/" in the taxa name), and taxa with a correlation magnitude greater than or equal to a specified cutoff (`cor_label`) will be labeled. This function uses the same options for `cor_type` and `y` as `cor_heatmap`.
  
```{r corRockyMtn, message=F, warning=F, fig.cap="Correlation Rocky Mountain Plot. Rocky mountain plot of Spearman's correlation betwen CLR taxa counts and class level taxa.", fig.width=8, fig.height=6}
micro.set %>% cor_rocky_mtn(table = "Class",
                            weight,
                            cor_label = 0.3)
```

# Alpha Diversities
  
  `alpha_div` calculates Good's Coverage and the following alpha diversities through rarefied bootstrap samples and attaches them as columns to the given *micro_set*:
  
  * Richness: $S_{obs}$ and Chao1
  * Evenness: Shannon's and Simpsons E
  * Diversity: Shannon's H and Simpsons D
  
  It will either calculate the alpha diversity based on the specified `table` and attach it within each table of your *micro_set*, or calculate the alpha diversity of each `table` if no table is specified. Either way the alpha diversities will be attached as if they were a part of your clinical data. If your *micro_set* contains OTU tables of different taxonomic ranks, we recommend only calculating and analyzing the alpha diversity of the lowest rank. 
  
  This step can be computationally intensive. Lowering the number of bootstrap iterations with the `iter` argument can speed this up. We recommend removing subjects with poor sequencing as well (for all analyses). There are two options for removing subjects with poor sequencing: 
  
1. `min_depth`: Remove libraries with sequencing depths (Total) below min_depth.
2. `min_goods`: Remove libraries with Good's Coverage below min_goods.

```{r microAlpha}
micro_alpha <- alpha_div(micro.set, 
                         table = "Family",  ## Table of interest
                         min_depth = 5000,  ## Requires a Seq Depth of 5000 to be included
                         min_goods = 80)    ## Requires a Good's coverage of %80 
```

## Regress on Alpha Diversity {#tar}
  
  Once alpha diversities are calculated, standard regression can be used to analyse alpha diversities. `micro_alpha_reg` is a simple wrapper function that will run linear regression on each alpha diversity with your specified covariate pattern within the specified rank. Covariates included in the function will be added together in your covariate structure. For instance, if we include *Group, Age, Sex*, the function will fit $$\hat{alpha} \sim Group + Age + Sex.$$ You can also include an interaction by typing *Group \* Age*.
  
  The function will output a summary table for every model containing a column for the alpha diversity measure, the model coefficients, the $\beta$ estimates, the standard errors of that estimate, the test statistic, the p-value, and a 95% confidence interval for the $\beta$ estimates.
  
```{r alphaReg, eval = F}
micro_alpha %>% 
  micro_alpha_reg(table = "Family", bpd1, gender) %>% 
  knitr::kable()
```

```{r, echo = F}
micro_alpha %>% micro_alpha_reg(table = "Family", bpd1, gender) %>% 
  mutate(Beta = round(Beta, 2), std.error = round(std.error, 2), t.stat = round(t.stat,4),
         p.value = round(p.value,4))
```

# Beta Diversities {#Beta_Div}
  
  `beta_div` is a simple wrapper around the `vegdist` function. It requires you to specify the OTU table and the method of beta diversity you want calculated.
  
```{r betaDiv}
## Beta Diversity
bray <- beta_div(micro.set, table = "Class", method = "bray")
```

## Heat Map

  The `beta_heatmap` function can be used to create a heat map of your calculated beta diversities ordered by some categorical variable of interest.
  
```{r betaHeatmap, fig.cap="Beta Diversity Heat Map. Heat map of Bray-Curtis beta diversity grouped by BPD severity.", fig.width=6, fig.height=4}
bray %>% 
  beta_heatmap(micro.set, bpd1)
```

## PERMANOVA
  
  The `vegan` package contains a useful function called `adonis2` for running a PERMANOVA test. PERMANOVA is a permutation based ANOVA using a distance matrix as your response. `micro_PERMANOVA` is simply a wrapper function to make working within your *micro_set* easy. The output is analogous to that of a standard ANOVA. Please note that the F statistic given is a "pseudo-F" statistic from a "pseudo-F test" since it is based off of a permutation distribution not a true F distribution.

```{r PERMANOVA}
micro_PERMANOVA(micro.set, ## micro_set to pull covariates from
                bray,      ## Beta diversity matrix (or any distance matrix)
                method = "bray",   ## method used to calculate the beta diversity
                bpd1, mom_ethncty_2)  ## Covariates
```

# Taxa Models and Plots 

## Filtering on a subset {#FF}
  
  There are several reasons you might want to aggregate rare taxa counts into an "Other" category. Some plots and figures (e.g. `corHeatmap`, `corRockyMtn`, ) will look nicer without all of the extra names cluttering the legend, some taxa might not be fully classified, or their models might be unstable and therefore a waste of computation time. We can use the `otu_filter` function to apply filtering requirements to each table within the *micro_set*. There are three functions built in to filter out taxa with low counts. 

1. `prev_cutoff` is a prevalence cutoff where $X%$ of subjects must have this taxa present or it will be included in the "Other" category. 
2. `ra_cutoff` is a relative abundance (RA) cutoff where at least one subject must have a RA above the cutoff or the taxa will be included in the "Other" category. 
3. `exclude_taxa` can be used to specify any taxa that you would like to make sure are included in the "Other" category. For instance, our OTU tables have taxa that are named "Unclassified" that we'd like to exclude. This can be a character string of any length.
    i) Please note that the taxa specified with `exclude_taxa` will be filtered out of every OTU table given. To avoid this, you can make a subset of the data before filtering.

```{r taxaFilter}
## Taxa names "Bacteria" are essentially unclassified
exclude_taxa <- c("Unclassified", "Bacteria")

micro.filt <- micro.set %>% 
  otu_filter(prev_cutoff = 1,               ## Prevalence cutoff
             ra_cutoff = 0.1,               ## Relative abundance cutoff
             exclude_taxa = exclude_taxa)   ## Uninteresting taxa
```
  
### Important Note on Filtering 

  Please note that we did not filter before calculating our diversity measures! If we aggregate rare taxa into a single category, this will bias all of our diversity measures. Our Sobs and Choa1 in particular will be greatly biased. If you do not wish to calculate alpha or beta diversities, these filtering options are also available in the initial [tidy_micro](#TM) read in step.
  
```{r readInFilter, eval = F}
## Named List
otu_tabs <- list(Phylum = phy, Class = cla, Ord = ord, Family = fam)

tidy.filt <- tidy_micro(otu_tabs = otu_tabs,  ## OTU Table
                        clinical = clin,      ## Clinical Data
                        prev_cutoff = 1,      ## Prevalence cutoff
                        ra_cutoff = 1,        ## Relative abundance cutoff
                        exclude_taxa = exclude_taxa)   ## Uninteresting taxa
```

## Modeling

### Negative Binomial Models {#NB}
  
  It is a standard practice to model the relative abundance of each taxa using a negative binomial distribution. The function `nb_mods` will fit negative binomial models for each taxa within a specified table using the observed counts and the total sequencing counts as an offset. It uses `glm.nb` from the `MASS` package to fit these models; the profile likelihood confidence intervals are also calculated through the `MASS` package using the `confint` function.  
  
  As in [micro_alpha_reg](#tar) and [bb_mods](#BB), `nb_mods` will take each variable you specify as a new term to add into the model. For instance, if we include *Group, Age, Sex* the function will run the model $$log(\hat{cts}) \sim \beta_0  + \beta_1 Group + \beta_2 Age + \beta_3 Sex + log(Total).$$ The offset of log(Total) will be included automatically and is recommended, however it is possible to remove this term with `Offset = FALSE`

```{r nbMods, message=F, warning=F}
nb_fam <- micro.filt %>%       ## micro_set
  nb_mods(table = "Family",    ## Rank of taxa we want to model
          bpd1)                ## The covariate in our model
```
 
 As in [micro_alpha_reg](#tar) and [bb_mods](#BB), you can also include interaction terms (such as *Age\*Sex*) as you can with any other model.
 
```{r nbInt, eval = F}
## If we wanted the covariates to be bpd1+gender+bpd1*gender we just need input bpd1*gender.

nb_int <- micro.filt %>% 
  nb_mods(table = "Class", bpd1*gender)
```
  
  Notice that the function will tell you how many different models converged and how many did not. To explore the convergent models we can access the **Convergent_Summary** from `nb_mods`. The convergent summary table gives the taxa name, the model coefficients, the estimated $\beta s$, profile likelihood confidence limits, the Z-score, the p-value from a Wald test, an FDR adjusted p-value, and finally an Anova test from a likelihood-ratio test. Below is the summary information from the first two convergent taxa.

```{r nbConvergentSummary}
nb_fam$Convergent_Summary %>% .[1:6,] %>% knitr::kable()
```
  
  `nb_mods` also provides an **Estimate_Summary** table that contains a table of model summaries for convergent taxa that is more readily exportable into publications. It gives the taxa name, the model coefficients, the exponentiated beta coefficients for the rate ratio, a Wald confidence interval, the Z-score, and FDR_Pvalue. For interaction terms, the rate ratios are calculated taking their main effects into account. That is, rate ratios are the exponentiation sum of interaction and main effect $\beta$ coefficients, and confidence intervals are for the exponentiation sum of the $\beta$ coefficients. The FDR p-values are for the individual $\beta$ coefficients. Below is the summary information from the first two convergent taxa.  

```{r nbEstimateSummary}
nb_fam$Estimate_Summary %>% .[1:4,] %>% knitr::kable()
```
  
  For the non-convergent taxa we can explore their summary information within the **RA_Summary** from `nb_mods`. This is a table of summary measures for all taxa (not just the non-convergent taxa) that is stratified by the categorical variables in the models. It includes the counts (n), percent of subjects with counts of 0, the average, median, standard deviation, IQR, percentiles of taxa RA, and finally an indicator for whether or not this taxa's negative binomial model converged. 

### Beta Binomial Models {#BB}

Another standard practice is to model taxa abundance using a beta binomial distribution. The function `bb_mods` will fit beta binomial model to each taxa within a specified table using the `vglm` function from the `VGAM` package. Confidence intervals are fit using the `confintvglm` function. The current default for confidence intervals around individual $\beta$ parameters are Wald intervals, although this can be change to profile likelihood confidence intervals using `CI_method = "profile"`. Profile likelihoods are much more computationally intensive.
  
```{r bbMods, message=F, warning=F, eval = F}
bb_fam <- micro.filt %>%       ## micro_set
  bb_mods(table = "Phylum",    ## Table we want to model
          bpd1)                ## The covariate in our model
```
  
  As in [micro_alpha_reg](#tar) and [nb_mods](#NB), `bb_mods` will take each variable you specify as a new term to add into the model. For instance, if we include *Group, Age, Sex* the function will run the model $$logit(\hat{RA}) \sim \beta_0 + \beta_1 Group + \beta_2 Age + \beta_3 Sex$$

 Again, as in [micro_alpha_reg](#tar) and [nb_mods](#NB), you can also include interaction terms (such as *Age\*Sex*) as you can with any other model.
 
```{r bbInt, eval = F}
## If we wanted the covariates to be bpd1+gender+bpd1*gender we just need input bpd1*gender.

bb_int <- micro.filt %>% 
  bb_mods(table = "Class", bpd1*gender)
```

## Plotting

  The model output from [nb_mods](#NB) and [bb_mods](#BB) can easily cause some information overload. `tidyMicro` includes several useful functions to visualize the results of all convergent models. They function the same way for both model types, so we will only show examples from our [nb_mods](#NB) output.

### Stacked Bar Charts
  
  We have created functions that will calculate the estimated RA of all taxa based on the convergent models. This gives us the ability to visualize stacked bar charts of taxa RA while controlling for other variables in the model. For instance, if our model is $$log(\hat{cts}) \sim \beta_0  + \beta_1 Group + \beta_2 Age + \beta_3 Sex + log(Total),$$ we can visualize the estimated differences in RA among different groups while holding Age and Sex constant.  
  
  The functions [nb_mods](#NB) and [bb_mods](#BB) will create these stacked bar charts based on the output of from `nb_mods` and `bb_mods`, respectively. They requires the name of a covariate in your model and can create plots based on main effects or interactions. If a continuous variable is one of the supplied covariates there are two options for visualization available through the `quant_style` argument. You can either visualize two points as two separate bars (`quant_style = "discrete"`) or visualize the continuous change (`quant_style = "continuous"`). By default the functions will use the first and third quartiles of the continuous variable as your endpoints, but this can be changed through the `range` option. `nb_bars` and `bb_bars` also give you the ability to aggregate estimated RA into and "Other" category just like the [ra_bars](#RA) function.
  
```{r nbFamBarCharts, warning=FALSE, fig.cap="Negative Binomial Bar Charts. Stacked bar charts of negative binomial estimated taxa RA by BPD severity.", fig.width=6, fig.height=4}
nb_fam %>% nb_bars(bpd1,                 ## Covariate of interest
                   top_taxa = 5,         ## How many named taxa we want
                   xlab = "Group", 
                   xaxis = c("1","2","3")) ## Labels
```

#### Object Flexibility

  Including every taxonomic level in your legend often looks cluttered. The [nb_bars](#NB) and [bb_bars](#BB) functions use the `Model_Coef` data frame from `nb_mods` and `bb_mods` output to create their bar charts. You can manipulate the taxa names here to avoid confusion further up in the pipeline. 
  
```{r manipulatedSBC, warning=FALSE, fig.width=6, fig.height=4}
nb_fam$Model_Coef$Taxa %<>% 
  stringr::str_split("/") %>%  ## Splitting by "/" in taxa names
  lapply(function(x) x[length(x)]) %>% ## selecting piece after the last "/"
  unlist ## Unlisting to put back into data frame

## Reordering to put "Other" at the bottom of the legend
## our functions usually do this automatically, but we'll need to do 
## this externally since we are messing with the output
non.other <- nb_fam$Model_Coef %>% 
  filter(Taxa != "Other") %>% 
  arrange(Taxa) %>% 
  distinct(Taxa) %>%
  pull(Taxa)

nb_fam$Model_Coef$Taxa <- factor(nb_fam$Model_Coef$Taxa,
                                 levels = c(non.other, "Other"))

nb_fam %>% nb_bars(bpd1,                 ## Covariate of interest
                   top_taxa = 5,         ## How many named taxa we want
                   xlab = "BPD Severity", main = "Cleaner Legend") ## Labels
```
  
### Rocky Mountain Plots
  
  The function `micro_rocky_mtn` displays magnitude of the log FDR adjusted p-values for each of the taxa in `nb_mods` as vertical bars next to each other along the x-axis. The direction of the bars will be determined by the direction of the estimated relationship. The taxa will be color coded by the phylum they belong to, and taxa that have FRD adjusted p-values for the specified covariate below your desired significance cutoff will be labeled. This significance cutoff is 0.05 by default and can be changed through the `alpha` argument. You can also turn off the labels with `sig_text = FALSE`.

```{r nbRockyMtn, fig.cap="Negative Bionomial Rocky Mountian Plot. Rocky mountain plot created from log FDR adjusted p-values of beta coefficients. Log p-values from beta estimate > 0 are multiplied by -1.", fig.width=10, fig.height=6, warning=F, message=F}
## Order level models
nb_ord <- micro.filt %>%       ## micro_set
  nb_mods(table = "Order",     ## Rank of taxa we want to model
          bpd1)                ## The covariate in our model

nb_ord %>%
  micro_rocky_mtn(bpd1, ## Covariate of interest
                  xlab = "Taxa", main = "Rocky Mountain Plot",
                  subtitle = "Direction of bar indicates direction of relationship", facet_labels = c("Moderate", "Severe"))
  
```

### Forest Plots

  `micro_forest` will create forest plots for a specified covariate from each taxa's [nb_mod](#NB) or [bb_mod](#BB). Forest plots display the estimated $\beta$ coefficients with their 95% confidence intervals.

```{r nbForest, warning = F, message = F, fig.wide = T, fig.cap="Forest Plot. A forest plot showing the beta estimates and 95% confidence intervals of selected covariate from each taxa model.", fig.width=6, fig.height=4}
## Class level models
nb_cla <- micro.filt %>%       ## micro_set
  nb_mods(table = "Class",     ## Rank of taxa we want to model
          bpd1)                ## The covariate in our model

nb_cla %>%
  micro_forest(bpd1, ## Covariate of interest
               main = "Forest Plot for BPD Severity")
```

### Heat Maps
  
  We can also create heat maps of the estimate coefficients from each taxa model to give us a visual of each estimate using `micro_heatmap`. This might be a helpful tool in assessing patterns in estimated relationships, or for detecting taxa that are strongly associated with our coefficients. We can also specify the number of taxa we wish to show with the `top_taxa`, as we can with [nb_bars](#NB) and [ra_bars](#RA). Here `top_taxa` is selecting the taxa with the largest estimated $beta$ coefficients. Coefficients with significant FDR adjusted p-values will be be marked in the center. The significance cutoff can be controlled with the `alpha` argument. The size and shape of this mark can be controlled through the `dot_size` and `dot_shape` arguments, respectively.

```{r microHeatmap, warning=F, message=F, fig.cap="Heat Map of Parameter Estimates. Heat map (or lasagna plot) created from beta estimates of taxa models. '*' indicates FDR adjusted p-values < 0.05.", fig.width=6, fig.height=4}
nb_cla %>% micro_heatmap(top_taxa = 7)
```

# Rank Sum Tests {#RS}

  Rank based tests are fairly common in microbiome analysis. We recommend fitting [negative binomial models](#NB) or [beta binomial models](#BB) when possible as they allow you to control for possible confounders, get meaningful estimates, test a more meaningful hypothesis, and often provide much more power. 
  That said, small sample sizes and zero-inflation can prevent [negative binomial](#NB) and [beta binomial](#BB) models from converging. If you have too small a sample size for any parametric testing or if you'd like to run a test on a non-convergent model, the function `micro_rank_sum` can be used. The function will run a rank sum test on a categorical variable (Wilcoxon for 2 levels and Kruskal-Wallis for more than 2). If you supply the output from `nb_mods` or `bb_mods`, it will run a rank sum test on every non-convergent taxa, otherwise it will run the test on every taxa in the specified table. By default `micro_rank_sum` will use taxa RA as the response variable, and this is recommended for microbiome rank sum tests. However you can control this using the `y` argument.
  
```{r rankSum, message=F, warning=F}
## Kurskal-Wallis on every taxa
micro.filt %>% 
  micro_rank_sum(table = "Family", grp_var = bpd1) %>% 
  knitr::kable()
```
  
```{r rankSumNB, message=F, warning=F}
## Kruskal-Wallis on non-convergent taxa
micro.filt %>% 
  micro_rank_sum(table = "Family", grp_var = bpd1, mod = nb_fam) %>% 
  knitr::kable()
```

# Chi-Squared Tests

  An alternative to rank sum tests on taxa RA is running a Chi-Squared test on the presence / absence of the taxa using `micro_chisq`. The same option to run the test on all taxa or only non-convergent taxa as in [micro_rank_sum](#RS) exists. The test will use presence / absence (bin) as its response variable.
  
```{r chisq, message=F, warning=F}
## Chisq on every taxa
micro.filt %>% 
  micro_chisq(table = "Family", grp_var = bpd1, simulate.p.value = T) %>% 
  knitr::kable()
```
  
```{r chisqNB, message=F, warning=F}
## Chisq on non-convergent taxa
micro.filt %>% 
  micro_chisq(table = "Family", grp_var = bpd1, mod = nb_fam) %>% 
  knitr::kable()
```
  
# Session Info

```{r}
sessionInfo()
```
