# Introduction to tsibble

#### Earo Wang

The tsibble package extends the tidyverse to temporal-context data. Built on top of the tibble, a tsibble (or `tbl_ts`) is a data-centric format, following the tidy data principle (Wickham 2014). Compared to the conventional time series objects in R, for example `ts` or `mts`, the tsibble preserves time indices as the essential component and makes heterogeneous data structures possible. Beyond the tibble-like representation, new syntax is introduced to impose additional and informative structures on the tsibble, which is referred to as “key” variables. Multiple keys separated by a vertical bar (`|`) or a comma (`,`) are expressive of nested or crossed variables. This binds hierarchical and grouped time series together into the `tbl_ts` class. The tsibble package aims at managing temporal data and getting analysis done in a tidy and modern manner.

The `tsibble()` creates a tsibble object, and `as_tsibble()` is an S3 method to coerce other objects to a tsibble. An object that a vector/matrix underlies, such as `ts`, `mts`, or `hts`, can be automated to a tsibble using `as_tsibble()` without any specification. If it is a tibble or data frame, the `as_tsibble()` requires a little more setup in order to identify the index and key variables.

## Index and key(s)

    library(tsibble)
    library(lubridate)
    weather <- nycflights13::weather %>%
      select(origin, time_hour, temp, humid, precip)
    weather

    #> # A tibble: 26,130 x 5
    #>   origin time_hour            temp humid precip
    #>   <chr>  <dttm>              <dbl> <dbl>  <dbl>
    #> 1 EWR    2013-01-01 00:00:00  37.0  54.0     0.
    #> 2 EWR    2013-01-01 01:00:00  37.0  54.0     0.
    #> 3 EWR    2013-01-01 02:00:00  37.9  52.1     0.
    #> 4 EWR    2013-01-01 03:00:00  37.9  54.5     0.
    #> 5 EWR    2013-01-01 04:00:00  37.9  57.0     0.
    #> # ... with 2.612e+04 more rows

The `weather` data included in the package `nycflights13` contains the hourly meteorological records (such as temperature, humid and precipitation) over the year of 2013 at three stations (i.e. JFK, LGA and EWR) in New York City. Since the `time_hour` is the only one column consisting of the timestamps, the `as_tsibble()` detects it as the index variable; alternatively, it would be more verbose to specify the argument `index = time_hour`. A tsibble is comprised of an index and key variables. In this case, the `origin` variable is the identifier created via the `id()` and passed to the `key` argument in the `as_tsibble()`. Therefore, the key together with the index uniquely identifies each observation, which gives a valid tsibble. In other words, each unit of observation is measured at a time point for a key or each combination of keys. Others—`temp`, `humid` and `precip`—are considered as measured variables.

    weather_tsbl <- as_tsibble(weather, key = id(origin))

    #> The `index` is `time_hour`.

    weather_tsbl

    #> # A tsibble: 26,130 x 5 [1HOUR]
    #> # Keys:      origin [3]
    #>   origin time_hour            temp humid precip
    #>   <chr>  <dttm>              <dbl> <dbl>  <dbl>
    #> 1 EWR    2013-01-01 00:00:00  37.0  54.0     0.
    #> 2 EWR    2013-01-01 01:00:00  37.0  54.0     0.
    #> 3 EWR    2013-01-01 02:00:00  37.9  52.1     0.
    #> 4 EWR    2013-01-01 03:00:00  37.9  54.5     0.
    #> 5 EWR    2013-01-01 04:00:00  37.9  57.0     0.
    #> # ... with 2.612e+04 more rows

The tsibble fully utilises the `print` method from tibble: a tsibble object (along with its dimension and time interval) and key variables in the header. Above displays the `weather_tsbl` its one-hour interval and the `origin` as keys. It should be noted that the tsibble does not attempt to arrange the data in time order. Given this format, it is much easier for users, in particular who are familiar with tidyverse, to perform common data tasks in temporal context. For example the `tsummarise()` (summarise over time) is used to examine daily highs and lows at each station. As a result, the index is updated to the `date` with one-day interval from `time_hour`; two new variables are created and computed for daily maximum and minimum temperatures.

    weather_tsbl %>%
      group_by(origin) %>%
      tsummarise(
        date = as_date(time_hour),
        temp_high = max(temp, na.rm = TRUE),
        temp_low = min(temp, na.rm = TRUE)
      )

    #> # A tsibble: 1,092 x 4 [1DAY]
    #> # Keys:      origin [3]
    #>   origin date       temp_high temp_low
    #>   <chr>  <date>         <dbl>    <dbl>
    #> 1 EWR    2013-01-01      41.0     34.0
    #> 2 EWR    2013-01-02      34.0     24.1
    #> 3 EWR    2013-01-03      34.0     26.1
    #> 4 EWR    2013-01-04      39.9     28.9
    #> 5 EWR    2013-01-05      44.1     32.0
    #> # ... with 1,087 more rows

## Nested and crossed keys

The key is not constrained to a single variable, but expressive of nested and crossed data structures (Wilkinson 2005). A built-in dataset `tourism` includes the quarterly overnight trips from 1998 Q1 to 2016 Q4 across Australia, which is sourced from Tourism Research Australia. The key structure is imposed by `Region | State, Purpose`. The `Region` and `State` naturally form a two-level geographical hierarchy: the lower-level regions are nested into the higher-level states. This nesting/hierarchical structure is indicated using a vertical bar (`|`). The crossing of `Purpose` (purpose of visiting) with the geographical variables suffices to validate the tsibble, where a comma (`,`) separates these two groups. Each observation is the number of trips made to a specific region for a certain purpose of travelling at one quarter of the year.

    as_tsibble(tourism, key = id(Region | State, Purpose), index = Quarter)

    #> # A tsibble: 23,408 x 5 [1QUARTER]
    #> # Keys:      Region | State, Purpose [308]
    #>   Quarter Region   State           Purpose  Trips
    #>     <qtr> <chr>    <chr>           <chr>    <dbl>
    #> 1 1998 Q1 Adelaide South Australia Business  135.
    #> 2 1998 Q2 Adelaide South Australia Business  110.
    #> 3 1998 Q3 Adelaide South Australia Business  166.
    #> 4 1998 Q4 Adelaide South Australia Business  127.
    #> 5 1999 Q1 Adelaide South Australia Business  137.
    #> # ... with 2.34e+04 more rows

The commonly used dplyr verbs, such as `filter()`, `summarise()` and `mutate()`, have been implemented to support the tsibble. To obtain the numerical summaries for the nesting of geography, the `summarise()` is performed in conjunction with the `Region | State` in the `group_by()`. This specification retains the hierarchical structure. The tsibble `summarise()` never collapses the rows over the time index, which is slightly different from the dplyr `summarise()`.

    tourism %>%
      group_by(Region | State) %>%
      summarise(Geo_Trips = sum(Trips))

    #> # A tsibble: 5,852 x 4 [1QUARTER]
    #> # Keys:      Region | State [77]
    #>   Quarter Region                     State              Geo_Trips
    #>     <qtr> <chr>                      <chr>                  <dbl>
    #> 1 1998 Q1 Adelaide                   South Australia       659.
    #> 2 1998 Q1 Adelaide Hills             South Australia         9.80
    #> 3 1998 Q1 Alice Springs              Northern Territory     20.2
    #> 4 1998 Q1 Australia's Coral Coast    Western Australia     133.
    #> 5 1998 Q1 Australia's Golden Outback Western Australia     162.
    #> # ... with 5,847 more rows

This syntactical approach appears more advantageous for the structural variables when coming to hierarchical and grouped time series forecast.

## Intervals

It has been seen that the tsibble handles regularly-spaced temporal data well, from seconds to years. The option `regular`, by default, is set to `TRUE` in the `as_tsibble()`. Specify `regular` to `FALSE` to create a tsibble for the data collected at irregular time interval. Below shows the scheduled date time of the flights in New York City:

    flights <- nycflights13::flights %>%
      mutate(
        sched_dep_datetime = make_datetime(year, month, day, hour, minute, 0),
        flight_num = paste0(carrier, flight)
      )

The key variable is the `flight_num`. With `regular = FALSE`, it turns to an irregularly-spaced tsibble, where `[!]` highlights the irregularity.

    flights_tsbl <- flights %>%
      as_tsibble(
        key = id(flight_num),
        index = sched_dep_datetime,
        regular = FALSE
      )
    flights_tsbl

    #> # A tsibble: 336,776 x 21 [!]
    #> # Keys:      flight_num [5,725]
    #>    year month   day dep_time sched_dep_time dep_delay arr_time
    #>   <int> <int> <int>    <int>          <int>     <dbl>    <int>
    #> 1  2013    11     3     1531           1540       -9.     1653
    #> 2  2013    11     4     1539           1540       -1.     1712
    #> 3  2013    11     5     1548           1540        8.     1708
    #> 4  2013    11     6     1535           1540       -5.     1657
    #> 5  2013    11     7     1549           1540        9.     1733
    #> # ... with 3.368e+05 more rows, and 14 more variables:
    #> #   sched_arr_time <int>, arr_delay <dbl>, carrier <chr>, flight <int>,
    #> #   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>,
    #> #   distance <dbl>, hour <dbl>, minute <dbl>, time_hour <dttm>,
    #> #   sched_dep_datetime <dttm>, flight_num <chr>

More functions on their way to deal with irregular temporal data in the future release.

## Reference

Wickham, Hadley. 2014. “Tidy Data.” Journal of Statistical Software 59 (10). Foundation for Open Access Statistics:1–23.

Wilkinson, Leland. 2005. The Grammar of Graphics (Statistics and Computing). Secaucus, NJ: Springer-Verlag New York, Inc.