---
title: "Propensity to Cycle Tool Advanced Workshop"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{pct_training}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
  %\VignetteDepends{tmap} 
bibliography: refs_training.bib
---

```{r, eval=FALSE, echo=FALSE}
header-includes:
- \usepackage{fancyhdr}
- \pagestyle{fancy}
- \fancyhead[CO,CE]{Propensity to Cycle Tool Training Course}
- \fancyfoot[CO,CE]{For source code and support see github.com/ITSLeeds and github.com/npct}
- \fancyfoot[LE,RO]{\thepage}
output: pdf_document
```


```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  out.width = "50%"
)
```

```{r, eval=FALSE, echo=FALSE}
# get citations
refs = RefManageR::ReadZotero(group = "418217", .params = list(collection = "JFR868KJ", limit = 100))
refs2 = RefManageR::ReadBib("vignettes/refs.bib")
refs = c(refs, refs2)
refs_df = as.data.frame(refs)
# View(refs_df)
# citr::insert_citation(bib_file = "vignettes/refs_training.bib")
RefManageR::WriteBib(refs, "vignettes/refs_training.bib")
# citr::tidy_bib_file(rmd_file = "vignettes/pct_training.Rmd", messy_bibliography = "vignettes/refs_training.bib")
```

# Introduction

This vignette supports workshops on advanced usage and development of the Propensity to Cycle Tool (PCT).
Beginner and intermediate PCT events focus on using the PCT via the web application hosted at [www.pct.bike](http://www.pct.bike/) and the data provided by the PCT in [QGIS](https://www.qgis.org/en/site/).

The focus here is on analysing cycling potential in the open source statistical programming language R, in which the majority of the PCT was built.
It will show how the code underlying the PCT works, how the underlying data can be accessed for reproducible analysis, and how the methods can be used to generate new scenarios of cycling uptake.

<!-- There will be 2 courses: -->

<!-- - In London on **Friday 21st June**  -->
<!-- - In Leeds on **Friday 2nd August (TBC)**  -->

<!-- # Prerequisites -->

<!-- This is an advanced training course with specific pre-requisites. -->
<!-- Please only register if you: -->

<!-- - Have experience analysing geographic area and route data of the type used in the PCT -->
<!-- - Have experience using software for geographic data, including working knowledge of the statistical programming language R -->
<!-- - Can bring a laptop that has up-to-date versions of R, RStudio and the necessary R packages installed (see next section) -->

<!-- In addition, if you want to do routing on your computer:  -->

<!-- - Register for a CycleStreets.net API key from: https://www.cyclestreets.net/api/ -->

<!-- If you are new to R this course may not be appropriate for you. -->
If you are an intermediate user, it may be worth brushing-up on your R skills, e.g. by taking a free online course such as that provided by [DataCamp](https://www.datacamp.com/courses/free-introduction-to-r) or by working through [Chapter 2 onwards of the open source book *Geocomputation with R*](https://geocompr.robinlovelace.net/spatial-class.html) (see reading list below for more transport-specific resources).

<!-- ## Prior reading -->

<!-- In addition to computer hardware (a laptop) and software (an up-to-date R set-up and experience using R) pre-requisites, you should have read, or at least have working knowledge of the contents of, the following publications, all of which are freely available online: -->

<!-- - The original paper on the Propensity to Cycle Tool [@lovelace_propensity_2017] -->
<!-- - The transport chapter ([12](https://geocompr.robinlovelace.net/transport.html)) in the open source book [*Geocomputation with R*](https://geocompr.robinlovelace.net/)  [@lovelace_geocomputation_2019] -->
<!-- - The vignette that describes the pct R package, which can be found here: https://itsleeds.github.io/pct/articles/pct.html -->

## Prerequisites

**To ensure your computer is ready for the course, you should be able to run the following lines of R code on your computer:**

```{r, eval=FALSE}
install.packages("remotes")
pkgs = c(
  "cyclestreets",
  "mapview",
  "pct",
  "sf",
  "stats19",
  "stplanr",
  "tidyverse",
  "devtools"
)
remotes::install_cran(pkgs)
# remotes::install_github("ITSLeeds/pct")
```

To test your computer is ready to work with PCT data in R, try running the following command:

 <!-- method for helping people set up their computers. Type this single line into the console and follow the instructions.  -->

```{r testcode, eval = FALSE }
source("https://raw.githubusercontent.com/ITSLeeds/TDS/master/code-r/setup.R") 
```

If the new method does not work or you would like to be more hands on, run the code below.
It should result in the map below, showing the % of short trips in Isle of Wight made by active modes.


```{r setup, out.width="30%", message=FALSE}
library(pct)
library(dplyr)   # in the tidyverse
library(tmap) # installed alongside mapview
region_name = "isle-of-wight"
max_distance = 7
zones_all = get_pct_zones(region_name)
lines_all = get_pct_lines(region_name)
# basic plot
plot(zones_all$geometry)
plot(lines_all$geometry[lines_all$all > 500], col = "red", add = TRUE)

# create 'active' desire lines (less than 5 km)
active = lines_all %>% 
  mutate(`Percent Active` = (bicycle + foot) / all * 100) %>% 
  filter(e_dist_km < max_distance)

# interactive plot
tmap_mode("view")
tm_shape(active) +
  tm_lines("Percent Active", palette = "RdYlBu", lwd = "all", scale = 9)
```

```{r, echo=FALSE, eval=FALSE}
# old code - create with leaflet
pal = colorBin(palette = "RdYlBu", domain = active$`Percent Active`, bins = c(0, 2, 4, 10, 15, 20, 30, 40, 90))
leaflet(data = active) %>% 
  addProviderTiles(providers$OpenStreetMap.BlackAndWhite) %>% 
  addPolylines(color = ~pal(`Percent Active`), weight = active$all / 100) %>% 
  addLegend(pal = pal, values = ~`Percent Active`)

pal = colorBin(palette = "RdYlBu", domain = car_dependent$`Percent Active`, bins = c(0, 20, 40, 60, 80, 100), reverse = TRUE)
leaflet(data = car_dependent) %>% 
  addProviderTiles(providers$OpenStreetMap.BlackAndWhite) %>% 
  addPolylines(color = ~pal(`Percent Drive`), weight = active$all / 100) %>% 
  addLegend(pal = pal, values = ~`Percent Drive`)
```

We can also use the data to explore entrenched car dependence, as follows:

```{r, out.width="30%"}
# Create car dependent desire lines
car_dependent = lines_all %>% 
  mutate(`Percent Drive` = (car_driver) / all * 100) %>% 
  filter(e_dist_km < max_distance)
tm_shape(car_dependent) +
  tm_lines("Percent Drive", palette = "-RdYlBu", lwd = "all", scale = 9)
```

\newpage

# Agenda

- 09:30 - 10:00: Arrival and set-up
- 10:00 - 11:00: Data and methods underlying the PCT
- 11:00 - 12:30: Getting and analysing PCT data
    - Comparing PCT results with existing road infrastructure
        - Getting data from the CyIPT
        - Identfying 'weak links'
        - Identify gaps in the network

Lunch break

- 13:30 - 15:30: Extending the PCT: minihack
    - Alternative scenarios of cycling uptake
        - A 'replace short car trips' scenario
    - New input data
        - Travel to stations
    - Other extensions of the PCT

Break and presentation of results

- 16:00 - 16:30: Policy implementation
    - Building open source web applications
    - Community-building
    - User interface
    - Policy uptake
    
# Exercises

## How the PCT works and what you can use it for

- Take a look at the image below from [@lovelace_propensity_2017]. Which parts of the process are of most use for your work?

```{r, echo=FALSE, out.width="90%"}
# u = "https://raw.githubusercontent.com/npct/pct-team/master/flow-model/flow-diag2.png"
# f = "vignettes/flow-diag2.png"
# download.file(u, f)
# knitr::include_graphics("flow-diag2.png")
knitr::include_graphics("https://raw.githubusercontent.com/npct/pct-team/master/flow-model/flow-diag2.png")
```

- In groups of 2, identify 3 real world questions/problems that you could use the PCT to help answer/solve

<!-- Communicating a vision -->
<!-- Deciding on network scale/size/shape -->
<!-- Generating business case for investment -->
<!-- Feed into consulation -->
<!-- Extend it to look at cycling integraton -->

The PCT provides data at 4 geographic levels:

- Zones
- Desire lines
- Routes
- Route networks

Which types of data are most appropriate to tackle each of the questions/problems you identified?

- Name 3 limitations of the data currently provided by the PCT and discuss how you could overcome them

<!-- OD data is old -> use new OD data -->
<!-- Only single stage journeys -> Model travel to rail, for example -->
<!-- It does not predict future levels of cycling following new infrastructure -> -->
<!--   Wait until follow-on work building CyIPT is developed -->

## Getting and viewing PCT data

- G1: Using the PCT's online interface, hosted at [www.pct.bike/m/?r=isle-of-wight](http://www.pct.bike/m/?r=isle-of-wight), identify the MSOA **zone** that has the highest number of people who cycle to work.

- G2: Using data downloaded with the command `get_pct_zones()`, identify the zone that has highest level of cycling with the function `top_n()` and save the result as an object called `z_highest_cycling` (hint: you may want to start by 'cleaning' the data you have downloaded to include only a few key columns with the function `select()`, as follows):

```{r}
library(pct)
library(dplyr) # suggestion: use library(tidyverse)
z_original = get_pct_zones("isle-of-wight")
z = z_original %>% 
  select(geo_code, geo_name, all, bicycle, car_driver)
```

```{r, echo=FALSE}
# the solution:
# View(z)
z_highest_cycling = z %>% 
  top_n(n = 1, wt = bicycle)
```

- G3: Use the `plot()` command to visualise where on the Ilse of Wight this 'high cycling' zone is (hint: you will need to use the `plot()` function twice, once to plot `z$geometry`, and again with the argument `add = TRUE` and a `col` argument to add the layer on top of the base layer and give it a colour).
The result should look something like something this:

```{r, echo=FALSE}
plot(z$geometry)
plot(z_highest_cycling$geometry, col = "red", add = TRUE)
```

- G4: Using the online interface, identify the top 5 MSOA to MSOA **desire lines** that have the highest number of people who cycle to work.

- G5: Using the function `get_pct_lines()`, identify the top 5 MSOA to MSOA **desire lines** that have the highest number of people who cycle to work (hint: you might want to start with the code shown below).
    - Bonus: also find the 5 desire lines with the highest number of people driving to work. Plot them and find the straight line distance of these lines with the function `st_length()`.
    
```{r get routes}
# Aim: get top 5 cycle routes
l_original_msoa = get_pct_lines("isle-of-wight")
l_msoa = l_original_msoa %>% 
  select(geo_code1, geo_code2, all, bicycle, car_driver, rf_avslope_perc, rf_dist_km)
```

```{r, echo=FALSE, warning=FALSE, fig.show='hold', fig.cap="Top 5 MSOA to MSOA desire lines with highest number of people cycling (left) and driving (right) in the Isle of Wight."}
# View(l)
l = l_msoa
l_top_cycling = l %>% 
  top_n(n = 5, wt = bicycle)
plot(z$geometry)
plot(l_top_cycling, add = TRUE, lwd = 5, col = "green")

# top 5 driving routes
l_top_driving = l %>% 
  top_n(n = 5, wt = car_driver)
plot(z$geometry)
plot(l_top_driving, add = TRUE, lwd = 5, col = "red")

# summary(sf::st_length(l_top_cycling))
# summary(sf::st_length(l_top_driving))

# library(tmap)
# tmap_mode("view")
# tm_shape(l_top_cycling) + tm_lines("green", lwd = 7) + tm_basemap(server = leaflet::providers$OpenStreetMap.BlackAndWhite)
# tm_shape(l_top_driving) + tm_lines("red", lwd = 7) + tm_basemap(server = leaflet::providers$OpenStreetMap.BlackAndWhite)
```

- G6 (Bonus):  Repeat the exercise but for LSOA to LSOA desire lines (by setting the argument `geography = "lsoa"`, remember to change the names of the objects you create). The results should look something like this:

```{r, echo=FALSE, warning=FALSE, fig.show='hold', fig.cap="Top 5 LSOA-LSOA desire lines with highest number of people cycling (left) and driving (right) in the Isle of Wight."}
# at the lsoa level
l_original_lsoa = get_pct_lines("isle-of-wight", geography = "lsoa")
l = l_original_lsoa %>% 
  select(geo_code1, geo_code2, all, bicycle, car_driver)
l_top_cycling = l %>% 
  top_n(n = 5, wt = bicycle)
plot(z$geometry)
plot(l_top_cycling, add = TRUE, lwd = 5, col = "green")

# top 5 driving routes
l_top_driving = l %>% 
  top_n(n = 5, wt = car_driver)
plot(z$geometry)
plot(l_top_driving, add = TRUE, lwd = 5, col = "red")
```

- G7: Why are the results different? What are the advantages and disadvantages of using smaller zones, as represented by the LSOA data above?

- G8 (bonus): do the same analysis but with the top **300** routes cycled and driven. Hint: set the line width with `lwd = l_top_cycling$bicycle / mean(l_top_cycling$bicycle)` to portray the relative importance of each route.

```{r, echo=FALSE, warning=FALSE, fig.show='hold', fig.cap="Top 300 LSOA-LSOA desire lines with highest number of people cycling (left) and driving (right) in the Isle of Wight."}
# at the lsoa level
l_top_cycling = l %>% 
  top_n(n = 300, wt = bicycle)
plot(z$geometry)
plot(l_top_cycling, add = TRUE, lwd = l_top_cycling$bicycle / mean(l_top_cycling$bicycle), col = "green")

# top 5 driving routes
l_top_driving = l %>% 
  top_n(n = 300, wt = car_driver)
plot(z$geometry)
plot(l_top_driving, add = TRUE, lwd = l_top_driving$car_driver / mean(l_top_driving$car_driver), col = "red")
```

## Modifying PCT data to identify routes/roads of interest

- M1: Building on the example above, add a new column called `pcycle` to the object `l_msoa` that contains the % who cycle to work (hint: you might want to start this by typing `l_msoa$pcycle = ...`) and plot the results (shown in left hand panel in plot below).

```{r p2}
l_msoa$pcycle = l_msoa$bicycle / l_msoa$all * 100
# plot(l_msoa["pcycle"], lwd = l_msoa$all / mean(l_msoa$all), breaks = c(0, 5, 10, 20, 50))
```

- M2 (bonus): identify road segments with the highest estimated number of people cycling currently, and under the Go Dutch scenario (hint: you can download the route network with `get_pct_rnet("isle-of-wight")`)

```{r eval=FALSE, echo=FALSE}
rnet = get_pct_rnet("isle-of-wight")
```

## Scenarios of change

- S1: Generate a 'Go Dutch' scenario for the Isle of Wight using the function `uptake_pct_godutch()` (hint: the following code chunk will create a 'Government Target' scenario):

```{r}
l_msoa$euclidean_distance = as.numeric(sf::st_length(l_msoa))
l_msoa$pcycle_govtarget = uptake_pct_govtarget(
  distance = l_msoa$rf_dist_km,
  gradient = l_msoa$rf_avslope_perc
  ) * 100 + l_msoa$pcycle
```

```{r change, echo=FALSE}
l_msoa$pcycle_dutch = uptake_pct_godutch(
  distance = l_msoa$rf_dist_km,
  gradient = l_msoa$rf_avslope_perc
  ) * 100 + l_msoa$pcycle
```

```{r dutch_pcycle, echo=FALSE, warning=FALSE, fig.show='hold', fig.cap="Percent cycling currently (left) and under a 'Go Dutch' scenario (right) in the Isle of Wight."}
plot(l_msoa["pcycle"], lwd = l_msoa$all / mean(l_msoa$all), breaks = c(0, 5, 10, 20, 50))
plot(l_msoa["pcycle_dutch"], lwd = l_msoa$all / mean(l_msoa$all), breaks = c(0, 5, 10, 20, 50))
# cor(l_original_msoa$dutch_slc / l_original_msoa$all, l_msoa$pcycle_dutch)
# cor(l_original_msoa$govtarget_slc / l_original_msoa$all, l_msoa$pcycle_govtarget)
# plot(l_original_msoa$dutch_slc / l_original_msoa$all, l_msoa$pcycle_dutch)
```

- S2: Think of alternative scenarios that would be useful for your work
- S3 (bonus): look inside the function [`pct_uptake_godutch()`](https://github.com/ITSLeeds/pct/blob/master/R/uptake.R#L36) - how could it be modified?

## Routing

- R1: Using the function `route_osrm()` find the route associated with the most cycled desire line in the Isle of Wight. The result should look similar to that displayed in the map below (hint: you may want to start your answer with the following lines of code - **warning: the function may need to run a few times before it works**):

```{r}
library(stplanr)
l_top = l_msoa %>% 
  top_n(n = 1, wt = bicycle)
```

```{r, eval=FALSE, echo=FALSE}
r_top = stplanr::route_osrm(l_top)
sf::write_sf(sf::st_as_sf(r_top), "r_top.geojson")
piggyback::pb_upload("r_top.geojson")
piggyback::pb_download_url()
```

```{r, echo=FALSE}
r_top = sf::read_sf("https://github.com/ITSLeeds/pct/releases/download/0.0.1/r_top.geojson")
tm_shape(r_top) +
  tm_lines(lwd = 5)
```

- R2: What are the problems associated with this route from a cycling perspective? Take a look at the help page opened by entering `?route_osrm` to identify the reason why the route is not particularly useful from a cycling perspective.

- R3: Regenerate the route using the function `line2route()`. What is the difference in the length between each route, and what other differences can you spot? **Note: this exercise requires an API Key from CycleStreets.net.**

```{r, echo=FALSE, eval=FALSE}
r_cs = stplanr::line2route(l_top)
leaflet() %>% 
  addTiles() %>% 
  addPolylines(data = r_cs)
```

- R4 (bonus): what features of a routing service would be most useful for your work and why? 

## Route networks

- RN1: Generate a 'route network' showing number of people walking in the top 30 routes in the Isle of Wight, allocated to the transport network (hint: use the `overline2()` function and begin the script as follows, the results should look similar to the results below):

```{r}
route_data = sf::st_sf(wight_lines_30, geometry = wight_routes_30$geometry)
```

```{r, echo=FALSE, message=FALSE}
rnet_walk = overline2(x = route_data, "foot")
tm_shape(rnet_walk) +
  tm_lines(lwd = "foot", scale = 9)
```

- RN2: Download the travel to school route network and compare the results with the route network created for RN1.
  - Which roads have greatest overlap between the two route networks?
  - For more information on the travel to school layer, see @goodman_scenarios_2019. What other trip purposes would you like to see in tools for cycle planning?

```{r, eval=FALSE, echo=FALSE}
rnet_school = get_pct_rnet(region = "isle-of-wight", purpose = "school")
plot(rnet_school)
```

```{r, echo=FALSE, eval=FALSE}
# Demo PCT Analysis#
# Make a commuting quiet route network for Isle of Wight
# and combine it with the travel to school route network

# Step 1: Load Library
library(tidyverse)
library(sf)
library(pct)
library(stplanr)

# Step 2: Get Data
routes_commute = get_pct_routes_quiet(region = "isle-of-wight",
                              purpose = "commute",
                              geography = "lsoa")

lines_commute = get_pct_lines(region = "isle-of-wight",
                              purpose = "commute",
                              geography = "lsoa")

rnet_school = get_pct_rnet(region = "isle-of-wight",
                           purpose = "school",
                           geography = "lsoa")

# Step 3: Prepare Data
lines_commute = lines_commute %>%
  st_drop_geometry() %>%
  select(id, bicycle, dutch_slc)

routes_commute = routes_commute %>%
  select(id)

# Join Cycling Levels to Routes
routes_commute = left_join(routes_commute, lines_commute)
plot(routes_commute["bicycle"])

# Make a commuting Rnet
rnet_commute = overline2(routes_commute, 
                         attrib = c("bicycle","dutch_slc"))
plot(rnet_commute["bicycle"])

# Combine commuting and travel to schools
rnet_school = rnet_school %>%
  select(dutch_slc)
rnet_commute = rnet_commute %>%
  select(dutch_slc)
rnet_commute$bicycle = NULL


rnet_both = rbind(rnet_commute, rnet_school)
rnet_both = overline2(rnet_both, 
                         attrib = c("dutch_slc"))
mapview::mapview(rnet_both, at = c(50,100,200,500,1000))

```

## Ideas for further work and a 'minihack'

- Create a route network reflecting where you would invest if the priority was reducing car trips of less than 5 km
- Design interventions to replace short car trips across London (or another region of your choice) using the PCT methods/data to support your decisions
- Identify quiet routes and design a quiet route network for city/region of your choice, e.g. Westminter
- Import alternative origin-destination datasets and use those as the basis for propensity to cycle analysis for trip purposes other than single stage commutes, encapsulated in the commut layer in the PCT
- Any other layers/scenarios/hacks: welcome! Comments in this repo's [issue tracker](https://github.com/ITSLeeds/pct/issues) also welcome.

<!-- - Identify priority routes to improve for bike share users by using OD data from TfL's bike share (if we get time to do this) -->

<!-- Demonstrate routing with cyclestreets.net -->

# References
