Advanced chunks

Decoupling chunks containers

In a shiny app there is often a need to have multiple chunks containers that may be used independently by certain parts of the application and may be used together by other parts of the application. The ability to have multiple chunks containers combined into one allows for greater flexibility of the code, which allows the developer to apply the DRY principle (stands for Do not repeat which means code that are used in multiple places should only be written and defined once) and optimize performance. Much of the functionality to use a single chunks container object independently has already been covered in the Basic chunks vignette, which you should read first before going any further.

Creating a chunks container and assigning it to a variable

In order to have multiple chunks containers, each must be accessible and distinguishable via a variable.

ch_0 <- teal.code::chunks_new()
ch_1 <- teal.code::chunks_new()
ch_2 <- teal.code::chunks_new()

# if data used is static, then it should be computed once
# isolating it into its own chunks container accomplishes this goal
ch_0$push(quote(df <- data.frame(a = runif(5, -10, 10), b = runif(5, -10, 10))))
ch_1$push(quote({
  Sys.sleep(1) # some really long running process
  foo_plot <- function(df_arg) plot(df_arg)
}))
ch_2$push(quote({
  Sys.sleep(1) # some really long running process
  foo_print <- function(df_arg) df_arg
}))

# chunks containers should be evaluated before being merged
# the reason for this will be explain later
teal.code::chunks_safe_eval(chunks = ch_0)

##             a          b
## 1  1.78274547 -1.5840153
## 2  9.61923385 -7.1294898
## 3  1.41337177  0.5395041
## 4 -0.05242216  5.5314265
## 5  0.10755724 -4.2628728

teal.code::chunks_safe_eval(chunks = ch_1)

## function(df_arg) plot(df_arg)
## <environment: 0x55c58bd3cb28>

teal.code::chunks_safe_eval(chunks = ch_2)

## function(df_arg) df_arg
## <environment: 0x55c58be50210>

The teal.code::chunks_push_chunks function is called to combine chunks containers together. Note that the base chunks container to start combining from in the following examples are all empty. However, that does not have to be the case. Starting with a clean slate is easier to comprehend.

Perhaps a part of your shiny application needs to output a plot

ch_plot <- teal.code::chunks_new()
teal.code::chunks_push_chunks(chunks = ch_plot, x = ch_0)
teal.code::chunks_push_chunks(chunks = ch_plot, x = ch_1)
teal.code::chunks_push(chunks = ch_plot, expression = quote(foo_plot(df)))
teal.code::chunks_safe_eval(chunks = ch_plot)

## NULL

Perhaps a part of your shiny application needs to output a table

ch_table <- teal.code::chunks_new()
teal.code::chunks_push_chunks(chunks = ch_table, x = ch_0)
teal.code::chunks_push_chunks(chunks = ch_table, x = ch_2)
teal.code::chunks_push(chunks = ch_table, expression = quote(foo_print(df)))
teal.code::chunks_safe_eval(chunks = ch_table)

##             a          b
## 1  1.78274547 -1.5840153
## 2  9.61923385 -7.1294898
## 3  1.41337177  0.5395041
## 4 -0.05242216  5.5314265
## 5  0.10755724 -4.2628728

Perhaps the main part of your shiny application needs to output a plot, a table, and the code to generate both

ch_ouput <- teal.code::chunks_new()
teal.code::chunks_push_chunks(chunks = ch_ouput, x = ch_0)
teal.code::chunks_push_chunks(chunks = ch_ouput, x = ch_1)
teal.code::chunks_push_chunks(chunks = ch_ouput, x = ch_2)
teal.code::chunks_push(chunks = ch_ouput, expression = quote(foo_plot(df)))
teal.code::chunks_push(chunks = ch_ouput, expression = quote(foo_print(df)))
teal.code::chunks_safe_eval(chunks = ch_ouput)

##             a          b
## 1  1.78274547 -1.5840153
## 2  9.61923385 -7.1294898
## 3  1.41337177  0.5395041
## 4 -0.05242216  5.5314265
## 5  0.10755724 -4.2628728

code <- teal.code::chunks_get_rcode(chunks = ch_ouput)
cat(
  paste(code, collapse = "\n")
)

## df <- data.frame(a = runif(5, -10, 10), b = runif(5, -10, 10))
## Sys.sleep(1)
## foo_plot <- function(df_arg) plot(df_arg)
## Sys.sleep(1)
## foo_print <- function(df_arg) df_arg
## foo_plot(df)
## foo_print(df)

When chunks containers are merged together, their code expressions are combined in the order that the containers were merged and their environments are combined into a single environment.

Even in the event that the expressions that are combined will lead to errors being thrown when evaluated, there is no way for the chunks object to know that before evaluation. Hence code expressions will not interrupt the merging of chunks containers together.

However, the potential conflict of when two chunks containers have the same variables which holds different values in their respective environments will throw an error when merging chunks containers.

ch_0 <- teal.code::chunks_new()
ch_1 <- teal.code::chunks_new()

ch_0$push(quote(a_var <- 0))
ch_1$push(quote(a_var <- 0))

teal.code::chunks_safe_eval(chunks = ch_0)

## [1] 0

teal.code::chunks_safe_eval(chunks = ch_1)

## [1] 0

# OK - operation is successful as the variable a_var holds the same value in both chunks
teal.code::chunks_push_chunks(chunks = ch_0, x = ch_1)

ch_2 <- teal.code::chunks_new()
ch_2$push(quote(a_var <- 1))
teal.code::chunks_safe_eval(chunks = ch_2)

## [1] 1

# ERROR - operation is rejected as the variable a_var holds different values
tryCatch(
  teal.code::chunks_push_chunks(chunks = ch_0, x = ch_2),
  error = function(e) e
)

## <simpleError in chunks$push_chunks(x = x, overwrite = overwrite): chunks_push_chunks does not allow overwriting already calculated values.
##  Following variables would have been overwritten:
##     - a_var>

Why should you evaluate before merging?

With the exception of code that is meant to return something at the end of a function, code usually modifies the environment and / or the variables inside the environment.

When chunks containers have code that have not been evaluated, these code will not block any merging operations as explained above. But if they are evaluated, then their environment will be updated and will reflect the intended state of the chunks container as described by the code expressions.

And as described above, it is the environment that will block the operation to merge chunks containers together when there are conflicts. So evaluating before merging will prevent some bugs in your app.

ch_0 <- teal.code::chunks_new()
ch_0$push(quote(a_var <- 0))
teal.code::chunks_safe_eval(chunks = ch_0)

## [1] 0

ch_2 <- teal.code::chunks_new()
ch_2$push(quote(a_var <- 1))

# note that ch_2 is not evaluated

# OK - operation is successful
teal.code::chunks_push_chunks(chunks = ch_0, x = ch_2)

Cloning a chunks container

A chunks container is an R6 reference class, so by default contains a clone method. A deep copy is needed if some part of your app needs to start from the state of an already existing chunks container and modify or add on to it, but some other part of your app still needs the original chunks container unaltered.

The example code below is another way to print the plot above, which starts combining chunks from a non-empty chunks container derived from cloning another chunks container

ch_0 <- teal.code::chunks_new()
ch_0$push(quote({
  df <- data.frame(a = runif(5, -10, 10), b = runif(5, -10, 10))
  foo_plot <- function(df_arg) plot(df_arg)
}))
teal.code::chunks_safe_eval(chunks = ch_0)

## function(df_arg) plot(df_arg)
## <environment: 0x55c58cf85370>

ch_plot_clone <- teal.code::chunks_deep_clone(ch_0)
teal.code::chunks_push(chunks = ch_plot_clone, expression = quote(foo_plot(df)))
teal.code::chunks_safe_eval(chunks = ch_plot_clone)

## NULL

Extracting values of variables in a chunks container

In order to extract values of variables inside a chunks container, the code that generates that variable must have been evaluated

ch_0 <- teal.code::chunks_new()
ch_0$push(quote(df <- data.frame(a = runif(5, -10, 10), b = runif(5, -10, 10))))
teal.code::chunks_safe_eval(chunks = ch_0)

##           a         b
## 1  7.440171  1.849777
## 2  8.350948  8.888268
## 3 -4.277250  3.613847
## 4 -6.229418 -7.860479
## 5  2.300668 -2.137086

env <- new.env()
env$df <- teal.code::chunks_get_var(var = "df", chunks = ch_0)

# Once extracted, the value can be passed into other chunks containers if needed

ch_print_df <- teal.code::chunks_new()
teal.code::chunks_reset(envir = env, chunks = ch_print_df)
teal.code::chunks_push(chunks = ch_print_df, expression = quote(df))
teal.code::chunks_safe_eval(chunks = ch_print_df)

##           a         b
## 1  7.440171  1.849777
## 2  8.350948  8.888268
## 3 -4.277250  3.613847
## 4 -6.229418 -7.860479
## 5  2.300668 -2.137086

Example

The following is a shiny app that uses the chunks container object in ways that mirror how a sophistical teal app would use it. This app constructs several chunks containers, allowing each to be used and combined by different parts of the app, which enables the app to apply the DRY principle and optimize performance.

library(shiny)
library(teal.code)
library(dplyr)
library(rlang)

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      # Input of Response can be chosen from Event Table
      selectInput(
        inputId = "filter_paramcd",
        label = "Filter Paramcd",
        choices = c("OS", "EFS", "PFS"),
        selected = "OS",
        multiple = TRUE
      ),
      selectInput(
        inputId = "filter_sex",
        label = "Filter sex",
        choices = c("F", "M", "U"),
        selected = c("F", "M", "U"),
        multiple = TRUE
      ),
      selectInput(
        inputId = "anl_columns",
        label = "ANL columns",
        choices = NULL,
        multiple = TRUE
      )
    ),
    mainPanel(
      verbatimTextOutput("code"),
      plotOutput("plot"),
      DT::DTOutput("anl_table")
    )
  )
)

server <- function(input, output, session) {
  # chunks container to hold the datasets used in this app.
  # The purpose of storing the code to generate the datasets is reproducibility.
  data_def_chunks <- chunks_new()
  chunks_push(
    chunks = data_def_chunks,
    expression = quote({
      library(shiny)
      library(teal.code)
      library(dplyr)
      library(rlang)
    })
  )
  chunks_push(
    chunks = data_def_chunks,
    expression = quote(
      adsl <- list(
        SUBJID = 1:100,
        STUDYID = c(rep(1, 20), rep(2, 50), rep(3, 30)),
        AGE = sample(20:88, 100, replace = T) %>% as.numeric(),
        SEX = sample(c("M", "F", "U"), 100, replace = T) %>% as.factor()
      ) %>%
        as_tibble()
    )
  )
  chunks_push(
    chunks = data_def_chunks,
    expression = quote(
      events <- list(
        SUBJID = rep(1:100, 3),
        STUDYID = rep(c(rep(1, 20), rep(2, 50), rep(3, 30)), 3),
        PARAMCD = c(rep("OS", 100), rep("EFS", 100), rep("PFS", 100)),
        AVAL = c(rexp(100, 1 / 100), rexp(100, 1 / 80), rexp(100, 1 / 60)) %>% as.numeric(),
        AVALU = rep("DAYS", 300) %>% as.factor()
      ) %>%
        as_tibble()
    )
  )
  # evaluating the code so that the datasets become available
  chunks_safe_eval(data_def_chunks)

  # reactive that needs to extract a value from another chunks container
  adsl_filtered_reactive <- reactive({
    env <- new.env()
    env$adsl <- chunks_get_var(var = "adsl", chunks = data_def_chunks)
    ch <- chunks_new()
    chunks_reset(chunks = ch, env = env)
    chunks_push(
      chunks = ch,
      expression = expr(adsl_filtered <- adsl %>% dplyr::filter(SEX %in% !!input$filter_sex))
    )
    chunks_safe_eval(ch)
    ch
  })

  # reactive that needs to extract a value from another chunks container
  events_filtered_reactive <- reactive({
    env <- new.env()
    env$events <- chunks_get_var(var = "events", chunks = data_def_chunks)
    ch <- chunks_new()
    chunks_reset(chunks = ch, env = env)
    chunks_push(
      chunks = ch,
      expression = expr(events_filtered <- events %>% dplyr::filter(PARAMCD %in% !!input$filter_paramcd))
    )
    chunks_safe_eval(ch)
    ch
  })

  # part of the app that merges two other chunks containers
  merge_data <- reactive({
    ch <- chunks_new()

    chunks_push_chunks(chunks = ch, x = adsl_filtered_reactive())
    chunks_push_chunks(chunks = ch, x = events_filtered_reactive())

    chunks_push(
      chunks = ch,
      expression = expr(anl <- left_join(adsl_filtered, events_filtered, by = c("STUDYID", "SUBJID")))
    )
    chunks_safe_eval(ch)
    ch
  })

  observeEvent(merge_data(), {
    anl <- chunks_get_var("anl", merge_data())
    updateSelectInput(
      session,
      "anl_columns",
      choices = colnames(anl),
      selected = colnames(anl)
    )
  })

  # need to clone as chunks are R6 object - this would change ch in merge_data()
  subset_anl <- reactive({
    ch <- chunks_deep_clone(merge_data())
    chunks_push(
      chunks = ch,
      expression = rlang::expr(anl_subset <- anl[!!input$anl_columns])
    )
    chunks_safe_eval(ch)
    ch
  })

  # a chunks container that needs to extract a value from another chunks container
  table_call <- reactive({
    env <- new.env()
    env$anl_subset <- chunks_get_var(chunks = subset_anl(), var = "anl_subset")
    ch <- chunks_new()
    chunks_reset(chunks = ch, env = env)
    chunks_push(
      chunks = ch,
      expression = rlang::expr({
        table <- DT::datatable(anl_subset)
        table
      })
    )
    chunks_safe_eval(ch)
    ch
  })

  # a chunks container that needs to extract a value from another chunks container
  plot_call <- reactive({
    env <- new.env()
    env$anl_subset <- chunks_get_var(chunks = subset_anl(), var = "anl_subset")
    ch <- chunks_new()
    chunks_reset(chunks = ch, env = env)
    chunks_push(
      chunks = ch,
      expression = rlang::expr({
        plot_output <- plot(anl_subset$AGE, anl_subset$AVAL)
        plot_output
      })
    )
    chunks_safe_eval(ch)
    ch
  })

  # Really long running process that should not be recomputed if can be avoided.
  #
  # The ability to put this process in its own chunks container, independent of the rest,
  # allows the app to avoid re-computation and yet be able to capture its code to ensure reproducibility.
  sleep_5_seconds <- reactive({
    ch <- chunks_new()
    chunks_push(
      chunks = ch,
      expression = quote(Sys.sleep(5))
    )
    chunks_safe_eval(ch)
    ch
  })

  # part of the app that needs to combine all chunks containers
  outputs_call <- reactive({
    ch <- chunks_new()
    chunks_push_chunks(chunks = ch, x = data_def_chunks)
    chunks_push_chunks(chunks = ch, x = sleep_5_seconds())
    chunks_push_chunks(chunks = ch, x = subset_anl())
    chunks_push_chunks(chunks = ch, x = plot_call())
    chunks_push_chunks(chunks = ch, x = table_call())
    ch
  })

  # part of the app that only needs a single chunks container
  output$anl_table <- DT::renderDT(chunks_get_var("table", table_call()))
  output$plot <- renderPlot(chunks_get_var("plot_output", plot_call()))

  output$code <- renderPrint({
    cat(
      paste(
        chunks_get_rcode(chunks = outputs_call()),
        collapse = "\n"
      )
    )
  })
}

shinyApp(ui, server)

Conclusion

The ability to have multiple chunks containers combined into one allows for greater flexibility of the code, which allows the developer to apply the DRY principle and optimize performance.

NEST coreDev