Filter panel for NEST developers

FilteredData

FilteredData is exposed to the apps/modules developer as a dataset argument in the modules. FilteredData manages filter panel by returning filtered data, combining reproducible filter calls from FilteredDatasets. FilteredData also contains all shiny modules displayed on the right panel in teal apps. FilteredData is a single object which is initialized in srv_teal module and its variant depends on TealData. If DDL returns CDISCTealData then CDISCFilteredData is initialized, otherwise FilteredData.

In the analytical modules datasets can be used to:

• obtain filtered and unfiltered data using datasets$get_data(<dataname>, filtered = <TRUE/FALSE>).
• get available datanames using datasets$datanames()
• get reproducible filter call using datasets$get_call(<dataname>)
• get reproducible data loading call datasets$get_code(<dataname>)
• get JoinKeys between two datasets using datasets$get_join_keys(<dataname1>, <dataname2>)
• get variable labels using datasets$get_varlabels(<dataname>)
• get parent dataset name using datasets$get_parentname(<dataname>) (only in case of CDISCFilteredData)

FilteredDataset

FilteredDataset is a class which keeps unfiltered data and returns filtered data based on the filter call derived from FilterStates. FilteredDataset class objects are initialized by FilteredData, one for each TealDataset. FilteredDataset contains single TealDataset object and one-to-many FilterStates depending on the type of object. FilteredDataset stores dataset attributes, joins keys to other datasets, and also combines and executes the code taken from FilterStates.

Following FilteredDataset derived classes are already implemented:

• DefaultFilteredDataset dispatched by TealDataset to manage filters for data.frame object.
• CDISCFilteredDataset dispatched by CDISCTealDataset to manage filters for data.frame matching ADAM standards.
• MAEFilteredDataset dispatched by MAETealDataset to manage filters for MultiAssayExperiment object.

FilterStates

FilterStates are initialized by the FilteredDataset when teal app starts. The type and number of FilterStates depends on the type of data included in TealDataset. If data in FilteredDataset is composed of multiple objects then the equivalent number of FilterStates is initialized. One FilterStates object is responsible to make one subset call. Consider the case of MultiAssayExperiment object which contains multiple experiments and patients data stored in separate slots. Each of the objects within MultiAssayExperiment can be filtered by separate calls. Each sub-element in TealDataset needs also separate inputs to select subset variables, which then should be applied to the same sub-element of the TealDataset.

Currently following variants are possible:

• DFFilterStates dispatched by data.frame, uses data.frame columns for filtering.
• MAEFilterStates dispatched by MultiAssayExperiment, uses columns of object kept in @colData slot for filtering.
• SEFilterStates dispatched by SummarizedExperiment, uses columns of objects kept in @colData and @rowData for filtering.
• MatrixFilterStates dispatched by matrix, uses matrix columns for filtering.

FilterStates serves two Shiny related purposes:

• ui/srv_add_filter_state allow to add FilterState for selected variable. Variables included in the module are the filterable colnames of the provided dataset. Variable selection adds FilterState to ReactiveQueue (stored in list private$queue[[queue_id]]). FilterState is dispatched automatically on a selected column class.

FilterState

This class controls single filter card and returns condition call depending on what is selected. FilterState is initialized each time when a user selects filter variable in FilterState$add_filter_variable module. FilterState is dispatched based on the type of the selected variable. Depending on the type of the filter state, there are different UI inputs - for example only numeric has use_inf checkbox button. private$selected, private$use_na, private$use_inf are reactive values and they trigger re-execution of FilterState$get_call whenever their values change. The constructor of FilterState has extract_type argument which impacts returned call. extract_type can be unspecified, "matrix" or "list" and its value corresponds to the type of the variable prefix in the returned condition call. For example if FilterState is initialized with extract_type = "matrix" then the variable in the condition call looks like <input_dataname>[, "<varname>"].

Overview

Above diagram presents the filter panel classes and their responsibilities when composing filter calls.

• FilterState$get_call() returns a single condition call based on single variable
• ReactiveQueue is a container which stores multiple condition calls. ReactiveQueue objects are kept in a list where the names of elements are set after the argument names.
• FilterStates$get_call() returns a single filter call by gathering conditions returned from FilterState and combining them by & operator grouped by argument name.
• FilteredDataset$get_call() returns the list of calls taken from FilterStates object(s).
• FilteredData$get_call(<dataname>) returns the list of calls from specified FilteredDataset.

Example

Calling datasets$get_call(<dataname>) in teal modules executes a chain of calls in all filter panel classes. Consider a scenario in which:

1. FilteredData has three FilteredDataset(s) ADSL , ADTTE, MAE

2. CDISCFilteredDataset contains data.frame (ADSL) which can be filtered only in one way executing single dplyr::filter call (this is why CDISCFilteredDataset has a single FilterStates)

3. FilterStates construct dplyr::filter call is based on the FilterState objects added to the ReactiveQueue.

4. ReactiveQueue is just a class to manage addition, storage and removal of FilterStates objects. This is its only responsibility and the class does not generate any code. DFFilterState contains only one ReactiveQueue which is not named - this tells FilterState that calls from each FilterState should go to unnamed argument in dplyr::filter.

5. When the end-user chooses some variable in “Add Filter Variable” section, then new FilterState is added to the ReactiveQueue and this new condition is added to dplyr::filter(ADSL, ...) call. In the example SEX and AGE has been added - which are automatically dispatched to relevant FilterState class (ChoiceFilterState and RangeFilterState). Since (5) have been added or changed, conditions (SEX == "F" and AGE >= 20 & AGE <= 60) are returned to the DFFilterStates (3) which combines them with & operator and puts to dplyr::filter call. CDISCFilteredDataset takes this one call and return to FilteredData as a list.

6. Second FilteredDataset for ADTTE works the same way as ADSL with one difference. dplyr::filter for ADTTE is followed by the merge call with FILTERED_ADSL - to be filtered by keys available parent.

7. FilteredDatasetMAE is based on MAETealDataset where raw_data contains multiple objects which can be filtered on. In general MultiAssayExperiment contains colData(MAE) which is a DataFrame with ADSL-like patient data. MAE contains also multiple experiments which can be extracted using MAE[["experiment name"]] and they can also be filtered in filter-panel. This means that FilteredDatasetMAE has multiple FilterStates objects: one for subjects data and one for each experiment.

8. MAEFilterStates object is initialized for subjects data and for this object SummarizedExperiment::subsetByColData function is applied. SummarizedExperiment::subsetByColData has two arguments x (data) and y (conditions). MAEFilterStates similar to DFFilterStates has one ReactiveQueue (9) list for y argument in the function. Adding new FilterStates triggers returning of the code similar to (4) and (5)

9. SEFilterStates is initialized per one experiment in the MAE data. This class is specific because SummarizedExperiment contains colData and rowData which correspond to select and subset arguments in subset.SummarizedExperiment function. Similar to subset.data.frame subset refers to the rows in the object, and select refers to the columns - but the call execution is little different because subsetting and selecting call refers to columns in colData and rowData (objects attached to this experiment).