# Questions and answers from over the years

# How could I generate a manifest file with filtering of Race and Ethnicity?

From https://support.bioconductor.org/p/9138939/.

    library(GenomicDataCommons,quietly = TRUE)

I made a small change to the filtering expression approach based on changes to lazy evaluation best practices. There is now no need to include the `~` in the filter expression. So:

    q = files() |>
      GenomicDataCommons::filter(
        cases.project.project_id == 'TCGA-COAD' &
          data_type == 'Aligned Reads' &
          experimental_strategy == 'RNA-Seq' &
          data_format == 'BAM')

And get a count of the results:

    count(q)

    ## [1] 1188

And the manifest.

    manifest(q)

    ## # A tibble: 1,188 × 26
    ##    id    proportion_reads_map…¹ access wgs_coverage proportion_base_mism…² acl_1
    ##    <chr>                  <dbl> <chr>  <chr>                         <dbl> <chr>
    ##  1 28c9…                  0.988 contr… Not Applica…                0.00223 phs0…
    ##  2 72a4…                  0.985 contr… Not Applica…                0.00514 phs0…
    ##  3 1347…                 NA     contr… Not Applica…               NA       phs0…
    ##  4 1db6…                  0.980 contr… Not Applica…                0.00300 phs0…
    ##  5 210c…                 NA     contr… Not Applica…               NA       phs0…
    ##  6 91b1…                 NA     contr… Not Applica…               NA       phs0…
    ##  7 dfc2…                  0.978 contr… Not Applica…                0.00330 phs0…
    ##  8 69b9…                  0.961 contr… Not Applica…                0.00317 phs0…
    ##  9 3f50…                 NA     contr… Not Applica…               NA       phs0…
    ## 10 c720…                  0.978 contr… Not Applica…                0.00517 phs0…
    ## # ℹ 1,178 more rows
    ## # ℹ abbreviated names: ¹​proportion_reads_mapped, ²​proportion_base_mismatch
    ## # ℹ 20 more variables: type <chr>, platform <chr>, created_datetime <chr>,
    ## #   md5sum <chr>, updated_datetime <chr>, pairs_on_diff_chr <int>, state <chr>,
    ## #   data_format <chr>, total_reads <int>, file_name <chr>,
    ## #   proportion_reads_duplicated <int>, submitter_id <chr>, data_category <chr>,
    ## #   file_size <dbl>, average_base_quality <int>, file_id <chr>, …

Your question about race and ethnicity is a good one.

    all_fields = available_fields(files())

And we can grep for `race` or `ethnic` to get potential matching fields to look at.

    grep('race|ethnic',all_fields,value=TRUE)

    ## [1] "cases.demographic.ethnicity"
    ## [2] "cases.demographic.race"
    ## [3] "cases.follow_ups.other_clinical_attributes.hormonal_contraceptive_type"
    ## [4] "cases.follow_ups.other_clinical_attributes.hormonal_contraceptive_use"
    ## [5] "cases.follow_ups.scan_tracer_used"

Now, we can check available values for each field to determine how to complete our filter expressions.

    available_values('files',"cases.demographic.ethnicity")

    ## [1] "not hispanic or latino" "not reported"           "hispanic or latino"
    ## [4] "unknown"                "_missing"

    available_values('files',"cases.demographic.race")

    ##  [1] "white"
    ##  [2] "not reported"
    ##  [3] "black or african american"
    ##  [4] "asian"
    ##  [5] "unknown"
    ##  [6] "american indian or alaska native"
    ##  [7] "native hawaiian or other pacific islander"
    ##  [8] "other"
    ##  [9] "not allowed to collect"
    ## [10] "_missing"

We can complete our filter expression now to limit to `white` race only.

    q_white_only = q |>
      GenomicDataCommons::filter(cases.demographic.race=='white')
    count(q_white_only)

    ## [1] 695

    manifest(q_white_only)

    ## # A tibble: 695 × 26
    ##    id    proportion_reads_map…¹ access wgs_coverage proportion_base_mism…² acl_1
    ##    <chr>                  <dbl> <chr>  <chr>                         <dbl> <chr>
    ##  1 72a4…                  0.985 contr… Not Applica…                0.00514 phs0…
    ##  2 1347…                 NA     contr… Not Applica…               NA       phs0…
    ##  3 1db6…                  0.980 contr… Not Applica…                0.00300 phs0…
    ##  4 210c…                 NA     contr… Not Applica…               NA       phs0…
    ##  5 dfc2…                  0.978 contr… Not Applica…                0.00330 phs0…
    ##  6 3f50…                 NA     contr… Not Applica…               NA       phs0…
    ##  7 8a26…                  0.987 contr… Not Applica…                0.00964 phs0…
    ##  8 a55b…                 NA     contr… Not Applica…               NA       phs0…
    ##  9 dee5…                  0.980 contr… Not Applica…                0.00295 phs0…
    ## 10 46d6…                 NA     contr… Not Applica…               NA       phs0…
    ## # ℹ 685 more rows
    ## # ℹ abbreviated names: ¹​proportion_reads_mapped, ²​proportion_base_mismatch
    ## # ℹ 20 more variables: type <chr>, platform <chr>, created_datetime <chr>,
    ## #   md5sum <chr>, updated_datetime <chr>, pairs_on_diff_chr <int>, state <chr>,
    ## #   data_format <chr>, total_reads <int>, file_name <chr>,
    ## #   proportion_reads_duplicated <int>, submitter_id <chr>, data_category <chr>,
    ## #   file_size <dbl>, average_base_quality <int>, file_id <chr>, …

# How can I get the number of cases with RNA-Seq data added by date to TCGA project with `GenomicDataCommons`?

- From https://support.bioconductor.org/p/9135791/

I would like to get the number of cases added (created, any logical datetime would suffice here) to the TCGA project by experiment type. I attempted to get this data via GenomicDataCommons package, but it is giving me I believe the number of files for a given experiment type rather than number cases. How can I get the number of cases for which there is RNA-Seq data?

    library(tibble)
    library(dplyr)

    ##
    ## Attaching package: 'dplyr'

    ## The following objects are masked from 'package:GenomicDataCommons':
    ##
    ##     count, filter, select

    ## The following objects are masked from 'package:stats':
    ##
    ##     filter, lag

    ## The following objects are masked from 'package:base':
    ##
    ##     intersect, setdiff, setequal, union

    library(GenomicDataCommons)

    cases() |>
      GenomicDataCommons::filter(
        ~ project.program.name=='TCGA' & files.experimental_strategy=='RNA-Seq'
      ) |>
      facet(c("files.created_datetime")) |>
      aggregations() |>
      unname() |>
      unlist(recursive = FALSE) |>
      as_tibble() |>
      dplyr::arrange(dplyr::desc(key))

    ## # A tibble: 200 × 2
    ##    doc_count key
    ##        <int> <chr>
    ##  1       271 2024-06-14t14:27:00.916424-05:00
    ##  2       416 2024-06-14t13:28:10.644120-05:00
    ##  3       150 2024-03-11t09:00:39.229286-05:00
    ##  4       151 2023-03-09t00:35:51.387873-06:00
    ##  5        79 2023-02-19t04:41:11.008116-06:00
    ##  6       458 2023-02-19t04:36:10.605050-06:00
    ##  7        80 2023-02-19t04:28:49.400023-06:00
    ##  8       178 2023-02-19t04:23:49.092629-06:00
    ##  9       516 2023-02-19t04:18:49.453628-06:00
    ## 10       179 2023-02-19t04:13:47.877168-06:00
    ## # ℹ 190 more rows