Displaying and filtering the Annotation Hub object in a browserPackage: AnnotationHub
Authors: Martin Morgan [cre],
Marc Carlson [ctb],
Dan Tenenbaum [ctb],
Sonali Arora [ctb]
Modified: 11 October, 2014
Compiled: Thu Sep 10 21:01:16 2015
The AnnotationHub server provides easy R / Bioconductor access to
large collections of publicly available whole genome resources,
e.g,. ENSEMBL genome fasta or gtf files, UCSC chain resources, ENCODE
data tracks at UCSC, etc.
The AnnotationHub package provides a client interface to resources stored at the AnnotationHub web service.
library(AnnotationHub)
The AnnotationHub package is straightforward to use.
Create an AnnotationHub object
ah = AnnotationHub()
Now at this point you have already done everything you need in order
to start retrieving annotations. For most operations, using the
AnnotationHub object should feel a lot like working with a familiar
list or data.frame.
Lets take a minute to look at the show method for the hub object ah
ah
## AnnotationHub with 35345 records
## # snapshotDate(): 2015-08-26
## # $dataprovider: BroadInstitute, UCSC, Ensembl, NCBI, Haemcode, Inparan...
## # $species: Homo sapiens, Mus musculus, Bos taurus, Pan troglodytes, Da...
## # $rdataclass: GRanges, FaFile, BigWigFile, OrgDb, ChainFile, Inparanoi...
## # additional mcols(): taxonomyid, genome, description, tags,
## # sourceurl, sourcetype
## # retrieve records with, e.g., 'object[["AH2"]]'
##
## title
## AH2 | Ailuropoda_melanoleuca.ailMel1.69.dna.toplevel.fa
## AH3 | Ailuropoda_melanoleuca.ailMel1.69.dna_rm.toplevel.fa
## AH4 | Ailuropoda_melanoleuca.ailMel1.69.dna_sm.toplevel.fa
## AH5 | Ailuropoda_melanoleuca.ailMel1.69.ncrna.fa
## AH6 | Ailuropoda_melanoleuca.ailMel1.69.pep.all.fa
## ... ...
## AH49436 | Xiphophorus_maculatus.Xipmac4.4.2.dna_rm.toplevel.fa
## AH49437 | Xiphophorus_maculatus.Xipmac4.4.2.dna_sm.toplevel.fa
## AH49438 | Xiphophorus_maculatus.Xipmac4.4.2.dna.toplevel.fa
## AH49439 | Xiphophorus_maculatus.Xipmac4.4.2.ncrna.fa
## AH49440 | Xiphophorus_maculatus.Xipmac4.4.2.pep.all.fa
You can see that it gives you an idea about the different types of data that are present inside the hub. You can see where the data is coming from (dataprovider), as well as what species have samples present (species), what kinds of R data objects could be returned (rdataclass). We can take a closer look at all the kinds of data providers that are available by simply looking at the contents of dataprovider as if it were the column of a data.frame object like this:
unique(ah$dataprovider)
## [1] "Ensembl" "EncodeDCC"
## [3] "UCSC" "Inparanoid8"
## [5] "NCBI" "NHLBI"
## [7] "ChEA" "Pazar"
## [9] "NIH Pathway Interaction Database" "RefNet"
## [11] "Haemcode" "GEO"
## [13] "BroadInstitute" "dbSNP"
In the same way, you can also see data from different species inside the hub by looking at the contents of species like this:
head(unique(ah$species))
## [1] "Ailuropoda melanoleuca" "Anolis carolinensis"
## [3] "Bos taurus" "Caenorhabditis elegans"
## [5] "Callithrix jacchus" "Canis familiaris"
And this will also work for any of the other types of metadata present. You can learn which kinds of metadata are available by simply hitting the tab key after you type 'ah$'. In this way you can explore for yourself what kinds of data are present in the hub right from the command line. This interface also allows you to access the hub programatically to extract data that matches a particular set of criteria.
Another valuable types of metadata to pay attention to is the rdataclass.
head(unique(ah$rdataclass))
## [1] "FaFile" "GRanges" "Inparanoid8Db" "OrgDb"
## [5] "TwoBitFile" "ChainFile"
The rdataclass allows you to see which kinds of R objects the hub will return to you. This kind of information is valuable both as a means to filter results and also as a means to explore and learn about some of the kinds of annotation objects that are widely available for the project. Right now this is a pretty short list, but over time it should grow as we support more of the different kinds of annotation objects via the hub.
Now lets try getting the Chain Files from UCSC using the query and subset methods to selectively pare down the hub based on specific criteria.
The query method lets you search rows for
specific strings, returning an AnnotationHub instance with just the
rows matching the query.
From the show method, one can easily see that one of the dataprovider is UCSC and there is a rdataclass for ChainFile
One can get chain files for Drosophila melanogaster from UCSC with:
dm <- query(ah, c("ChainFile", "UCSC", "Drosophila melanogaster"))
dm
## AnnotationHub with 45 records
## # snapshotDate(): 2015-08-26
## # $dataprovider: UCSC
## # $species: Drosophila melanogaster
## # $rdataclass: ChainFile
## # additional mcols(): taxonomyid, genome, description, tags,
## # sourceurl, sourcetype
## # retrieve records with, e.g., 'object[["AH15102"]]'
##
## title
## AH15102 | dm3ToAnoGam1.over.chain.gz
## AH15103 | dm3ToApiMel3.over.chain.gz
## AH15104 | dm3ToDm2.over.chain.gz
## AH15105 | dm3ToDm6.over.chain.gz
## AH15106 | dm3ToDp3.over.chain.gz
## ... ...
## AH15142 | dm2ToDroVir3.over.chain.gz
## AH15143 | dm2ToDroWil1.over.chain.gz
## AH15144 | dm2ToDroYak1.over.chain.gz
## AH15145 | dm2ToDroYak2.over.chain.gz
## AH15146 | dm1ToDm2.over.chain.gz
Query has worked and you can now see that the only species present is Drosophila melanogaster.
The metadata underlying this hub object can be retrieved by you
df <- mcols(dm)
By default the show method will only display the first 5 and last 5 rows. There are already thousands of records present in the hub.
length(ah)
## [1] 35345
Lets look at another example, where we pull down only Inparanoid8 data from the hub and use subset to return a smaller base object (here we are finding cases where the genome column is set to panda).
ahs <- query(ah, c('inparanoid8', 'ailMel1'))
ahs
## AnnotationHub with 0 records
## # snapshotDate(): 2015-08-26
We can also look at the AnnotationHub object in a browser using the
display() function. We can then filter the AnnotationHub object
for chainFile_ by either using the Global search field on the top
right corner of the page or the in-column search field for rdataclass'.
d <- display(ah)
Displaying and filtering the Annotation Hub object in a browser
By default 1000 entries are displayed per page, we can change this using the filter on the top of the page or navigate through different pages using the page scrolling feature at the bottom of the page.
We can also select the rows of interest to us and send them back to
the R session using 'Return rows to R session' button ; this sets a filter
internally which filters the AnnotationHub object. The AHID'S selected
are displayed at the top of the page.
AnnotationHub to retrieve dataLooking back at our chain file example, if we are interested in the file dm1ToDm2.over.chain.gz, we can gets its metadata using
dm
## AnnotationHub with 45 records
## # snapshotDate(): 2015-08-26
## # $dataprovider: UCSC
## # $species: Drosophila melanogaster
## # $rdataclass: ChainFile
## # additional mcols(): taxonomyid, genome, description, tags,
## # sourceurl, sourcetype
## # retrieve records with, e.g., 'object[["AH15102"]]'
##
## title
## AH15102 | dm3ToAnoGam1.over.chain.gz
## AH15103 | dm3ToApiMel3.over.chain.gz
## AH15104 | dm3ToDm2.over.chain.gz
## AH15105 | dm3ToDm6.over.chain.gz
## AH15106 | dm3ToDp3.over.chain.gz
## ... ...
## AH15142 | dm2ToDroVir3.over.chain.gz
## AH15143 | dm2ToDroWil1.over.chain.gz
## AH15144 | dm2ToDroYak1.over.chain.gz
## AH15145 | dm2ToDroYak2.over.chain.gz
## AH15146 | dm1ToDm2.over.chain.gz
dm["AH15146"]
## AnnotationHub with 1 record
## # snapshotDate(): 2015-08-26
## # names(): AH15146
## # $dataprovider: UCSC
## # $species: Drosophila melanogaster
## # $rdataclass: ChainFile
## # $title: dm1ToDm2.over.chain.gz
## # $description: UCSC liftOver chain file from dm1 to dm2
## # $taxonomyid: 7227
## # $genome: dm1
## # $sourcetype: Chain
## # $sourceurl: http://hgdownload.cse.ucsc.edu/goldenpath/dm1/liftOver/dm...
## # $sourcelastmodifieddate: NA
## # $sourcesize: NA
## # $tags: liftOver, chain, UCSC, genome, homology
## # retrieve record with 'object[["AH15146"]]'
We can download the file using
dm[["AH15146"]]
## Chain of length 11
## names(11): chr2L chr2R chr3L chr3R chr4 chrX chrU chr2h chr3h chrXh chrYh
Each file is retrieved from the AnnotationHub server and the file is also cache locally, so that the next time you need to retrieve it, it should download much more quickly.
AnnotationHub objectsWhen you create the AnnotationHub object, it will set up the object
for you with some default settings. See ?AnnotationHub for ways to
customize the hub source, the local cache, and other instance-specific
options, and ?hubOption to set package-global options for use across
sessions.
If you look at the object you will see some helpful information about it such as where the data is cached and where online the hub server is set to.
ah
## AnnotationHub with 35345 records
## # snapshotDate(): 2015-08-26
## # $dataprovider: BroadInstitute, UCSC, Ensembl, NCBI, Haemcode, Inparan...
## # $species: Homo sapiens, Mus musculus, Bos taurus, Pan troglodytes, Da...
## # $rdataclass: GRanges, FaFile, BigWigFile, OrgDb, ChainFile, Inparanoi...
## # additional mcols(): taxonomyid, genome, description, tags,
## # sourceurl, sourcetype
## # retrieve records with, e.g., 'object[["AH2"]]'
##
## title
## AH2 | Ailuropoda_melanoleuca.ailMel1.69.dna.toplevel.fa
## AH3 | Ailuropoda_melanoleuca.ailMel1.69.dna_rm.toplevel.fa
## AH4 | Ailuropoda_melanoleuca.ailMel1.69.dna_sm.toplevel.fa
## AH5 | Ailuropoda_melanoleuca.ailMel1.69.ncrna.fa
## AH6 | Ailuropoda_melanoleuca.ailMel1.69.pep.all.fa
## ... ...
## AH49436 | Xiphophorus_maculatus.Xipmac4.4.2.dna_rm.toplevel.fa
## AH49437 | Xiphophorus_maculatus.Xipmac4.4.2.dna_sm.toplevel.fa
## AH49438 | Xiphophorus_maculatus.Xipmac4.4.2.dna.toplevel.fa
## AH49439 | Xiphophorus_maculatus.Xipmac4.4.2.ncrna.fa
## AH49440 | Xiphophorus_maculatus.Xipmac4.4.2.pep.all.fa
By default the AnnotationHub object is set to the latest
snapshotData and a snapshot version that matches the version of
Bioconductor that you are using. You can also learn about these data
with the appropriate methods.
snapshotDate(ah)
## [1] "2015-08-26"
If you are interested in using an older version of a snapshot, you can
list previous versions with the possibleDates() like this:
pd <- possibleDates(ah)
pd
## [1] "2013-03-19" "2013-03-21" "2013-03-26" "2013-04-04" "2013-04-29"
## [6] "2013-06-24" "2013-06-25" "2013-06-26" "2013-06-27" "2013-10-29"
## [11] "2013-11-20" "2013-12-19" "2014-02-12" "2014-02-13" "2014-03-31"
## [16] "2014-04-27" "2014-05-11" "2014-05-13" "2014-05-14" "2014-05-22"
## [21] "2014-07-02" "2014-07-09" "2014-12-15" "2014-12-24" "2015-01-08"
## [26] "2015-01-14" "2015-03-09" "2015-03-11" "2015-03-12" "2015-03-25"
## [31] "2015-03-26" "2015-05-06" "2015-05-07" "2015-05-08" "2015-05-11"
## [36] "2015-05-14" "2015-05-21" "2015-05-22" "2015-05-26" "2015-07-17"
## [41] "2015-07-27" "2015-07-31" "2015-08-10" "2015-08-13" "2015-08-14"
## [46] "2015-08-17" "2015-08-26"
Set the dates like this:
snapshotDate(ah) <- pd[1]
sessionInfo()
## R version 3.2.2 (2015-08-14)
## Platform: x86_64-apple-darwin13.4.0 (64-bit)
## Running under: OS X 10.9.5 (Mavericks)
##
## locale:
## [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] AnnotationHub_2.0.4 BiocStyle_1.6.0
##
## loaded via a namespace (and not attached):
## [1] Rcpp_0.12.0 futile.logger_1.4.1
## [3] BiocInstaller_1.18.4 formatR_1.2
## [5] GenomeInfoDb_1.4.2 XVector_0.8.0
## [7] futile.options_1.0.0 bitops_1.0-6
## [9] tools_3.2.2 zlibbioc_1.14.0
## [11] digest_0.6.8 evaluate_0.7.2
## [13] RSQLite_1.0.0 shiny_0.12.2
## [15] DBI_0.3.1 curl_0.9.3
## [17] parallel_3.2.2 rtracklayer_1.28.10
## [19] httr_1.0.0 stringr_1.0.0
## [21] knitr_1.11 Biostrings_2.36.4
## [23] S4Vectors_0.6.5 IRanges_2.2.7
## [25] stats4_3.2.2 Biobase_2.28.0
## [27] R6_2.1.1 AnnotationDbi_1.30.1
## [29] BiocParallel_1.2.21 XML_3.98-1.3
## [31] lambda.r_1.1.7 magrittr_1.5
## [33] Rsamtools_1.20.4 htmltools_0.2.6
## [35] BiocGenerics_0.14.0 GenomicRanges_1.20.6
## [37] GenomicAlignments_1.4.1 mime_0.4
## [39] interactiveDisplayBase_1.6.0 xtable_1.7-4
## [41] httpuv_1.3.3 stringi_0.5-5
## [43] RCurl_1.95-4.7