---
title: "About BioCor"
abstract: >
  Describes the background of the package, important functions defined in the
  package and some of the applications and usages, including the integration with
  other packages and analysis, and comparisons with related packages. Some
  frequent or important questions about the package are answered at the end of
  the document. For more advanced usage you can look at the other vignette.
date: "`r BiocStyle::doc_date()`"
package: "`r BiocStyle::pkg_ver('BioCor')`"
output:
  BiocStyle::html_document:
    fig_caption: true
    code_folding: show
    self_contained: yes
    toc_float:
      
      collapsed: true
      toc_depth: 3
author:
- name: Lluís Revilla
  affiliation: 
    - August Pi i Sunyer Biomedical Research Institute (IDIBAPS); Liver Unit, Hospital Clinic
  email: lrevilla@clinic.cat
- name: Juan José Lozano
  affiliation:
    - Centro de Investigación Biomédica en Red de Enfermedades Hepaticas y Digestivas (CIBEREHD); Barcelona, Spain
- name: Pau Sancho-Bru
  affiliation:
   - August Pi i Sunyer Biomedical Research Institute (IDIBAPS); Liver Unit, Hospital Clinic
opengraph:
  image:
    src: man/figures/logo.png
    alt: "BioCor logo"
  twitter:
    creator: "@Lluis_Revilla"
    card: summary
vignette: >
  %\VignetteIndexEntry{About BioCor}
  %\VignetteEncoding{UTF-8}  
  %\VignetteEngine{knitr::rmarkdown}
editor_options: 
  chunk_output_type: console
---

```{r knitsetup, message=FALSE, warning=FALSE, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, warning = TRUE, crop = FALSE)
library("BiocStyle")
```


# Introduction

Methods to find similarities have been developed for several purposes, being Jaccard and Dice similarities the most known. In bioinformatics much of the research on the topic is centered around [Gene Ontologies](https://www.geneontology.org/) because they provide controlled vocabularies, as part of their mission:

> The mission of the GO Consortium is to develop an up-to-date, comprehensive, computational model of biological systems, from the molecular level to larger pathways, cellular and organism-level systems.

However, there is another resource of similarities between genes: metabolic pathways. Metabolic pathways describe the relationship between genes, proteins, lipids and other elements of the cells. A pathway describes, to some extent, the function in which it is involved in the cell. There exists several databases about which gene belong to which pathway. Together with pathways, gene sets related to a function or to a phenotype are a source of information of the genes function. With this package we provide the methods to calculate functional similarities based on this information.

Here we provides functions to calculate *functional similarities distances* for pathways, gene sets, genes and clusters of genes. The name BioCor stands from biological correlation, shortened to BioCor, because as said we look if some genes are in the same pathways or gene sets as other genes. 

BioCor is different from `r BiocStyle::Biocpkg("GeneOverlap")` because here we use the Dice index instead of the Jaccard index (although we provide a function to change from one to the other, see [this section](#D2J))and that package only allows to compare pathways but not genes or groups of genes. But `r BiocStyle::Biocpkg("GeneOverlap")` provides some functionalities to plot the similarity scores and provides the associated p-value to the comparison of pathways.

The development of this package aimed initially to improve clustering of genes by functionality in weighted gene co-expression networks using `r BiocStyle::CRANpkg("WGCNA")`. The package has some functions to combine similarities in order to integrate with `WGCNA`. For other uses you can check the `r BiocStyle::Biocpkg("BioCor", "BioCor_2_advanced.html", label = "advanced vignette.")`.

# Citation

You can cite the package as:

```{r eval = FALSE}
citation("BioCor")
```


# Installation

The BioCor package is available at [Bioconductor](https://bioconductor.org) and can be downloaded and installed via BiocManager:

```{r install, eval=FALSE}
install.packages("BiocManager")
BiocManager::install("BioCor")
```

You can install the latest version of `r BiocStyle::Githubpkg("llrs/BioCor")` from Github with:

```{r github, eval = FALSE}
library("devtools")
install_github("llrs/BioCor")
```

# Using BioCor

```{r preload, include=FALSE}
library("BioCor")
library("org.Hs.eg.db")
library("reactome.db")
```

## Preparation

We can load the package and prepare the data for which we want to calculate the similarities:

```{r load}
library("BioCor")
## Load libraries with the data of the pathways
library("org.Hs.eg.db")
library("reactome.db")
genesKegg <- as.list(org.Hs.egPATH)
genesReact <- as.list(reactomeEXTID2PATHID)
# Remove genes and pathways which are not from human pathways 
genesReact <- lapply(genesReact, function(x){
    unique(grep("R-HSA-", x, value = TRUE))
    })
genesReact <- genesReact[lengths(genesReact) >= 1] 
```

To avoid having biased data it is important to have all the data about the pathways and genes associated to all pathways for the organism under study. Here we assume that we are interested in human pathways. We use this two databases KEGG and Reactome as they are easy to obtain the data. However KEGG database is no longer free for large retrievals therefore it is not longer updated in the Bioconductor annotation packages.

However, one can use any list where the names of the list are the genes and the elements of the list the pathways or groups where the gene belong. One could also read from a GMT file or use GeneSetCollections in addition or instead of those associations from a pathway database and convert it to list using:

```{r GSEABase, eval = FALSE}
library("GSEABase")
paths2Genes <- geneIds(getGmt("/path/to/file.symbol.gmt",
                 geneIdType=SymbolIdentifier()))

genes <- unlist(paths2Genes, use.names = FALSE)
pathways <- rep(names(paths2Genes), lengths(paths2Genes))
genes2paths <- split(pathways, genes) # List of genes and the gene sets
```

With `genes2paths` we have the information ready to use.

## Pathway similarities

We can compute similarities (Dice similarity, see [question 1](#FAQ1) of FAQ) between two pathways or between several pathways and combine them, or not:

```{r pathSim}
(paths <- sample(unique(unlist(genesReact)), 2))
pathSim(paths[1], paths[2], genesReact)

(pathways <- sample(unique(unlist(genesReact)), 10))
mpathSim(pathways, genesReact)
```

When the method to combine the similarities is set to `NULL` `mpathSim()` returns a matrix of pathway similarities, otherwise it combines the values. In the next section we can see the methods to combine pathway similarities.

### Combining values {#combining}

To combine values we provide a function with several methods:

```{r combineScores}
sim <- mpathSim(pathways, genesReact)
methodsCombineScores <- c("avg", "max", "rcmax", "rcmax.avg", "BMA",
                          "reciprocal")
sapply(methodsCombineScores, BioCor::combineScores, scores = sim)
```

We can also specify the method to combine the similarities in `mpathSim()`, `geneSim()`, `mgeneSim()`, `clusterSim()`, `mclusterSim()`, `clusterGeneSim()` and `mclusterGeneSim()`, argument method. By default the method is set to "max" to combine pathways (except in mpathSim where the default is to show all the pathway similarities) and "BMA" to combine similarities of genes or for cluster analysis. This function is adapted from `r Biocpkg("GOSemSim")` package.

The function `combineScoresPar()` allows to use a parallel background (using `r Biocpkg("BiocParallel")`) to combine the scores. It is recommended to use a parallel background if you calculate more than 300 gene similarities. It also have an argument in case you want to calculate the similarity scores of several sets.

## Gene similarities {#geneSim}

To compare the function of two genes there is the `geneSim` function and `mgeneSim` function for several comparisons. In this example we compare the genes BRCA1 and BRCA2 and NAT2, which are the genes 672, 675 and 10 respectively in ENTREZID:

```{r geneSim}
geneSim("672", "675", genesKegg)
geneSim("672", "675", genesReact)

mgeneSim(c("BRCA1" = "672", "BRCA2" = "675", "NAT2" = "10"), genesKegg)
mgeneSim(c("BRCA1" = "672", "BRCA2" = "675", "NAT2" = "10"), genesReact)
```

Note that for the same genes each database or list provided has different annotations, which result on different similarity scores. In this example BRCA1 has `r ncol(geneSim("672", "675", genesKegg, NULL))` and `r ncol(geneSim("672", "675", genesReact, NULL))` pathways in KEGG and Reactome respectively and BRCA2 has `r nrow(geneSim("672", "675", genesKegg, NULL))` and `r nrow(geneSim("672", "675", genesReact, NULL))` pathways in KEGG and Reactome respectively which results on different scores.

## Gene cluster similarities

There are two methods:

* Combining all the pathways for each cluster and compare between them.
* Calculate the similarity between genes of a cluster and the other cluster.

### By pathways {#clusterSim}

As explained, in this method all the pathways of a cluster are compared with all the pathways of the other cluster. If a method to combine pathways similarities is not provided, all pathway similarities are returned:

```{r clusterSim}
clusterSim(c("672", "675"), c("100", "10", "1"), genesKegg)
clusterSim(c("672", "675"), c("100", "10", "1"), genesKegg, NULL)

clusters <- list(cluster1 = c("672", "675"),
                 cluster2 = c("100", "10", "1"),
                 cluster3 = c("18", "10", "83"))
mclusterSim(clusters, genesKegg, "rcmax.avg")
mclusterSim(clusters, genesKegg, "max")
```

### By genes {#clusterGeneSim}

In this method first the similarities between each gene is calculated, then the similarity between each group of genes is calculated. Requiring two methods to combine values, the first one to combine pathways similarities and the second one to combine genes similarities. If only one is provided it returns the matrix of similarities of the genes of each cluster:

```{r clusterGeneSim}
clusterGeneSim(c("672", "675"), c("100", "10", "1"), genesKegg)
clusterGeneSim(c("672", "675"), c("100", "10", "1"), genesKegg, "max")

mclusterGeneSim(clusters, genesKegg, c("max", "rcmax.avg"))
mclusterGeneSim(clusters, genesKegg, c("max", "max"))
```

Note the differences between `mclusterGeneSim()` and `mclusterSim()` in the similarity values of the clusters. If we set `method = c("max", "max")` in `mclusterGeneSim()` then the similarity between the clusters is the same as `clusterSim()`.


## Converting similarities {#D2J}

If needed, Jaccard similarity can be calculated from Dice similarity using `D2J()`:

```{r sims}
D2J(sim)
```

Also if one has a Jaccard similarity and wants a Dice similarity, can use the `J2D()` function.

# High volumes of gene similarities

We can compute the whole similarity of genes in KEGG or Reactome by using :

```{r whole_db, eval=FALSE}
## Omit those genes without a pathway
nas <- sapply(genesKegg, function(y){all(is.na(y)) | is.null(y)})
genesKegg2 <- genesKegg[!nas]
m <- mgeneSim(names(genesKegg2), genesKegg2, method  = "max")
```

It takes around 5 hours in one core but it requires high memory available.

If one doesn't have such a memory available can compute the similarities by pieces, and then fit it in another matrix with:

```{r whole_db2, eval=FALSE}
sim <- AintoB(m, B)
```

Usually B is a matrix of size `length(genes)`, see `?AintoB()`.

# An example of usage

In this example I show how to use BioCor to analyse a list of genes by functionality. 
With a list of genes we are going to see how similar are those genes:

```{r hclust1, fig.cap="Gene clustering by similarities", fig.wide = TRUE}
#| fig.alt = "Dendrogram of the similarities of genes according to Reactome."
genes.id <- c("10", "15", "16", "18", "2", "9", "52", "3855", "3880", "644", 
              "81327", "9128", "2073", "2893", "5142", "60", "210", "81", 
              "1352", "88", "672", "675")
genes.id <- mapIds(org.Hs.eg.db, keys = genes.id, keytype = "ENTREZID", 
                   column = "SYMBOL")
genes <- names(genes.id)
names(genes) <- genes.id
react <- mgeneSim(genes, genesReact)
## We remove genes which are not in list (hence the warning):
nan <- genes %in% names(genesReact)
react <- react[nan, nan]
hc <- hclust(as.dist(1 - react))
plot(hc, main = "Similarities between genes")
```

Now we can see the relationship between the genes. We can group them for a cluster analysis to visualize the relationship between the clusters:

```{r hclust3, fig.cap="Clustering using clusterSim", fig.wide = TRUE}
#| fig.alt = "Dendrogram of clusters of genes according to Reactome."
mycl <- cutree(hc, h = 0.2)
clusters <- split(genes[nan], as.factor(mycl))
# Removing clusters of just one gene
(clusters <- clusters[lengths(clusters) >= 2])
names(clusters) <- paste0("cluster", names(clusters))
## Remember we can use two methods to compare clusters
sim_clus1 <- mclusterSim(clusters, genesReact)
plot(hclust(as.dist(1 - sim_clus1)), 
     main = "Similarities between clusters by pathways")
```

```{r hclust3b, fig.cap="Clustering using clusterGeneSim", fig.wide=TRUE}
#| fig.alt = "Dendrogram of clusters according to similarities between genes from Reactome pathways."
sim_clus2 <- mclusterGeneSim(clusters, genesReact)
plot(hclust(as.dist(1 - sim_clus2)), 
     main ="Similarities between clusters by genes")
```

Each method results in a different dendrogram as we can see  on Figure \@ref(fig:hclust3) compared to Figure \@ref(fig:hclust3b).

# Comparing with GOSemSim

In this section I will compare the functional similarity of BioCor with the closely related package `r Biocpkg("GOSemSim")`. The genes and gene clusters used were extracted from GOSemSim's vignette, we only change the ontology, instead of the molecular function, the biological process will be used:

```{r GOSemSim}
hsGO <- GOSemSim::godata('org.Hs.eg.db', ont = "BP", computeIC = FALSE)
```

I will compare the functions geneSim from section [geneSim and mgeneSim from GOSemSim](https://bioconductor.org/packages/release/bioc/vignettes/GOSemSim/inst/doc/GOSemSim.html#genesim-and-mgenesim) with both data sets from KEGG and Reactome:

```{r geneSimGOSemSim}
goSemSim <- GOSemSim::geneSim("241", "251", semData = hsGO, 
                              measure = "Wang", combine="BMA")
# In case it is null
sim <- ifelse(is.na(goSemSim), 0, getElement(goSemSim, "geneSim"))
BioCor::geneSim("241", "251", genesReact, "BMA") - sim

genes <- c("835", "5261","241", "994")
goSemSim <- GOSemSim::mgeneSim(genes, semData = hsGO, 
                   measure = "Wang", combine = "BMA",
                   verbose = FALSE, drop = NULL)
BioCor::mgeneSim(genes, genesReact, "BMA", round = TRUE) - goSemSim
```

We can observe there is more similarity according to the gene ontology than according to the pathways. 
See FAQ [question 8](#conflict) about the use of `BioCor::` and `GOSemSim::`.

If named characters are passed they are used to name the resulting matrix:

```{r named}
genes <- c("CDC45", "MCM10", "CDC20", "NMU", "MMP1")
genese <- mapIds(org.Hs.eg.db, keys = genes, column = "ENTREZID", 
                 keytype = "SYMBOL")
BioCor::mgeneSim(genese, genesReact, "BMA")
```

We can further compare the cluster similarities from the [next section of the vignette](https://bioconductor.org/packages/release/bioc/vignettes/GOSemSim/inst/doc/GOSemSim.html#clustersim-and-mclustersim):

```{r clusterSimGOSemSim}
gs1 <- c("835", "5261","241", "994", "514", "533")
gs2 <- c("578","582", "400", "409", "411")
BioCor::clusterSim(gs1, gs2, genesReact, "BMA") - 
    GOSemSim::clusterSim(gs1, gs2, hsGO, measure = "Wang", combine = "BMA")

x <- org.Hs.egGO
hsEG <- mappedkeys(x)
set.seed(123)
(clusters <- list(a=sample(hsEG, 20), b=sample(hsEG, 20), c=sample(hsEG, 20)))
BioCor::mclusterSim(clusters, genesReact, "BMA") - 
    GOSemSim::mclusterSim(clusters, hsGO, measure = "Wang", combine = "BMA")
```

# WGCNA and BioCor 

`r BiocStyle::CRANpkg("WGCNA")` uses the correlation of the expression data of several samples to cluster genes. Sometimes, from a biological point of view the interpretation of the resulting modules is difficult, even more when some groups of genes end up not having an enrichment in previously described functions. 
BioCor was originally thought to be used to overcome this problem: to help clustering genes, not only by correlation but also by functionality. 

In order to have groups functionally related, functional similarities can enhance the clustering of genes when combined with experimental correlations. The resulting groups will reflect, not only the correlation of the expression provided, but also the functionality known of those genes.

We propose the following steps:

1. Calculate the similarities for the expression data 
2. Calculate the similarities of the genes in the expression 
3. Combine the similarities
4. Calculate the adjacency 
5. Identify modules with hierarchical clustering

Here we provide an example on how to use BioCor with WGCNA:

`sim` is a list where each element is a matrix of similarities between genes 
Our normalized expression is in the `expr` variable, a matrix where the samples are in the rows and genes in the columns.

```{r wgcna, eval=FALSE}
expr.sim <- WGCNA::cor(expr) # or bicor

## Combine the similarities
similarity <- similarities(c(list(exp = expr.sim), sim), mean, na.rm = TRUE)

## Choose the softThreshold
pSFT <- pickSoftThreshold.fromSimilarity(similarity)

## Or any other function we want
adjacency <- adjacency.fromSimilarity(similarity, power = pSFT$powerEstimate)

## Once we have the similarities we can calculate the TOM with TOM
TOM <- TOMsimilarity(adjacency) ## Requires adjacencies despite its name 
dissTOM <- 1 - TOM
geneTree <- hclust(as.dist(dissTOM), method = "average")
## We can use a clustering tool to group the genes
dynamicMods <- cutreeHybrid(dendro = geneTree, distM = dissTOM,
                            deepSplit = 2, pamRespectsDendro = FALSE,
                            minClusterSize = 30)
moduleColors <- labels2colors(dynamicMods$labels)
```

Once the modules are identified using the functional similarities of this package and the gene correlations, one can continue with the workflow of WGCNA.

An important aspect in this process is deciding how to combine the similarities and the expression data: 
 - If the functional similarities play a huge role, we will end up having only those genes closely related to the same functions. 
 - If the functional similarities play a low role, it will be similarly to only use WGCNA, and the genes won't be functionally related.

For these reasons it is better to use weights between `0.5` and `1` for expression if you use `weighted.sum` or similar functions.

There are several things to take into account when choosing a way to combine:
 - The size of the gray or 0 modules (those who don't show a specific pattern)
 - The number and size of the modules created.
 - The way the similarities are combined

Violin plots may help to view the differences in size and distribution of the modules across different methods of combining the similarities.

# FAQ

## How is defined the pathway similarity? {#FAQ1}

BioCor uses the [Sørensen–Dice index](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient): 
The dice similarity is the double of the genes shared by the pathways divided by the number of genes in each pathway.

We can calculate the similarity between two pathways ($x$, $w$) with:

$$Dice(x, w) = \frac{2 |x \cap w|}{|x| + |w|}$$

This is implemented in the `diceSim` function, which results is similar to Jaccard index:

$$Jaccard(x, w) = \frac{|x \cap w|}{|x \cup w|}$$

Both Jaccard index and dice index are between 0 and 1 ($[0, 1]$). To calculate the Jaccard index from the `diceSim` use the `D2J` function.

## Why does BioCor use the dice coefficient and not the Jaccard ?

We consider Dice coefficient better than Jaccard because it has higher values for the same comparisons, which reflects that including a gene in a pathway is not easily done.

## How does BioCor combine similarities between several pathways of two genes?

Although the recommend method is the "max" method, (set as default), there are implemented other methods in `combineScores` of the `r BiocStyle::Biocpkg("GOSemSim")` package which I borrowed[^1]. 

[^1]: See the [Combining values section](#combining) and the help page of `combineScores`.

## Why do you recommend using the max method to combine similarities scores for pathways?

The purpose of combining the scores is usually to find the relationships between genes through their pathways. The higher the similarity is between two pathway of two genes, the higher functionality do the genes share, even if those genes have other non-related functions. 

## How to detect which functional relationship is more important between two genes?

If two genes are involved in the same pathways usually they have (to some extent, maybe indirect) interactions. To detect which relationship is more important between two genes one could measure other similarities scores and check the stoichiometry of the pathways and measure the expression changes and correlation between them or use dynamic simulations of the pathways.

## How to detect with which genes is my gene of interest related?

You can measure the [gene similarity](#geneSim) between those genes and also measure the expression correlation of your gene of interest with other genes. 

## Why isn't available a method for calculating GO similarities?

This is covered by the `r BiocStyle::Biocpkg("GOSemSim")` package, you can use it to produce a similarity matrix (i.e. use `mgeneSim`). You can parallelize it with `foreach` package or `BiocParallel` if your list of genes is big. 

## I get an error! How do I solve this? {#conflict}

If the error is like this:

```{r, eval = FALSE}
Error in FUN(X[[i]], ...) : 
  trying to get slot "geneAnno" from an object of a basic class ("list") with no slots
```

And you have loaded the `GOSemSim` library, R is calling the GOSemSim function of the same name. Use `BioCor::` to call the function from `BioCor` (f.ex: `BioCor::geneSim`)

If the error is not previously described in the [support forum](https:support.bioconductor.org), post a question there. 

My apologies if you found a bug or an inconsistency between what `BioCor` should do and what it actually does. Once you checked that it is a bug, please let me know at the `r Githubpkg("llrs/BioCor/issues", "issues")` page of Github.

# Session Info {.unnumbered}

```{r session, code_folding = FALSE}
sessionInfo()
```
