Computing similarity matrices
Similarity matrices can be computed with the function get_sim_grid. Full detail on using the function can be found in the help file: ?get_sim_grid, but briefly, the typical ingredients for computing a similarity matrix are:
- An
ontology_index object, e.g. hpo obtained with data(hpo)
- A
numeric vector of information contents named by term, e.g. see get_term_info_content in the ontologyIndex package. Defaults to descendants_IC(ontology).
- Some term sets - i.e. a collection of ontologically annotated objects, or from R’s point-of-view: a
list of character vectors of term IDs. Typically, term annotations are stored in delimited lists in tables, so strsplitting the column of a data.frame is often needed.
- A choice of term similarity - either
"lin" or "resnik".
library(ontologyIndex)
library(ontologySimilarity)
data(hpo)
set.seed(1)
information_content <- descendants_IC(hpo)
term_sets <- replicate(simplify=FALSE, n=7, expr=minimal_set(hpo, sample(hpo$id, size=8)))
sim_mat <- get_sim_grid(ontology=hpo, term_sets=term_sets)
sim_mat
## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 1.00000000 0.2446414 0.2077991 0.2002343 0.2654494 0.2614949
## [2,] 0.24464140 1.0000000 0.2079281 0.2604863 0.2292824 0.2146407
## [3,] 0.20779911 0.2079281 1.0000000 0.2385239 0.2144670 0.2044241
## [4,] 0.20023428 0.2604863 0.2385239 1.0000000 0.2450322 0.1496954
## [5,] 0.26544937 0.2292824 0.2144670 0.2450322 1.0000000 0.3306834
## [6,] 0.26149488 0.2146407 0.2044241 0.1496954 0.3306834 1.0000000
## [7,] 0.07281863 0.1442771 0.2353572 0.1871736 0.2133036 0.1669386
## [,7]
## [1,] 0.07281863
## [2,] 0.14427710
## [3,] 0.23535722
## [4,] 0.18717361
## [5,] 0.21330360
## [6,] 0.16693857
## [7,] 1.00000000
sim_mat can then of course be used by other packages’ clustering functions. Note that many of these expect a distance matrix, not a similarity matrix. Thus transformation may be required, e.g.
dist_mat <- max(sim_mat) - sim_mat
plot(hclust(as.dist(dist_mat)))
Assessing statistical significance of similarity
Given a collection of objects annotated with ontological terms, you can represent them as a list of character vectors of term IDs for compatibility with ontologySimilarity’s functions: a list of term sets.
Given a particular subgroup of interest - for example, in the context of the collection of all genes’ GO annotations, this could be a group of genes identified through an experiment - a natural question might be, ‘are these objects as a group particularly similar to one another?’.
We can attempt to answer that question by calculating a ‘group similarity’: a measure of similarity for a group of ontologically annotated objects, or ‘list of term sets’. ontologySimilarity supports two different methods for calculating the similarity of a group.
"average" - taking the average similarity over all pairs of members."min" - taking the minimum similarity over all pairs of members.
The function get_sim is used to calculate these similarities. The argument group_sim can be either "average" or "min" accordingly.
The value of the similarity itself doesn’t tell you how similar the group members are relative to other subsets. So to evaluate the signficance of the group similarity of subset S within population of term sets L, we compute a p-value by permutation test: that is, we compute the proportion of the ‘null’ distribution of similarities of all subsets of the same size as S that are greater than the similarity of S.
This is done by passing the indices of S within L to the function get_sim_p. Instead of computing the p-value exactly by comparing to the similarity of all choose(length(L), length(S)) permutations, the function get_sim_p samples a specified maximum number of random subsets.
For efficiency in the situation where only statistically significant results are of interest, get_sim_p allows the user to configure a threshold triggering the sampling to stop early, should the strength of evidence against it being significant become strong enough. Precisely, this is done using the arguments:
min_its - the minimum number of samples to draw from the null distribution.max_its - the maximum number of samples.signif - p-value considered ‘significant’.log_dismiss - the threshold log probability of significance, below which sampling stops. Probability of significance is computed using the binomial approximation to the Normal. It is up to the user to decide whether this approximation is appropriate given the other parameters.
The function get_sim_p requires an argument called pop_sim which stores information on the similarities of the entire collection of objects. Depending on the type of parameter you pass, it is done in different ways which suit different scenarios.
Using a similarity matrix
One way is to pass a matrix having the similarity between all term set pairs. The null distribution is then estimated by measuring the similarity of random subsets of the matrix. This is the fastest method if such a matrix is available as it doesn’t require any between-object similarities to be calculated. However computing such a matrix may be infeasible if N is large. Note that this method can be used for any matrix of similarities, and no references to ontologies are required.
Using a sim_index object
One way is to pass an argument of class sim_index as created with the function create_sim_index. To create the index you need to pass the list of term sets and an ontology argument. Note that if you do not wish to store the sim_index object you can call the function get_sim_p_from_ontology. See ?create_sim_index and ?get_sim_p_from_ontology for more details. The null distribution of group similarities is then estimated by calculating group similarities of random subsets of L. This method is useful when you don’t want to compute and store a similarity matrix - for example if N is large.
See ?get_sim and ?get_sim_p for further details.
Example
In this example we’ll generate a collection 100 random sets of HPO terms and map them to minimal sets. We’ll then assess whether the subgroup comprising the first 10 sets is significantly similar - first by using the ontology_index object and information content directly, then using a similarity matrix.
collection <- replicate(simplify=FALSE, n=100, expr=minimal_set(hpo, sample(hpo$id, size=8)))
#lets measure the group similarity of objects 1-10
group <- 1:10
get_sim_p_from_ontology(
ontology=hpo,
information_content=information_content,
term_sets=collection,
group=group)
## [1] 0.9240759
sim_mat <- get_sim_grid(ontology=hpo, term_sets=collection)
#p-value by "matrix" method
get_sim_p(
sim_mat,
group=group)
## [1] 0.9110889
Against a vector
Another scenario might be: you’ve measured the similarity between a collecton of ontologically annotated objects and a foreign object, and have obtained a vector of similarities, and now, you’re interested in whether some subgroup of the collection are particularly similar on average. In this case you can pass a numeric vector to get_sim_p.
In this example, we’ll generate some hypothetical genes with GO annotation [i.e. randomly sample 8 GO terms per gene and map to minimal sets]. Then - supposing we are interested in genes which have functions bearing on the ‘golgi apparatus’ - use get_profile_sims to calculate the similarity of the genes to an ‘ontological profile’ containing golgi related terms. Finally - supposing for example genes 1-3 came up in the results of an experiment - we’ll calculate a similarity p-value of genes 1-3 to the ‘golgi GO profile’ using get_sim_p, passing it the pre-computed profile similarities.
data(go)
genes <- replicate(simplify=FALSE, n=100, expr=minimal_set(go, sample(go$id, size=8)))
names(genes) <- paste("gene", 1:length(genes))
genes[1:3]
## $`gene 1`
## [1] "GO:0042808" "GO:0090448" "GO:0001983" "GO:0044179" "GO:0010026"
## [6] "GO:0042171" "GO:0000903" "GO:0019502"
##
## $`gene 2`
## [1] "GO:1904778" "GO:0070210" "GO:1990253" "GO:0090577" "GO:0006777"
## [6] "GO:0071179" "GO:0035934" "GO:0072664"
##
## $`gene 3`
## [1] "GO:0046421" "GO:0009237" "GO:1903351" "GO:0060836" "GO:0050202"
## [6] "GO:0002530" "GO:0042703" "GO:1903865"
go_profile <- as.character(go$id[grep(x=go$name, pattern="golgi apparatus", ignore.case=TRUE)])
go$name[go_profile]
## GO:0005794
## "Golgi apparatus"
## GO:0034067
## "protein localization to Golgi apparatus"
## GO:0044177
## "host cell Golgi apparatus"
## GO:0044431
## "Golgi apparatus part"
## GO:0045053
## "protein retention in Golgi apparatus"
## GO:0048280
## "vesicle fusion with Golgi apparatus"
## GO:1904381
## "Golgi apparatus mannose trimming"
profile_sims <- get_profile_sims(ontology=go, term_sets=genes, profile=go_profile)
profile_sims
## gene 1 gene 2 gene 3 gene 4 gene 5 gene 6
## 0.16006231 0.36330686 0.15036853 0.22652958 0.18794782 0.17695368
## gene 7 gene 8 gene 9 gene 10 gene 11 gene 12
## 0.19529763 0.22087894 0.21197303 0.17336796 0.20836591 0.21939688
## gene 13 gene 14 gene 15 gene 16 gene 17 gene 18
## 0.13014403 0.15867005 0.17499251 0.11778417 0.20916234 0.24588939
## gene 19 gene 20 gene 21 gene 22 gene 23 gene 24
## 0.19569134 0.17265063 0.13880939 0.20579631 0.21861271 0.21572184
## gene 25 gene 26 gene 27 gene 28 gene 29 gene 30
## 0.14901410 0.21713831 0.15142360 0.24041101 0.26822096 0.19571609
## gene 31 gene 32 gene 33 gene 34 gene 35 gene 36
## 0.21803567 0.15733712 0.33844835 0.16718688 0.17042658 0.19293417
## gene 37 gene 38 gene 39 gene 40 gene 41 gene 42
## 0.20607445 0.14545869 0.20865281 0.18442403 0.11716919 0.10020522
## gene 43 gene 44 gene 45 gene 46 gene 47 gene 48
## 0.19929645 0.17007827 0.21805553 0.09413583 0.21786613 0.14317189
## gene 49 gene 50 gene 51 gene 52 gene 53 gene 54
## 0.13590454 0.23048749 0.05805750 0.24503326 0.25506341 0.19694521
## gene 55 gene 56 gene 57 gene 58 gene 59 gene 60
## 0.13164966 0.13642425 0.11844335 0.15015852 0.26751020 0.21750894
## gene 61 gene 62 gene 63 gene 64 gene 65 gene 66
## 0.22589648 0.10518084 0.17486739 0.20168300 0.18741066 0.16159574
## gene 67 gene 68 gene 69 gene 70 gene 71 gene 72
## 0.17234251 0.16520989 0.13031360 0.09782839 0.20161116 0.15014327
## gene 73 gene 74 gene 75 gene 76 gene 77 gene 78
## 0.10236836 0.25371221 0.18303291 0.11624643 0.14907601 0.23161513
## gene 79 gene 80 gene 81 gene 82 gene 83 gene 84
## 0.22735275 0.08978890 0.17229930 0.15556540 0.24565904 0.21398295
## gene 85 gene 86 gene 87 gene 88 gene 89 gene 90
## 0.18211758 0.17422597 0.12916763 0.34176380 0.16773958 0.11742309
## gene 91 gene 92 gene 93 gene 94 gene 95 gene 96
## 0.20856422 0.22196440 0.17466233 0.28268721 0.18958718 0.24425677
## gene 97 gene 98 gene 99 gene 100
## 0.08392443 0.14486453 0.17380324 0.18886146
#Note that you can pass character vectors to get_sim_p
get_sim_p(profile_sims, c("gene 1", "gene 2", "gene 3"))
## [1] 0.1098901
Sampling the null distribution
Because of the ‘early stopping’, get_sim_p should be sufficiently performant to loop or lapply over a long list of subgroups. However, p-values for groups of the same size are technically being computed by comparison with the same null distribution. Thus, if the number of applications required is very long, the user could consider storing the null distributions for particular group sizes, and computing p-values later by comparing with the actual group similarities (calculated with get_sim).
The function sample_group_sim can be used to sample from the null distribution of similarities.
group_sim <- get_sim(sim_mat, group=group)
samples <- sample_group_sim(sim_mat, group_size=length(group))
hist(samples)
abline(v=group_sim, col="red")
