AnalysisReport-1_HeuristicMethod.Rmd

---
title: "Selection of L-shaped genes using a heuristic algorithm"
author: "Berta Miró, Francesc Carmona,  Diego Arango and Alex Sánchez Pla"
institute: "Departament of Genetics Microbiology and Statistics, Universitat de Barcelona<br> CIBBIM-Nanomedicine. Biomedical Research in Digestive Tumors, (VHIR), Barcelona<br> Statistic and Bioinformatics Unit. Vall d'Hebron Research Institute.  (VHIR). Barcelona"
date: "`r Sys.Date()`"

output:
   html_document:
     toc: true
     toc_depth: 3
     number_sections: true
     code_folding: show
     theme: cosmo
bibliography: mybibfile.bib
link-citations: yes

# link-citations: yes
# theme args should be one of: "default", "cerulean", "journal", "flatly", "darkly", "readable", "spacelab", "united", "cosmo", "lumen", "paper", "sandstone", "simplex", "yeti"
# highlight arg should be one of: "default", "tango", "pygments", "kate", "monochrome", "espresso", "zenburn", "haddock", "breezedark", "textmate"
# Information on themes and highlights: https://bookdown.org/yihui/rmarkdown/html-document.html#appearance_and_style
---

```{r setup, include=FALSE}
options(htmltools.dir.version = FALSE)
```

```{r include=FALSE}
require(knitr)
opts_chunk$set(
concordance=FALSE, echo=TRUE, cache=FALSE, warning=FALSE, error=FALSE, message=FALSE)
```

# Introduction

This document contains examples / applications on how to select L-shaped genes from two data matrices, an expression and a methylation matrix, matched by rows (genes) and columns(samples), using the algorithms implemented by the authors in the `Lheuristic` package and described in section 3, *Scoring Scatterplots*.

The idea is to "keep-it-simple" so that it can serve as the basis for other applications such as a program that optimizes the parameters or for a graphical user interface based in Shiny.

Essentially what a user may want to do uysing this package is:

-   Data input

    -   Select (Load) expression values from a csv file

    -   Select (Load) methylation values from a csv file

    -   Data preprocessing may need to be done automatically so that its results can be used for filtering.

    -   Set parameter values

        -   For data use (all genes or only those that pass certain filters ($\mbox{corr}< 0$))
        -   For L-shape selection method: Here one may want to select the method and its parameters (each method has a different set of parameters)

-   Data processing (For each method and set of parameters ...)

    -   Run the computation and "mark" L-shape genes
    -   Draw the scatterplots of either all genes or only selected genes

-   Data output (For each method and set of parameters ...)

    -   Save (Download) the resulting gene list(s)
    -   Save (Download) the scatterplots

```{r directoris, echo=FALSE}
options(digits=4)
workingDir <- getwd()
dadesDir <-  file.path(workingDir, "dades")
resultsDir <- file.path(workingDir, "results")
```

```{r loadpackages}
if (!(require(Lheuristic))) devtools::install_github("aspresearch/Lheuristic")
if (!(require(VennDiagram))) install.packages("VennDiagram")
if (!(require(printr))) install.packages("printr")

```

# The data for the analysis

## Real datasets

There are several datasets available for analysis obtained from distinct sources:

1.  DA1. Expression microarrays (*DAMicroarrays.csv*) and methylation array (*DAMetilacion.csv*) on 25 cell llines
2.  DA2. Expression RNAseq (*DARNAseq.csv*) and methylation array (*DAMetilacion.csv*) on 25 cell llines
3.  GEO1. Expression microarrays (Illumina beadchips, *geoMicroarrays.csv*) and methylation array (Illumina 25kMethArray, *geoMetilacion.csv*) on 25 CRC samples. The data have been collected from the GEO database records: GSE25062 for the methylation data and GSE25070 for the expression data. The dataset has been created by taking the expression values available for 26 CRC tumors and matching with their corresponding methylation values.
4.  *TCGA* dataset has been obtained from The Cancer Genome Atlas Database (TCGA) (Colon Adenocarcinoma (*COAD*) Nature 2012 dataset) and downloaded through the cBioportal website.

A summary of each datasets follows below:

```{r readDataDA}
load(file="dades/DataMatrices-DA.Rda")
cat("DA Microarray data : ", dim(DAExprData), "\n")
cat("DA Methylation data: ", dim(DAMetilData), "\n")
```

```{r readGEOdata}
load(file="dades/DataMatrices-GEO.Rda")
cat("GEO Microarray data : ", dim(geoExprData), "\n")
cat("GEO Methylation data: ", dim(geoMetilData), "\n")
```

```{r readTCGAdata}
load(file="dades/DataMatrices-TCGA.Rda")
cat("TCGA Microarray data : ", dim(TCGAExprData), "\n")
cat("TCGA Methylation data: ", dim(TCGAMetilData), "\n")
```

```{r inCommon}
inCommon<- length(intersect(rownames(DAExprData), rownames(geoExprData)))
inCommon2 <- length(intersect(rownames(DAExprData), rownames(TCGAExprData)))
inCommon3 <- length(intersect(rownames(geoExprData), rownames(TCGAExprData)))
```

There are \Sexpr{inCommon} genes in common between the DAX dataset and the GEO datasets.
There are \Sexpr{inCommon2} common genes between the DAX dataset and the TCGA dataset and \Sexpr{inCommon3} common genes between the GEO dataset and the TCGA dataset.

This can be visualized using a Venn diagram

```{r vennCommonGenesin3lists}
library(VennDiagram)
myVenn4<- venn.diagram(x=list(DA=rownames(DAExprData),
                              GEO=rownames(geoExprData),
                              TCGA=rownames(TCGAExprData)),
                              filename=NULL, lty = "blank",
                              fill=c("pink1", "skyblue", "mediumorchid"),
                       main="Genes in common between the three datasets")
grid.newpage()
grid.draw(myVenn4)
```

The data for these analyses must have a common structure: **Each pair of matrices (Expression-Methylation) must have the same rome and column names**, that is both datasets must contain information for the same genes and same samples at their corresponding positions.

This can be checked using a simple function such as `checkPairing` available in the package.

```{r checkPairing}
library(Lheuristic)
(try(if(!checkPairing(DAExprData, DAMetilData)) stop("Row names and/or column names do not match")))
(try(if(!checkPairing(geoExprData, geoMetilData)) stop("Row names and/or column names do not match")))
```

# Expression-Methylation scatterplots

When one is studying the relation between methylation and expression for a bunch of genes it may be convenient to (be able) to plot the scatterplots depicting the relation between these variables.
Function *plotGenesMat* allows to draw such plots.

Some examples of using this function with the first four genes of the example datasets are shown in figures \@ref(fig:plot4Genes1), \@ref(fig:plot4Genes2) and \@ref(fig:plot4Genes3) respectively.

```{r plot4Genes1, echo=FALSE, message=FALSE, results='hide', fig.cap='Scatter plot of first four genes in DA dataset (microarrays)"<a name="plot4Genes1'}
#selectedGenes <- c("A1BG","A2ML1",  "A4GALT", "AAAS" )
selectedGenes <- c("ALDH1A2", "ALDH1A3", "APCDD1", "ARHGDIB")
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=DAMetilData[selectedGenes, ],
              expres=DAExprData[selectedGenes,], x1=1/3, x2=2/3,
              percY1=1/3, percY2=2/3,
              fileName=NULL, plotGrid = TRUE)
par(opt)
```

```{r plot4Genes2, echo=FALSE, message=FALSE, results='hide', fig.cap='Scatter plot of first four genes in GEO dataset (microarrays)'}
#selectedGenes <- c("A1BG","A2ML1",  "A4GALT", "AAAS" )
selectedGenes <- c("ALDH1A2", "ALDH1A3", "APCDD1", "ARHGDIB")
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=geoMetilData[selectedGenes,],
              expres=geoExprData[selectedGenes,],
              x1=1/3, x2=2/3, percY1=1/3, percY2=2/3,
              fileName=NULL, plotGrid = TRUE)
par(opt)
```

```{r plot4Genes3, echo=FALSE, message=FALSE, results='hide'}
#selectedGenes <- c("A1BG","A2ML1",  "A4GALT", "AAAS" )
selectedGenes <- c("ALDH1A2", "ALDH1A3", "APCDD1", "ARHGDIB")
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=TCGAMetilData[selectedGenes,],
              expres=TCGAExprData[selectedGenes,],
              x1=1/3, x2=2/3, percY1=1/3, percY2=2/3,
              fileName=NULL, plotGrid = TRUE)
par(opt)
```

Looking at the figure above shows that, although the genes may behave similarly between datasets, methods for selecting GRM must be robust and adaptable to, for example, distinct sample sizes.

## A specific gene Gene ZBTB18

Figure \@ref(fig:plotZBTB18) shows how the scatterplot looks like for a gene that has been described as regulated by methylation @Bazzocco2021.

The plots show that this gene will be selected by a method that selects genes negatively correlated rather than "L-shaped", although a grid could be adequately tuned for selecting it as shown in the plot.

```{r plotZBTB18, echo=FALSE, message=FALSE, results='hide', fig.cap='Scatter plots of gene ZBTB18. The images above suggest that this gene will be selected by a method that selects genes negatively correlated rather than L-shaped, even if, as the  images below show,  a grid can be tuned so that the gene can be seen as L-shaped'}
selectedGene <- c("ZBTB18", "ZBTB18")
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=DAMetilData[selectedGene, ],
              expres=DAExprData[selectedGene,], x1=1/3, x2=2/3,
              percY1=1/3, percY2=2/3,
              fileName=NULL, plotGrid = TRUE)
abline( lm(DAExprData["ZBTB18",]~ DAMetilData["ZBTB18", ]))
plotGenesMat (mets=DAMetilData[selectedGene, ],
              expres=DAExprData[selectedGene,], x1=1/3, x2=2/3,
              percY1=1/2, percY2=3/4,
              fileName=NULL, plotGrid = TRUE)
abline( lm(DAExprData["ZBTB18",]~ DAMetilData["ZBTB18", ]))
par(opt)
```

# Separating TRUE and FALSE L-shaped genes

## Genes extracted from DA dataset

Two small sets of genes\[\^1\] have been compiled with genes that were clearly L-shaped or clearly non-L-Shaped.
Because these sets have been prepared arbitrarily we decide not to use them as "TRUE POSITIVE" and "TRUE NEGATIVES" except for illustrative purposes.

```{r trueOrfalseLgenes1}
# Genes True / False in DA DataSet
# trueLGeneDF <-read.table(file.path(dadesDir, "genesTrueLNEW.txt"))
# (DAtrueLGeneNames <- as.character(trueLGeneDF[,1]))
# falseLGeneDF <- read.table(file.path(dadesDir, "genesFalseLNEW.txt"))
# (DAfalseLGeneNames <- as.character(falseLGeneDF[,1]))
# DAtrueLExpr <- DAExprData[rownames(DAExprData) %in% DAtrueLGeneNames ,]
# DAfalseLExpr <- DAExprData[rownames(DAExprData) %in% DAfalseLGeneNames ,]
# DAtrueLMet <- DAMetilData[rownames(DAMetilData) %in% DAtrueLGeneNames ,]
# DAfalseLMet <- DAMetilData[rownames(DAMetilData) %in% DAfalseLGeneNames ,]
# save(DAtrueLExpr, DAfalseLExpr, DAtrueLMet, DAfalseLMet, file="dades/DATRueFalse.RDA")
load(file="dades/DATRueFalse.RDA")
```

Figures \@ref(fig:plotLshapedTRUE1) and \@ref(fig:plotFALSE1) show the first four genes of each type for illustrative purposes.

```{r plotLshapedTRUE1, fig.cap='Example of L-shaped genes in the DA dataset'}
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=DAtrueLMet[1:4,], expres=DAtrueLExpr[1:4,],
              x1=1/3, x2=2/3, percY1=1/3, percY2=2/3,
              fileName=NULL, plotGrid = TRUE)
par(opt)
```

```{r plotFALSE1,  fig.cap='Example of non-L-shaped genes in the DA dataset'}
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=DAfalseLMet[1:4,], expres=DAfalseLExpr[1:4,],
              x1=1/3, x2=2/3, percY1=1/3, percY2=2/3,
               fileName=NULL, plotGrid = TRUE)
par(opt)
```

## GEO's TRUE and FALSE L-shaped genes list

Similarly to what we have done with the researcher's dataset we have **visually** selected a set of L-shaped genes Two small sets of genes have been compiled with genes that were clearly L-shaped or clearly non-L-Shaped.
Because these sets have been prepared arbitrarily we decide not to use them as "TRUE POSITIVE" and "TRUE NEGATIVES" except for illustrative purposes.

```{r trueOrfalseLgenes2}
# Genes True / False in GEO dataset
# GEOTrueFalse <- read.table(file.path(dadesDir, "GEOTrueFalse.txt"))
# (GEOtrueLGeneNames <- as.character(GEOTrueFalse[GEOTrueFalse[,2]!=0,1]))
# (GEOfalseLGeneNames <- as.character(GEOTrueFalse[GEOTrueFalse[,2]==0,1]))
# GEOtrueLExpr <- geoExprData[rownames(geoExprData) %in% GEOtrueLGeneNames ,]
# GEOfalseLExpr <- geoExprData[rownames(geoExprData) %in% GEOfalseLGeneNames ,]
# GEOtrueLMet <- geoMetilData[rownames(geoMetilData) %in% GEOtrueLGeneNames ,]
# GEOfalseLMet <- geoMetilData[rownames(geoMetilData) %in% GEOfalseLGeneNames ,]
# save(GEOtrueLExpr, GEOfalseLExpr, GEOtrueLMet, GEOfalseLMet, file="dades/GEOTRueFalse.RDA")
load(file="dades/GEOTRueFalse.RDA")
```

Figures \@ref(fig:plotLshapeTRUE2) and \@ref(fig:plotFALSE2) show the first four genes of each type for illustrative purposes.

```{r plotLshapeTRUE2, fig.cap='Example of L-shaped genes in the GEO dataset'}
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=GEOtrueLMet[1:4,], expres=GEOtrueLExpr[1:4,],
               fileName=NULL, plotGrid = TRUE)
par(opt)
```

```{r plotFALSE2,  fig.cap='Example of non-L-shaped genes in the GEO dataset'}
opt<- par(mfrow=c(2,2))
plotGenesMat (mets=GEOfalseLMet[1:4,], expres=GEOfalseLExpr[1:4,],
              x1=1/3, x2=2/3, y1=y1, y2=y2,
               fileName=NULL, plotGrid = TRUE)
par(opt)
```

## Simulating L-shaped scatterplots

An algorithm to simulate L-shaped genes has been developed in order to test for the capability of the scoring system to discriminate between TRUE or FALSE L-Shape

# Scoring scatterplots

## The "three band rule"

After trying different approaches to detect L-shapes, one often comes back to a naive approach like *"L-shaped" genes should show an L shape in the scatterplot, that is, values should tend to be scattered near the vertical and horizontal axes, and the more we move from these positions the least L-shaped the gene should be*.

This idea can be made more explicit by introducing a "three-band rule" as follows:

1.  Overimpose a $3\times 3$ grid on the scatterplot.

2.  Classify the scatterplot as **"L" or "non-L"** based on a small set of conditions:

    2.1 There must be a *minimum* number of points in the upper-left (cell (1,1)) and lower right (cell (3,3)) corners of the grid.

    2.2 There must be a *maximum* number of points in the upper right (cell (1,3)) because points there mean hypermethylation and hyperexpression which is the opposite of what we are looking for.

    2.3 We will usually *not require to have a minimum of points in cell (3,1)* unless we are really willing to have an L-shape (in our setting we will also be happy tho recover diagonals, which also reflect a negative correlation!).

3.  Score points on each subgrid in such a way that

    3.1 Points in permitted regions (left-outer margin, i.e. cells: (1,1), (2,2), (3,1), (3,2), (3,3)) score positively if the scatterplot has been classified as L or zero if it has been classified as non-L.

    3.2 Points in non-desired regions (outer band. i.e. cells (1,2), (1,3), (2,3)) score negatively in all cases.

    3.3 Some regions may be declared neutral and not-score, such as cell (2,2).

4.  *Use cross-validation to tune scoring parameters* (**if a set of positive and negative L-shaped genes is available**).

The previous scheme can be summarized using the following equation.
$$
S(X) = W_L \circ X \times \mathbf{1}_L(X) + W_{L^C} \circ X \times \mathbf{1}_{L^c}(X),
$$ where

-   ${X}$ is the matrix of *counts*, i.e. the number of counts in each cell of the grid,
-   ${W_L}$ is the matrix of scores per cell and point *if the scatterplot has been classified as* $L$,
-   ${W_{L^c}}$ is the matrix of scores per cell and point *if the scatterplot has been classified as non-*$L$ ($L^c$), and $\circ$ represents the hadamard product of the two matrices $W_{L/L^c}$ (i.e. elementwise multiplication of the two matrices) and $\mathbf{1}_{L/L^c}()$ is the indicator function for $L$ or $L^c$.

The fact that the scatterplot is assigned to $L$ or $L^c$ can also be described as the hadamard product of three matrices: $$
\mathbf{1}_L(X) = \bigwedge_{i,j} X \circ C \circ \left( mMP \times \sum_{i,j}x_{ij}\right),
$$ where

-   ${X}$ is the matrix of *counts*, i.e. the number of counts in each cell of the grid,
-   $C$ is the matrix of conditions to be verified *if the scatterplot has to be classified as* $L$,
-   $mMP$ is the matrix of minimum and Maximum Percentages of points to have in each cell *if the scatterplot has to be classified as* $L$,
-   $\circ$ represents the pointwise logical operation which allows that the product of the three cells becomes a logical operation and
-   $\bigwedge_{i,j}$ represents an logical "AND" operation of all cells, that is if all cells are TRUE the result is assigned to $L$ and if one fails it is assigned to $L^c$.

This idea is summarized in figure <a href="`{r anchorLoc('Lscore')`">here</a>

```{=tex}
\begin{figure}[htbp] <!-- ?--->
\centering
    <img src="./images/Lscoring.png" width="50%">
\caption{The heuristic method is based in scoring differently depending on where the points are found on a grid<a name="Lscore}"></a>
\end{figure}
```
<!-- ?--->

## Computing on a grid

We have developed several functions to help detect and select L--shape scatterplots.
Their use is described in the package help but they are illustrated here to clarify the code below.

-   *calcFreqs* counts the number of points in each cell of the grid for given vertical (defined by parameters *x1*, *x2*) and horizontal lines (defined by parameters *y1*, *y2*, *percY1*, *percY2*) <!-- * _binScore_ scores a scatterplot following the \"three bands rule\" described below.--->

-   *binScore* classifies (*scores binarily*) a scatterplot based on the rules described above, that is it checks if the minimal assumptions for an L-shape hold or not.
    It needs a matrix of *min-max frequency counts*.

-   *numScore* scores a scatterplot using a matrix of weights that defines the score given to each point depending on the cell where it is located.

    -   If the scatterplot has been classified as having L-shape all points are scored, those in favorable regions score positively and those in non-favorable regions negatively.
    -   If the scatterplot has not been classifed as "L" only points in non-favourable regions score negatively. <!-- * _scoreGenesMat_ is a wrapper for counting the L-shape genes found in two matrices that is it applies the _binScore_ function to all the genes in a (pair of) matrices.--->

-   *scoreGenesMat* is a wrapper for scoring the genes provided in two related matrices that is, it first applies the *binScore* function and depending on its results it computes the *numericScore* function with all the genes in the (pair of) matrices.

-   Function *plotGenesMat* is not a computing function but it is worth to enumerate it here because it complements the other functions by allowing to visualize the data that have generated a certain score from a given scatterplot.

# Examples (1): Scoring the TRUE/FALSE DA dataset

The first example below show that genes that have been marked as TRUE or FALSE L in the DA dataset **may** score different.

```{r testScoreGenes1TRUE}
xVecTrue<- as.numeric(DAtrueLMet[1,])
yVecTrue<- as.numeric(DAtrueLExpr[1,])
reqPercentages <- matrix (c(10, 20, 0, 5, 0, 20, 0, 5, 5), nrow=3, byrow=TRUE)

messageTitle("Frequency count in first \"TRUE\" gene")
(geneGridTrue <- calcFreqs(xMet=xVecTrue, yExp=yVecTrue, x1=1/3, x2=2/3,
                          y1=NULL, y2=NULL, percY1=1/3, percY2=2/3))
(maxminCountsT <- toReqMat (sum(geneGridTrue), reqPercentages))
(aWeightMifL=matrix (c(2,-2,-25,1,0,-2,1,1,2), nrow=3, byrow=TRUE))
(aWeightMifNonL=matrix (c(0,-2,-25,0,0,-2,0,0,0), nrow=3, byrow=TRUE))

messageTitle("Binary and numeric soring in first \"TRUE\" gene")
(binSc<- binScore (geneGridTrue, maxminCountsT ))
(nsT<- numScore(geneGridTrue, LShaped = binSc, aWeightMifL, aWeightMifNonL))
```

```{r testScoreGenes1FALSE}
xVecFalse<- as.numeric(DAfalseLMet[1,])
yVecFalse<- as.numeric(DAfalseLExpr[1,])

messageTitle("Frequency count in first \"FALSE\" gene")
(geneGridFalse <- calcFreqs(xMet=xVecFalse, yExp=yVecFalse, x1=1/3, x2=2/3,
                          y1=NULL, y2=NULL, percY1=1/3, percY2=2/3))
(maxminCountsF <- toReqMat (sum(geneGridFalse), reqPercentages))
# Same value as maxminCountsT because it depends only on required percentages and sample size
messageTitle("Binary and numeric soring in first \"FALSE\" gene")
(binSc<-binScore (geneGridFalse, maxminCountsF))
(nsF<- numScore(geneGridFalse, LShaped = binSc, aWeightMifL, aWeightMifNonL))
```

```{r testScoreGenes2a, echo=FALSE}
messageTitle("Scoring TRUE L-genes in the toy example dataset")
sampleSize <- dim(DAtrueLMet)[2]
(thisWeightMifL=matrix (c(2,-2,-sampleSize,1,0,-2,1,1,2), nrow=3, byrow=TRUE))
(thisWeightMifNonL=matrix (c(0,-2,-sampleSize,0,0,-2,0,0,0), nrow=3, byrow=TRUE))
(reqPercentages <- matrix (c(10, 20, 0, 5, 0, 30, 10, 5, 20), nrow=3, byrow=TRUE))
sum(reqPercentages)
scoresTrue <- scoreGenesMat (mets=DAtrueLMet, expres=DAtrueLExpr,
                              aReqPercentsMat=reqPercentages,
                              aWeightMifL=thisWeightMifL,
                              aWeightMifNonL=thisWeightMifNonL )
cat("Number of scatterplots scored  : ", dim(scoresTrue)[1],"\n")
cat("Number of L-shape scatterplots : ", sum(scoresTrue[,1]),"\n")
table(scoresTrue[,1])
table(scoresTrue[,2])
```

```{r testScoreGenes2b, echo=FALSE}
messageTitle("Scoring FALSE L-genes in the toy example dataset")
sampleSize <- dim(DAfalseLMet)[2]
scoresFalse<- scoreGenesMat (mets=DAfalseLMet, expres=DAfalseLExpr,
                             aReqPercentsMat=reqPercentages,
                             aWeightMifL=thisWeightMifL,
                             aWeightMifNonL=thisWeightMifNonL )
cat("Number of scatterplots scored  : ", dim(scoresFalse)[1],"\n")
cat("Number of L-shape scatterplots : ", sum(scoresFalse[,1]),"\n")
table(scoresFalse[,1])
table(scoresFalse[,2])
```

# Examples (1):  Putting all together to select L-shaped genes

The goal of developing all these functions is being able to select L-shaped genes from a paired dataset (expression-methylation) in a way that is as flexible and as rapid and as good as possible.

Essentially the process of selecting L-shape genes consists of three steps:

1.  Select datasets (a pair of row-column matched matrices, one for expression and one for methylation).

2. Set parameters:

2.1 Grid definition

2.2 Binary Scoring

2.3.Numerical Scoring

4.  Score the selected data and return classificaation (scores and group) and plots for each gene.

## Parameters selection

As it has been shown in the examples above the functions may be applied sequentially to an expression-methylation pair using the *scoreGenesMat* function.

We have selected a different set of parameters for DA and GEO datasets than for TCGA data, because as the sample size increases it seems reasonable (necessary) to be mor permisive.
Strictly speaking only required percentages have been changed, not weights.

### Parameters for DA and GEO datasets (small samples)

```{r}
require(kableExtra)
showTable <- function(x){
 x %>% as.table() %>%
     kbl() %>%
     kable_styling(full_width = F)
}
```

```{r setParameters4DAandGEO}
reqPercentages   <- matrix (c(10, 20, 1, 4, 30, 20, 0, 5, 10), nrow=3, byrow=TRUE)
sum(reqPercentages)
maxminCounts <- toReqMat(dim(DAMetilData)[2], reqPercentages) # Informative. NOt used in calculations because it is computed within the loop

theWeightMifL=matrix (c(2,-2,-25,1,0,-2,1,1,2), nrow=3, byrow=TRUE)

theWeightMifNonL=matrix (c(0,-2,-25,0,0,-2,0,0,0), nrow=3, byrow=TRUE)
```

```{r}
showTable(reqPercentages)
showTable(maxminCounts)
showTable(theWeightMifL)
showTable(theWeightMifNonL)
```

### Parameters for TCGA datasets (big samples)

```{r setParameters4TCGA}
reqPercentages4TCGA <- matrix (c(4, 20, 5, 2, 40, 20, 1, 3, 5), nrow=3, byrow=TRUE)
sum(reqPercentages4TCGA)
maxminCounts4TCGA <- toReqMat(dim(TCGAMetilData)[2], reqPercentages4TCGA)

theWeightMifL=matrix (c(2,-2,-25,1,0,-2,1,1,2), nrow=3, byrow=TRUE)
theWeightMifNonL=matrix (c(0,-2,-25,0,0,-2,0,0,0), nrow=3, byrow=TRUE)
```

```{r}
showTable(reqPercentages)
showTable(maxminCounts)
showTable(theWeightMifL)
showTable(theWeightMifNonL)
```

## Scoring datasets

Once the parameters have been set we can proceed to score and classify each dataset.

```{r selectLGenesFromAllDA1}
sampleSize <- dim(DAMetilData)[2]
numGenes <-   dim(DAMetilData)[1]

messageTitle("Scoring ALL genes in the DA (microarrays) dataset")

scoresDA1 <- scoreGenesMat (mets=DAMetilData[1:numGenes,],
							expres=DAExprData[1:numGenes,],
                            aReqPercentsMat=reqPercentages,
                            aWeightMifL=theWeightMifL,
                            aWeightMifNonL=theWeightMifNonL )
cat("Number of scatterplots scored  : ", dim(scoresDA1)[1],"\n")
cat("Number of L-shape scatterplots : ", sum(scoresDA1[,1]),"\n")
head(scoresDA1)
table(scoresDA1[,1])
```

```{r selectLGenesFromAllGEO}
messageTitle("Scoring ALL genes in the GEO dataset")
sampleSize <- dim(geoMetilData)[2]
numGenes <-   dim(geoMetilData)[1]

scoresGEO <- scoreGenesMat (mets=geoMetilData[1:numGenes,],
                            expres=geoExprData[1:numGenes,],
                            aReqPercentsMat=reqPercentages,
                            aWeightMifL=theWeightMifL,
                            aWeightMifNonL=theWeightMifNonL )
cat("Number of scatterplots scored  : ", dim(scoresGEO)[1], "\n")
cat("Number of L-shape scatterplots : ", sum(scoresGEO[,1]), "\n")
table(scoresGEO[,1])
```

```{r selectLGenesFromAllTCGA}
(sampleSize <- dim(TCGAMetilData)[2])
(numGenes <-   dim(TCGAMetilData)[1])
theGenes <- 1:numGenes
reqPercentages <- matrix (c(3, 20, 5, 2, 40, 20, 1, 3, 6), nrow=3, byrow=TRUE)
sum(reqPercentages)
(maxminCounts <- toReqMat(sampleSize, reqPercentages))

(theWeightMifL=matrix (c(2,-2,-sampleSize/5,1,0,-2,1,1,2), nrow=3, byrow=TRUE))
(theWeightMifNonL=matrix (c(0,-2,-sampleSize/5,0,0,-2,0,0,0), nrow=3, byrow=TRUE))

 messageTitle("Scoring ALL genes in the TCGA (microarrays) dataset")

# theGenes <- c("ALDH1A2", "ALDH1A3", "APCDD1", "ARHGDIB", "ARHGDIG", "APC")
scoresTCGA <- scoreGenesMat (mets=TCGAMetilData[theGenes,],
							              expres=TCGAExprData[theGenes,],
							               x1=1/3, x2=2/3,
                            aReqPercentsMat=reqPercentages,
                            aWeightMifL=theWeightMifL,
                            aWeightMifNonL=theWeightMifNonL )
cat("Number of scatterplots scored  : ", dim(scoresTCGA)[1],"\n")
cat("Number of L-shape scatterplots : ", sum(scoresTCGA[,1]),"\n")
head(scoresTCGA)
table(scoresTCGA[,1])
```

## Results visualization

We may use the scores obtained to sort genes from `most" to`least" L-shaped.

```{r sortGenes}
orderDA1<- order(scoresDA1[,1], scoresDA1[,2], rownames(scoresDA1),
                 method="radix", decreasing=c(TRUE, TRUE, FALSE))
orderGEO<- order(scoresGEO[,1], scoresGEO[,2], rownames(scoresGEO),
                 method="radix", decreasing=c(TRUE, TRUE, FALSE))
orderTCGA<- order(scoresTCGA[,1], scoresTCGA[,2], rownames(scoresTCGA),
                 method="radix", decreasing=c(TRUE, TRUE, FALSE))
```

We can now use this ordering to plot all genes starting by those that we consider L-shaped.

Alternatively, instead of plotting all genes we may select L genes and plot only these.
The resulting plots are available in files

-   `DAExprLGenesScores.pdf`
-   `geoExprLGenesScores.pdf`
-   `TCGAExprLGenesScores.pdf`

```{r selectLGenes1}
LgenesDAExpr <- DAExprData[scoresDA1[,"logicSc"],]
dim(LgenesDAExpr)
geneListLDAExpr <- rownames(DAExprData[scoresDA1[,"logicSc"],])
plotGenesMat (mets=DAMetilData[geneListLDAExpr,],
              expres=DAExprData[geneListLDAExpr,],
              fileName ="DAExprLGenesScores.pdf",
              text4Title = scoresDA1[geneListLDAExpr,"numericSc"])
LgenesGEOExpr <- geoExprData[scoresGEO[,"logicSc"],]
dim(LgenesGEOExpr)
geneListLGEOExpr <- rownames(geoExprData[scoresGEO[,"logicSc"],])
plotGenesMat (mets=geoMetilData[geneListLGEOExpr,],
              expres=geoExprData[geneListLGEOExpr,],
              fileName ="geoExprLGenesScores.pdf",
              text4Title = scoresGEO[geneListLGEOExpr,"numericSc"])

LgenesTCGA <- TCGAExprData[scoresTCGA[,"logicSc"],]
dim(LgenesTCGA)
geneListLTCGA <- rownames(TCGAExprData[scoresTCGA[,"logicSc"],])
plotGenesMat (mets=TCGAMetilData[geneListLTCGA,],
              expres=TCGAExprData[geneListLTCGA,],
              fileName ="TCGALGenesScores.pdf",
              text4Title = scoresTCGA[geneListLTCGA,"numericSc"])

save(geneListLDAExpr, geneListLGEOExpr,  geneListLTCGA,
     file=file.path(resultsDir, "geneListsL1.RData"))

myVenn4<- venn.diagram(x=list(DAMicroarrays=geneListLDAExpr,
                              GEOData=geneListLGEOExpr,
                              TCGAData=geneListLTCGA),
                              filename=NULL, lty = "blank",
                              fill=c("skyblue", "red", "yellow"))
grid.draw(myVenn4)
```

Or it can be applied to a pre-filtered subset, such as genes showing a significantly negative correlation.

# References