%\VignetteIndexEntry{CSAMA 2014: RNA-Seq differential expression workflow}
%\VignettePackage{DESeq2}
%\VignetteEngine{knitr::knitr}

% To compile this document
% library('knitr'); rm(list=ls()); knit('RNA-Seq-Analysis-Lab.Rnw')

\documentclass[12pt]{article}
\usepackage{theorem}

\newcommand{\deseqtwo}{\Biocpkg{DESeq2}}
\newcommand{\lowtilde}{\raise.17ex\hbox{$\scriptstyle\mathtt{\sim}$}}

\theoremstyle{break} \newtheorem{Ex}{Exercise}

<<knitr, echo=FALSE, results="hide">>=
library("knitr")
opts_chunk$set(tidy=FALSE,dev="png",fig.show="hide",
               fig.width=4,fig.height=4.5,
               message=FALSE)
@ 

<<style, eval=TRUE, echo=FALSE, results="asis">>=
BiocStyle::latex()
@

<<loadDESeq2, echo=FALSE>>=
# in order to print version number below
library("DESeq2")
@


\author{Michael Love$^{1*}$, Simon Anders$^{2}$, Wolfgang Huber$^{2}$ \\[1em] \small{$^{1}$ Department of Biostatistics, Dana Farber Cancer Institute and} \\ \small{Harvard School of Public Health, Boston, US;} \\ \small{$^{2}$ European Molecular Biology Laboratory (EMBL), Heidelberg, Germany} \\ \small{\texttt{$^*$michaelisaiahlove (at) gmail.com}}}

\title{CSAMA 2014: RNA-Seq differential expression workflow}

\begin{document}

\maketitle

\begin{abstract}
  This lab will walk you through an end-to-end RNA-Seq differential expression workflow.  We
  will start from the FASTQ files, align to the reference genome, prepare gene expression
  values as a count table by counting the sequenced fragments, perform differential gene expression analysis, and
  visually explore the results. 
  
  This lab covers the basic introduction. 
  For a more in-depth explanation of the advanced details, we advise you to proceed to the vignette of
  the \Biocpkg{DESeq2} package package, \emph{Differential analysis of count data -- the
  \emph{DESeq2} package}.
  For a treatment of exon-level differential expression, we refer to the vignette 
  of the \Biocpkg{DEXSeq} package, \emph{Analyzing RNA-seq data for differential exon usage with the \emph{DEXSeq} package}.
  % basic gene set enrichment
\end{abstract}

<<options, results="hide", echo=FALSE>>=
options(digits=3, width=80)
@


\tableofcontents

\section{Input data}

\label{Haglund}As example data for this lab, we will use publicly available data from the article by Felix Haglund et al.,
\emph{Evidence of a Functional Estrogen Receptor in Parathyroid Adenomas}, J Clin Endocrin Metab, Sep
2012\footnote{\url{http://www.ncbi.nlm.nih.gov/pubmed/23024189}}.

The purpose of the experiment was to investigate the role of the
estrogen receptor in parathyroid tumors. The investigators derived 
primary cultures of parathyroid adenoma cells from 4 patients. These 
primary cultures were treated with diarylpropionitrile (DPN), 
an estrogen receptor $\beta$ agonist, or with 4-hydroxytamoxifen (OHT).  
RNA was extracted at 24 hours and 48 hours from cultures under 
treatment and control.  The blocked design of the experiment allows
for statistical analysis of the treatment effects while controlling for patient-to-patient variation.

Part of the data from this experiment is provided in the Bioconductor data package
\Biocexptpkg{parathyroidSE}.



\subsection{Preparing count matrices}

As input, the \deseqtwo{} package expects count data as obtained, e.\,g.,
from RNA-Seq or another high-throughput sequencing experiment, in the form of a
matrix of integer values. The value in the $i$-th row and the $j$-th column of
the matrix tells how many reads have been mapped to gene $i$ in sample $j$.
Analogously, for other types of assays, the rows of the matrix might correspond
e.\,g.\ to binding regions (with ChIP-Seq) or peptide sequences (with
quantitative mass spectrometry).

The count values must be raw counts of sequencing reads. This is important
for \deseqtwo{}'s statistical model to hold, as only the actual
counts allow assessing the measurement precision correctly. Hence, please
do not supply other quantities, such as (rounded) normalized counts, or
counts of covered base pairs -- this will only lead to nonsensical results.

\subsection{Aligning reads to a reference}

The computational analysis of an RNA-Seq experiment begins earlier: 
what we get from the sequencing machine is a set of FASTQ files that contain the nucleotide sequence of each read and a quality score at each position. 
These reads must first be aligned to a reference genome or transcriptome.
It is important to know if the sequencing experiment was single-end or
paired-end, as the alignment software will require the user to specify
both FASTQ files for a paired-end experiment. The output of this alignment step is commonly stored in a 
file format called \href{http://samtools.github.io/hts-specs}{\texttt{BAM}}.

A number of software programs exist to align reads to the reference genome, 
and the development is too rapid for this document to provide a current 
list. We recommend consulting benchmarking papers that discuss the advantages
and disadvantages of each software, which include accuracy, ability to 
align reads over splice junctions, speed, memory footprint, and many other 
features. 

Here we use the TopHat2 spliced alignment 
software\footnote{\url{http://tophat.cbcb.umd.edu/}} \cite{Kim2013TopHat2}
in combination with the Bowtie index available at the Illumina iGenomes 
page\footnote{\url{http://tophat.cbcb.umd.edu/igenomes.html}}.  
For full details on this software and on the iGenomes, users should follow the links
to the manual and information provided in the links in the footnotes.
For example, the paired-end RNA-Seq reads for the \Biocexptpkg{parathyroidSE} package were aligned 
using TopHat2 with 8 threads, with the call:

\begin{verbatim}
  tophat2 -o file_tophat_out -p 8 genome file_1.fastq file_2.fastq 
\end{verbatim}

\begin{verbatim}
  samtools sort -n file_tophat_out/accepted_hits.bam _sorted
\end{verbatim}

The second line sorts the reads by name rather than by genomic position, which is necessary
for counting paired-end reads within Bioconductor.
This command uses the SAMtools software\footnote{\url{http://samtools.sourceforge.net}} \cite{Li2009SAMtools}.
The BAM files for a number of sequencing runs can then be used to generate
count matrices, as described in the following section.

\subsection{Example BAM files}

% tangential here...
% The Bioconductor data package \Biocexptpkg{parathyroidSE} provides the data from
% the Haglund et al.\ experiment in the form of a \Rclass{SummarizedExperiment} object, which contains
% a count table, with rows for genes and columns for samples. The table entries indicate 
% how many reads have been unambiguously mapped to each gene in each of the experiment's
% 23 samples. Furthermore, 
Besides tha main count table, which we will use later, the \Biocexptpkg{parathyroidSE} package also
contains a small subset of the raw data 
from the Haglund et al.\ experiment, namely three BAM file
each with a subset of the reads from three of the samples. We will use these files to demonstrate
how a count table can be constructed from BAM files. Afterwards, we will load the full count table
corresponding to all samples and all data, which is already provided in the same package,
and will continue the analysis with that full table.

We load the data package with the example data

<<loadExpPck>>=
library( "parathyroidSE" )
@

The R function \Rfunction{system.file} can be used to find out where on your computer the files from a package
have been installed. Here we ask for the full path to the \texttt{extdata} directory, which is part of
the \Biocexptpkg{parathyroidSE} package:

<<findExtData>>=
extDataDir <- system.file("extdata", package = "parathyroidSE", mustWork = TRUE)
extDataDir
@

In this directory, we find the three BAM files (and some other files):

<<LookIntoExtData>>=
list.files( extDataDir )
@

Typically, we have a table with experimental meta data for our samples. For these three files, it is as follows:

\begin{tabular}{llll}
sampleName & fileName & treatment & time \\ \hline
Ctrl\_24h\_1 & SRR479052.bam & Control & 24h \\
Ctrl\_48h\_1 & SRR479053.bam & Control & 48h \\
DPN\_24h\_1 & SRR479054.bam & DPN & 24h 
\end{tabular}

To avoid mistakes, it is helpful to store such a sample table explicitly in a text file and load it. 

\begin{Ex}Construct
a table with the content shown above on your disk using a spread sheet application such as Microsoft 
Excel and save the sheet in CSV (comma-separated values) format (or simply use a plain text editor). 
\end{Ex}

Load it with:

<<loadSampleTable,eval=FALSE>>=
sampleTable <- read.csv( "/path/to/your/sampleTable.csv", header=TRUE )
@

<<makeSampleTable,echo=FALSE>>=
sampleTable <- data.frame(
   sampleName = c( "Ctrl_24h_1", "Ctrl_48h_1", "DPN_24h_1" ),
   fileName = c( "SRR479052.bam", "SRR479053.bam",  "SRR479054.bam" ),
   treatment =  c( "Control", "Control", "DPN" ),
   time = c( "24h", "48h", "24h" ) )
@

This is how the sample table should look like

<<showSampleTable>>=
sampleTable
@

Using the fileName column in the table, we construct the full paths to the files we 
want to perform the counting operation on:

<<getBamList>>=
bamFiles <- file.path( extDataDir, sampleTable$fileName )
bamFiles
@

\subsection{Counting reads in genes}\label{sec:count}

To count how many read map to each gene, we need transcript annotation. Download
the current GTF file with human gene annotation from Ensembl. (In case the network
is too slow for that, use the truncated version of this file, called \texttt{Homo\_sapiens.GRCh37.75.subset.gtf.gz}, 
which we have placed on the course server.)

From this file, the function \Rfunction{makeTranscriptDbFromGFF} from the \Biocpkg{GenomicFeatures}
constructs a database of all annotated transcripts.

<<eval=FALSE>>=
library( "GenomicFeatures" )
hse <- makeTranscriptDbFromGFF( "/path/to/your/genemodel_file.GTF", format="gtf" )
exonsByGene <- exonsBy( hse, by="gene" )
@ 

<<loadExonsByGene, echo=FALSE>>=
library( "GenomicFeatures" )
hse <- makeTranscriptDbFromGFF( "Homo_sapiens.GRCh37.75.subset.gtf.gz", format="gtf" )
exonsByGene <- exonsBy( hse, by="gene" )
@  

In the last step, we have used the \Rfunction{exonsBy} function to bring the transcriptome data 
base into the shape of a list of all genes, 

<<showExons>>=
exonsByGene
@

\begin{Ex}
Note the warnings issued by \Rfunction{makeTranscriptDbFromGFF}. Can we safely ignore them?
\end{Ex}

\begin{Ex}
In \Robject{exonsByGene}, inspect the genomic intervals given for the exons of the first gene. Note that
they are not disjunct (they overlap). How comes? Will this influence results in the following step? 
\end{Ex}

After these preparations, the actual counting is easy. The function \Rfunction{summerizeOverlap} from
the \Biocpkg{GenomicAlignments} package will do this.

<<countIt>>=
library( "GenomicAlignments" )
se <- summarizeOverlaps( exonsByGene, BamFileList( bamFiles ), mode="Union",
	singleEnd=FALSE, ignore.strand=TRUE, fragments=TRUE )
@

We use the counting mode \Robject{"Union"}, which indicates that those reads which overlap any portion of
exactly one feature are counted. For more information on the different counting modes,
see the help page for \Rfunction{summarizeOverlaps}.
As this experiment produced paired-end reads, we specify \Robject{singleEnd = FALSE}.
As protocol was not strand-specific, we specify \Robject{ignore.strand = TRUE}.
\Robject{fragments = TRUE} indicates that we also want to count reads with unmapped pairs. 
This last argument is only for use with paired-end experiments.

Remember that we have only used a small subset of reads from the
original experiment: for 3 samples and for 100 genes. Nevertheless, we can still 
investigate the resulting \Rclass{SummarizedExperiment} by looking at the counts
in the \Robject{assay} slot, the phenotypic data about the samples in \Robject{colData} slot
(in this case an empty \Rclass{DataFrame}), and the data about the genes in the \Robject{rowData} slot.
Figure~\ref{figure/beginner_plotSE} explains the basic structure of the \Rclass{SummarizedExperiment} class.

<<examineSE>>=
se
head( assay(se) )
colSums( assay(se) )
colData(se)
rowData(se)
@ 

Note that the \Robject{rowData} slot is a \Rclass{GRangesList}, which contains
all the information about the exons for each gene, i.e., for each row of the count table.

The \Robject{colData} slot, so far empty, should contain all the meta data. We hence assign our sample table to
it:

<<setColData>>=
colData(se) <- DataFrame( sampleTable )
@

We can also use the sampleName table to name the columns of our data matrix:

<<setColNames>>=
colnames(se) <- sampleTable$sampleName
head( assay(se) )
@

This \Rclass{SummarizedExperiment} object \Robject{se} is then all we need to start our analysis.
%In the following section we will show how to create the data object which is used in \Rpackage{DESeq2}, 
%either using the \Rclass{SummarizedExperiment}, or in general, a count table which has 
%been loaded into R.



<<beginner_plotSE, dev="pdf", echo=FALSE>>=
par(mar=c(0,0,0,0))
plot(1,1,xlim=c(0,100),ylim=c(0,100),bty="n",
     type="n",xlab="",ylab="",xaxt="n",yaxt="n")
polygon(c(45,80,80,45),c(10,10,70,70),col=rgb(1,0,0,.5),border=NA)
polygon(c(45,80,80,45),c(68,68,70,70),col=rgb(1,0,0,.5),border=NA)
text(62.5,40,"assay(s)")
text(62.5,30,"e.g. 'counts'")
polygon(c(20,40,40,20),c(10,10,70,70),col=rgb(0,0,1,.5),border=NA)
polygon(c(20,40,40,20),c(68,68,70,70),col=rgb(0,0,1,.5),border=NA)
text(30,40,"rowData")
polygon(c(45,80,80,45),c(75,75,90,90),col=rgb(.5,0,.5,.5),border=NA)
polygon(c(45,47,47,45),c(75,75,90,90),col=rgb(.5,0,.5,.5),border=NA)
text(62.5,82.5,"colData")
@ 

\incfig{figure/beginner_plotSE}{.5\textwidth}{Diagram of \Rclass{SummarizedExperiment}}{
  The component parts of a \Rclass{SummarizedExperiment} object. 
  The \Robject{assay(s)} (red block) contains the matrix (or matrices) of 
  summarized values, the \Robject{rowData} (blue block) contains information about the 
  genomic ranges, and the \Robject{colData} (purple block) contains information 
  about the samples or experiments. The highlighted line in each block represents the 
  first row (note that the first row of \Robject{colData} lines up with the first
  column of the \Robject{assay}.
}


\subsection{The DESeqDataSet, column metadata, and the design formula}

Bioconductor software packages often have a special class of data object,
which contains special slots and requirements. The data object class in \deseqtwo{}
is the \Rclass{DESeqDataSet}, which is built on top of the \Rclass{SummarizedExperiment} class.
One main differences is that the \Robject{assay} slot is instead accessed using the
\Rfunction{count} accessor, and the values in this matrix must be non-negative integers.

A second difference is that the \Rclass{DESeqDataSet} has an associated ``design formula''.
The design is specified at the beginning of the analysis, as it will inform 
many of the \deseqtwo{} functions how to treat the samples in the analysis 
(one exception is the size factor estimation, i.\,e., the adjustment for differing
library sizes, which does not depend on the design formula). 
The design formula tells which variables in the column metadata table (\Robject{colData})
specify the experimental design and how these factors should be used in the analysis.

The simplest design formula for differential expression would be
\Rfunction{\lowtilde{} condition}, where \Robject{condition} is a
column in \Robject{colData(dds)} which specifies which of two (or more
groups) the samples belong to.  For the parathyroid experiment, we
will specify \Rfunction{\lowtilde{} patient + treatment}, which means
that we want to test for the effect of treatment (the last factor),
controlling for the effect of patient (the first factor).

You can use R's formula notation to express any
experimental design that can be described within an ANOVA-like framework.
Note that \deseqtwo{} uses the same formula notation as, for instance,
the \Rfunction{lm} function of base R. If the question of interest is whether a fold change
due to treatment is different across groups, for example across patients, ``interaction
terms'' can be included using models such as \Rfunction{\lowtilde{} patient + treatment + patient:treatment}.
More complex designs such as these are covered in the other \deseqtwo{} vignette.

We now use R's \Rfunction{data} command to load a prepared \Rclass{SummarizedExperiment} 
that was generated from the publicly available sequencing data files
associated with the Haglund et al.~paper, described on page~\pageref{Haglund}.
The steps we used to produce this object were equivalent to those you worked through in Section~\ref{sec:count}, except that 
we used the complete set of samples and all reads.

<<loadEcs>>=
data( "parathyroidGenesSE" )
se <- parathyroidGenesSE
colnames(se) <- se$run
@

Supposing we have constructed a \Rclass{SummarizedExperiment} using one of the 
methods described in the previous section, we now need to make sure that the 
object contains all the necessary information about the samples, 
i.e., a table with metadata on the count table's columns stored in the \Robject{colData} slot:

<<coldataSE>>=
colData(se)[1:5,1:4]
@

Here we see that this object already contains an informative \Robject{colData} slot -- 
because we have already prepared it for you, as described in the \Biocexptpkg{parathyroidSE} vignette.
However, when you work with your own data, you will have to add the pertinent sample / phenotypic 
information for the experiment at this stage.
We highly recommend keeping this information in a comma-separated value (CSV) or 
tab-separated value (TSV) file, which can be exported from an Excel spreadsheet, and the assign
this to the \Robject{colData} slot, as shown in the previous section. 

Make sure that the order
of rows in your column data table matches the order of columns in the \Robject{assay} data slot.

\begin{Ex}
How have we ensured this when building the \Rclass{se} object in the previous section?
\end{Ex}

Once we have our fully annotated \Rclass{SummerizedExperiment} object, we can construct a 
\Rclass{DESeqDataSet} object from it, which will then form the staring point of the actual \Biocpkg{DESeq2}
package, described in the following sections. Here, we use the \Rclass{SummerizedExperiment} object
we got from the \Biocexptpkg{parathyroidSE} package and augment it by specifying an appropriate
design formula.

<<fromSE>>=
ddsFull <- DESeqDataSet( se, design = ~ patient + treatment )
@ 

Note that there are two alternative functions, \Rfunction{DESeqDataSetFromMatrix}
and \Rfunction{DESeqDataSetFromHTSeq}, which allow you to get started in case
you have your data not in the form of a \Rclass{SummerizedExperiment} object, but
either as a simple matrix of count values or a s output files from the \emph{htseq-count}
script from the \emph{HTSeq} Python package.


\subsection{Collapsing technical replicates}

There are a number of samples which were sequenced in multiple
runs. For example, sample \Robject{SRS308873} was sequenced twice. To see,
we list the respective columns of the \Rfunction{colData}. (The use
of \Rfunction{as.data.frame} forces R to show us the full list, not just
the beginning and the end as before.)

<<columnData>>=
as.data.frame( colData( ddsFull )[ ,c("sample","patient","treatment","time") ] )
@ 

We recommend to first add together technical replicates (i.e., libraries
derived from the same samples), such that we have one column per sample. 
We have implemented a convenience function for this, which can take 
am object, either \Rclass{SummarizedExperiment} or \Rclass{DESeqDataSet},
and a grouping factor, in this case the sample name, 
and return the object with the counts summed up for each unique sample. 
This will also rename the columns of the object, such that they match the unique names which
were used in the grouping factor. Optionally, we can provide
a third argument, \Robject{run}, which can be used to paste together 
the names of the runs which were collapsed to create the new object.
Note that \Robject{dds\$variable} is equivalent to \Robject{colData(dds)\$variable}.

<<collapse>>=
ddsCollapsed <- collapseReplicates( ddsFull,
                                   groupby = ddsFull$sample, 
                                   run = ddsFull$run )
head( as.data.frame( colData(ddsCollapsed)[ ,c("sample","runsCollapsed") ] ), 12 )
@ 

We can confirm that the counts for the new object are equal to the 
summed up counts of the columns that had the same value for the 
grouping factor:

<<checkCollapsed>>=
original <- rowSums( counts(ddsFull)[ , ddsFull$sample == "SRS308873" ] )
all( original == counts(ddsCollapsed)[ ,"SRS308873" ] )
@ 

\section{Running the DESeq2 pipeline}\label{sec:runDESeq}

Here we will analyze a subset of the samples, namely those taken after 48 hours, 
with either control, DPN or OHT treatment, taking into account the multifactor design. 

\subsection{Preparing the data object for the analysis of interest}

First we subset the relevant columns from the full dataset:

<<subsetCols>>=
dds <- ddsCollapsed[ , ddsCollapsed$time == "48h" ]
@

Sometimes it is necessary to drop levels of the factors, in case that all the
samples for one or more levels of a factor in the design have been removed.
If time were included in the design formula, the following 
code could be used to take care of dropped levels in this column.

<<refactor>>=
dds$time <- droplevels( dds$time )
@

It will be convenient to make sure that \texttt{Control} is the \emph{first} level in the
treatment factor, so that the default log2 fold changes are calculated as treatment over
control and not the other way around. The function \Rfunction{relevel} achieves this:

<<relevel>>=
dds$treatment <- relevel( dds$treatment, "Control" )
@

A quick check whether we now have the right samples:

<<multifactorColData>>=
as.data.frame( colData(dds) )
@ 

\subsection{Running the pipeline}

Finally, we are ready to run the differential expression pipeline.
With the data object prepared, the \deseqtwo{} analysis can now be run with a single
call to the function \Rfunction{DESeq}:

%<<subsetRows, echo=FALSE>>=
%idx <- which(rowSums(counts(dds)) > 0)[1:4000]
%dds <- dds[idx,]
%@ 

<<runDESeq, cache=TRUE>>=
dds <- DESeq(dds)
@ 

This function will print out a message for the various steps it performs.
These are described in more detail in the manual page for \Rfunction{DESeq}, 
which can be accessed by typing \Robject{?DESeq}. Briefly these are:
the estimation of size factors (which control for differences in the library size
of the sequencing experiments), the estimation of dispersion for each gene,
and fitting a generalized linear model.

A \Rclass{DESeqDataSet} is returned which contains all the fitted information
within it, and the following section describes how to extract out 
results tables of interest from this object.

\subsection{Inspecting the results table}

Calling \Rfunction{results} without any arguments will extract
the estimated log2 fold changes and $p$ values for the last variable
in the design formula. If there are more than 2 levels for this variable
-- as is the case in this analysis -- \Rfunction{results} will extract the 
results table for a comparison of the last level over the first level.
The following section describes how to extract other comparisons.

<<extractResults>>=
res <- results( dds )
res
@

As \Robject{res} is a \Robject{DataFrame} object, it carries metadata with
information on the meaning of the columns:

<<resCols>>=
mcols(res, use.names=TRUE)
@

The first column, \Robject{baseMean}, is a just the average of the normalized count values,
dividing by size factors, taken over all samples. The remaining four columns refer to a specific
\emph{contrast}, namely the comparison of the levels \emph{DPN}
versus \emph{Control} of the factor variable \emph{treatment}. See the help page 
for \Rfunction{results} (by typing \Rfunction{?results}) for information
on how to obtain other contrasts.

The column \Robject{log2FoldChange} is the effect size estimate. It tells us 
how much the gene's expression seems to have changed due to treatment with DPN
in comparison to control. This value is reported on a logarithmic scale to base 2: for example, a log2 fold change
of 1.5 means that the gene's expression is increased by a multiplicative factor of $2^{1.5}\approx 2.82$.

Of course, this estimate has an uncertainty associated with it, which is available in the
column \Robject{lfcSE}, the standard error estimate for the log2 fold change estimate.  We
can also express the uncertainty of a particular effect size estimate as the result of a
statistical test. The purpose of a test for differential expression is to test whether the
data provides sufficient evidence to conclude that this value is really different from
zero. \deseqtwo{} performs for each gene a \emph{hypothesis test} 
to see whether evidence is sufficient to decide against the \emph{null hypothesis}
that there is no effect of the treatment on the gene and that the observed difference
between treatment and control was merely caused by experimental variability (i.\,e., the
type of variability that you can just as well expect between different samples in the same
treatment group). As usual in statistics, the result of this test is reported as a 
\emph{$p$ value}, and it is found in the column \Robject{pvalue}. (Remember that a $p$ value indicates
the probability that a fold change as strong as the observed one, or even stronger, would
be seen under the situation described by the null hypothesis.)

We note that a subset of the $p$ values in \Robject{res} are \Robject{NA} (``not
available''). This is \Rfunction{DESeq}'s way of reporting that all counts for this gene
were zero, and hence not test was applied. In addition, $p$ values can be 
assigned \Robject{NA} if the gene was excluded from analysis because it 
contained an extreme count outlier. For more information, see the outlier
detection section of the advanced vignette.

\subsection{Other comparisons}

In general, the results for a comparison of any two levels of
a variable can be extracted using the \Robject{contrast} 
argument to \Rfunction{results}. The user should specify three values: 
the name of the variable, the name of the
level in the numerator, and the name of the level in the denominator.
Here we extract results for the log2 of the fold change of DPN $/$ Control.

<<resultsOther>>=
res <- results( dds, contrast = c("treatment", "DPN", "Control") )
res
@ 

If results for an interaction term are desired, the \Robject{name}
argument of \Rfunction{results} should be used. Please see the more advanced
vignette for more details.

\subsection{Adding gene names}

Our result table only uses Ensembl gene IDs, but gene names may be more
informative. Bioconductor's annotation packages help with mapping various
ID schemes to each other.

We load the annotation package \Rpackage{org.Hs.eg.db}:

<<loadOrg>>=
library( "org.Hs.eg.db" )
@

This is the organism annotation package (``org'') for \emph{Homo sapiens}
(``Hs''), organized as an \Rpackage{AnnotationDbi} package (``db''), using
Entrez Gene IDs (``eg'') as primary key.

To get a list of all available key types, use
<<keyType>>=
columns(org.Hs.eg.db)
@

Converting IDs with the native functions from the \Rpackage{AnnotationDbi}
package is currently a bit cumbersome, so we provide the following convenience function
(without explaining how exactly it works):

<<convertIDs>>=
convertIDs <- function( ids, fromKey, toKey, db, ifMultiple=c( "putNA", "useFirst" ) ) {
   stopifnot( inherits( db, "AnnotationDb" ) )
   ifMultiple <- match.arg( ifMultiple )
   suppressWarnings( selRes <- AnnotationDbi::select( 
      db, keys=ids, keytype=fromKey, columns=c(fromKey,toKey) ) )
   if( ifMultiple == "putNA" ) {
      duplicatedIds <- selRes[ duplicated( selRes[,1] ), 1 ]   
      selRes <- selRes[ ! selRes[,1] %in% duplicatedIds, ] }
   return( selRes[ match( ids, selRes[,1] ), 2 ] )
}
@

This function takes a list of IDs as first argument and their key type as the
second argument. The third argument is the key type we want to convert to,
the fourth is the \Rclass{AnnotationDb} object to use. Finally, the last
argument specifies what to do if one source ID maps to several target IDs: should 
the function return an NA or simply the first of the multiple IDs?

To convert the Ensembl IDs in the rownames of \Robject{res} to gene symbols
and add them as a new column, we use:
<<addSymbols>>=
res$hgnc_symbol <- convertIDs( row.names(res), "ENSEMBL", "SYMBOL", org.Hs.eg.db )
res$entrezgene <- convertIDs( row.names(res), "ENSEMBL", "ENTREZID", org.Hs.eg.db )
res
@

Now the results have the desired external gene ids:

<<showAnnot>>=
head(res,4)
@ 

\begin{Ex}
Go to the Ensembl web site, select the \emph{Biomart} tab, and redo our BioMart query by manually inputting
some of the Ensembl IDs and finding the HGNC names. What other data is available from this mart? Can you 
modify the code chunk above to add a column \Robject{chrom} to the \Robject{res} object that tells us 
for each gene which chromosome it resides on?
\end{Ex}

\section{Further points}

\subsection{Multiple testing}
Novices in high-throughput biology often assume that thresholding these $p$ values at a low value,
say 0.01, as is often done in other settings, would be appropriate -- but it is not. We briefly
explain why:

There are \Sexpr{sum( res$pvalue < 0.01, na.rm=TRUE )} genes with a $p$ value below
0.01 among the \Sexpr{sum( !is.na(res$pvalue) )} genes, for which the test succeeded
in reporting a $p$ value:

<<rawpvalue>>=
sum( res$pvalue < 0.01, na.rm=TRUE )
table( is.na(res$pvalue) )
@

Now, assume for a moment that the null hypothesis is true for all genes, i.e., no
gene is affected by the treatment with DPN. Then, by the definition of \emph{$p$ value},
we expect up to 1\% of the genes to have a $p$ value below 0.01. This amounts to
\Sexpr{round( sum( !is.na(res$pvalue) ) * 0.01 )} genes.
If we just considered the list of genes with a $p$ value below 0.01 as differentially
expressed, this list should therefore be expected to contain up to 
$\Sexpr{round( sum( !is.na(res$pvalue) ) * 0.01)} / \Sexpr{sum( res$pvalue < 0.01, na.rm=TRUE ) } = 
 \Sexpr{round( sum( !is.na(res$pvalue) ) * 0.01 / sum( res$pvalue < 0.01, na.rm=TRUE ) * 100, 0 )}$\% 
false positives!

\deseqtwo{} uses the so-called Benjamini-Hochberg (BH) adjustment; in brief, this method calculates
for each gene an \emph{adjusted $p$ value} which answers the following question: if one
called significant all genes with a $p$ value less than or equal to this gene's $p$ value threshold,
what would be the fraction of false positives (the \emph{false discovery rate}, FDR) among
them (in the sense of the calculation outlined above)? These values, called the BH-adjusted
$p$ values, are given in the column \Robject{padj} of the \Robject{results} object.

Hence, if we consider a fraction of 10\% false positives acceptable, we can consider all
genes with an \emph{adjusted} $p$ value below 10\%=0.1 as significant. How many such genes
are there?

<<adjpvalue>>=
sum( res$padj < 0.1, na.rm=TRUE )
@

We subset the results table to these genes and then sort it by the log2 fold change
estimate to get the significant genes with the strongest down-regulation

<<strongGenes>>=
resSig <- res[ which(res$padj < 0.1 ), ]
head( resSig[ order( resSig$log2FoldChange ), ] )
@

and with the strongest upregulation

<<strongGenesUp>>=
tail( resSig[ order( resSig$log2FoldChange ), ] )
@

\subsection{Diagnostic plots}

A so-called MA plot provides a useful overview for an experiment
with a two-group comparison:

<<beginner_MA>>=
plotMA( res, ylim = c(-3, 3) )
@ 

The plot (Fig.\ \ref{figure/beginner_MA}) represents each gene with a dot. The $x$ axis
is the average expression over all samples, the $y$ axis the log2
fold change between treatment and control. Genes with an adjusted $p$ value
below a threshold (here 0.1, the default) are shown in red. 
 
\incfig{figure/beginner_MA}{.5\textwidth}{MA-plot}{
  The MA-plot shows the log2 fold changes from the treatment over
  the mean of normalized counts, i.e. the average of counts normalized by
  size factor. The \deseqtwo{} package incorporates a prior on log2
  fold changes, resulting in moderated estimates 
  from genes with low counts and highly variable counts,
  as can be seen by the narrowing of spread of points on the left side of the plot.
}

\begin{Ex}
Are the fold changes seen in this data set strong or weak compared to other gene expression data
you may have seen? Can you find the names of the genes with the strongest differences?
\end{Ex}

This plot demonstrates that only genes with a large average normalized count contain
sufficient information to yield a significant call.

Also note \deseqtwo's shrinkage estimation of log fold changes (LFCs): When count values
are too low to allow an accurate estimate of the LFC, the value is ``shrunken'' towards
zero to avoid that these values, which otherwise would frequently be 
unrealistically large, dominate the top-ranked log fold changes.

Whether a gene is called significant depends not only on its LFC but also on its
within-group variability, which \deseqtwo{} quantifies as the \emph{dispersion}. For 
strongly expressed genes, the dispersion can be understood as a squared coefficient of
variation: a dispersion value of 0.01 means that the gene's expression tends to differ by
typically $\sqrt{0.01}=10\%$ between samples of the same treatment group. For weak genes,
the Poisson noise is an additional source of noise, which is added to the dispersion.

The function \Rfunction{plotDispEsts} visualizes \deseqtwo's dispersion estimates:

<<beginner_dispPlot>>=
plotDispEsts( dds, ylim = c(1e-6, 1e1) )
@

\incfig{figure/beginner_dispPlot}{.5\textwidth}{Plot of dispersion estimates}{See text for details}

The black points are the dispersion estimates for each gene as obtained by considering
the information from each gene separately. Unless one has many samples, these
values fluctuate strongly around their true values. Therefore, we fit the red
trend line, which shows the dispersions' dependence on the mean, and then shrink each
gene's estimate towards the red line to obtain the final estimates (blue points)
that are then used in the hypothesis test. The blue circles above the main ``cloud'' of points
are genes which have high gene-wise dispersion estimates which are labelled as 
dispersion outliers. These estimates are therefore not shrunk toward the fitted trend line.

\begin{Ex}
What can you learn from the dispersion plot about the typical within-group variability of
gene-expression in the parathyphoid data set?
\end{Ex}

\incfig{figure/beginner_pvalHist}{.5\textwidth}{Histogram}{of the $p$ values returned by the
  test for differential expression}

Another useful diagnostic plot is the histogram of the $p$ values (Fig.~\ref{figure/beginner_pvalHist}).

<<beginner_pvalHist, dev="pdf">>=
hist( res$pvalue, breaks=20, col="grey" )
@

\subsection{Independent filtering}

The MA plot (Figure~\ref{figure/beginner_MA}) highlights an important property of RNA-Seq data.
For weakly expressed genes, we have no chance of seeing differential
expression, because the low read counts suffer from so high Poisson
noise that any biological effect is drowned in the uncertainties from the
read counting. 
We can also show this by examining the ratio of small $p$ values
(say, less than, 0.01) for genes binned by mean normalized count:

<<beginner_ratioSmallP, dev="pdf", fig.width=8, fig.height=4>>=
# create bins using the quantile function
qs <- c( 0, quantile( res$baseMean[res$baseMean > 0], 0:7/7 ) )
# "cut" the genes into the bins
bins <- cut( res$baseMean, qs )
# rename the levels of the bins using the middle point
levels(bins) <- paste0("~",round(.5*qs[-1] + .5*qs[-length(qs)]))
# calculate the ratio of $p$ values less than .01 for each bin
ratios <- tapply( res$pvalue, bins, function(p) mean( p < .01, na.rm=TRUE ) )
# plot these ratios
barplot(ratios, xlab="mean normalized count", ylab="ratio of small $p$ values")
@

\incfig{figure/beginner_ratioSmallP}{.9\textwidth}{Ratio of small p values}{for groups of genes 
binned by mean normalized count}

At first sight, there may seem to be little benefit in filtering out these genes. After
all, the test found them to be non-significant anyway. However, these genes have an
influence on the multiple testing adjustment, whose performance improves if such genes are
removed. By removing the weakly-expressed genes from the input to the FDR procedure, we
can find more genes to be significant among those which we keep, and so
improved the power of our test. This approach is known as \emph{independent filtering}. 

The \deseqtwo{} software automatically performs independent filtering which
maximizes the number of genes which will have adjusted $p$ value less 
than a critical value (by default, \Robject{alpha} is set to 0.1). 
This automatic independent filtering
is performed by, and can be controlled by, the \Rfunction{results} function.
We can observe how the number of rejections changes for various
cutoffs based on mean normalized count. The following optimal threshold
and table of possible values is stored as an \emph{attribute} of the 
results object.

<<beginner_filtByMean, dev="pdf">>=
attr(res,"filterThreshold")
plot(attr(res,"filterNumRej"),type="b",
     xlab="quantiles of 'baseMean'",
     ylab="number of rejections")
@

\incfig{figure/beginner_filtByMean}{.5\textwidth}{Independent filtering.}{\deseqtwo{}
automatically determines a threshold, filtering on mean normalized count,
which maximizes the number of genes which will have an adjusted
$p$ value less than a critical value.}

The term \emph{independent} highlights an important caveat. Such filtering is permissible
only if the filter criterion is independent of the actual test
statistic~\cite{Bourgon:2010:PNAS}. Otherwise, the filtering would invalidate the test and
consequently the assumptions of the BH procedure.  This is why we filtered on the average
over \emph{all} samples: this filter is blind to the assignment of samples to the
treatment and control group and hence independent.
 

\subsection{Exporting results}

Finally, we note that you can easily save the results table in a CSV file,
which you can then load with a spreadsheet program such as Excel:

<<geneAnnotRes>>=
res[1:2,]
@ 

<<writeCSV, eval=FALSE>>=
write.csv( as.data.frame(res), file="results.csv" )
@

\subsection{Gene-set enrichment analysis}

Do the genes with a strong up- or down-regulation have something in
common? We perform next a gene-set enrichment analysis (GSEA) to
examine this question. 

As noted in the lecture, gene-set enrichment analysis with RNA-Seq data entails some subtleties. Briefly, 
a number of different approaches have been proposed that each imply slightly different 
null hypotheses that are being tested against, and their biologically interpretation differs.
This is a topic of ongoing research. 
We here present a relatively simplistic approach, to demonstrate the basic ideas, but note that a more 
careful treatment will be needed for more definitive results. 

We use the gene sets in the Reactome database

<<loadReactome>>=
library( "reactome.db" )
@

This database works with Entrez IDs, so we will need the \Robject{entrezgene}
column that we added earlier to the \Robject{res} object.

First, we subset the results table, \Robject{res}, to only those genes for
which the Reactome database has data (i.e, whose Entrez ID we
find in the respective key column of \Robject{reactome.db}) and
for which the DESeq2 test gave an adjusted $p$ value that was not \Robject{NA}.

<<subsetTores2>>=
res2 <- res[ res$entrezgene %in% keys( reactome.db, "ENTREZID" ) & 
   !is.na( res$padj ) , ]
head(res2)
@

Using \Rfunction{select}, a function from \Rpackage{AnnotationDbi} for querying
database objects, we get a table with the mapping from Entrez IDs to
Reactome Path IDs

<<queryReactome>>=
reactomeTable <- AnnotationDbi::select( reactome.db, 
   keys=as.character(res2$entrezgene), keytype="ENTREZID", 
   columns=c("ENTREZID","REACTOMEID") )
head(reactomeTable)
@
%
The next code chunk transforms this table into an \emph{incidence matrix}. This is
a Boolean matrix with one row for each Reactome Path and one column for each 
gene in \Robject{res2}, which tells us which genes are members of which
Reactome Paths. (If you want to understand how this chunk exactly works, 
read up about the \Rfunction{tapply} function.)

<<CreateIncidenceMatrix>>=
incm <- do.call( rbind, with(reactomeTable, tapply( 
   ENTREZID, factor(REACTOMEID), function(x) res2$entrezgene %in% x ) ))
colnames(incm) <- res2$entrez

str(incm)
@

We remove all rows corresponding to Reactome Paths with less than 20 or more than 80
assigned genes.

<<PruneIncm>>=
within <- function(x, lower, upper) (x>=lower & x<=upper)
incm <- incm[ within(rowSums(incm), lower=20, upper=80), ]
@

To test whether the genes in a Reactome Path behave in a special way in our experiment,
we calculate a number of statistics, including a $t$-statistic to see whether the 
average of the genes' $\log_2$ fold change values in the gene set
is different from zero.  To facilitate the computations, we define a
little helper function:

<<testFun>>=
testCategory <- function( reactomeID ) {
  isMember <- incm[ reactomeID, ]
  data.frame( 
     reactomeID  = reactomeID,
     numGenes    = sum( isMember ),
     avgLFC      = mean( res2$log2FoldChange[isMember] ),
     sdLFC       = sd( res2$log2FoldChange[isMember] ),
     zValue      = mean( res2$log2FoldChange[isMember] ) / 
                     sd( res2$log2FoldChange[isMember] ),
     strength    = sum( res2$log2FoldChange[isMember] ) / sqrt(sum(isMember)),
     pvalue      = t.test( res2$log2FoldChange[ isMember ] )$p.value,
     reactomeName = reactomePATHID2NAME[[reactomeID]],
     stringsAsFactors = FALSE ) }
@

The function can be called with a Reactome Path ID:

<<testTestFun>>=
testCategory("109606")
@
%
As you can see the function not only performs the $t$ test and returns the p value
but also lists other useful information such as the number of genes in the 
category, the average log fold change, a ``strength'' measure (see below) and
the name with which Reactome describes the Path.

We call the function for all Paths in our incidence matrix and collect
the results in a data frame:
%
<<runGSEA,cache=TRUE>>=
reactomeResult <- do.call( rbind, lapply( rownames(incm), testCategory ) )
@
%
As we performed many tests, we should again use a multiple testing adjustment.
%
<<padjGSEA>>=
reactomeResult$padjust <- p.adjust( reactomeResult$pvalue, "BH" )
@

This is a list of Reactome Paths which are significantly differentially expressed
in our comparison of DPN treatment with control, sorted according to sign and strength
of the signal:

{\small
<<GSEAresult>>=
reactomeResultSignif <- reactomeResult[ reactomeResult$padjust < 0.05, ]
reactomeResultSignif[ order(reactomeResultSignif$strength), ]
@
}

However, as discussed in the lecture, it is highly questionable that a t test is appropriate
here. After all, genes in a set are typically correlated, and this violates the assumption
of independence that is at the core of a t test. Hence, should we really look at p values
from t tests? A p value obtained by sample permutation would solve this issue as sample
permutation preserves and so accounts for gene-gene correlation. However, with only
four subjects, we do not have enough samples for this.

Hence, it may be more prudent to disregard these questionable p values altogether and instead
look at a more primitive statistic, such as simply the average LFC within a path, 
perhaps divided by the standard deviation. We have stored this above as \Robject{zValue}.

{\small
<<GSEAresult2>>=
head( reactomeResult[ order(reactomeResult$zValue), ] )

head( reactomeResult[ order(reactomeResult$zValue, decreasing=TRUE), ] )
@
}

If such an analysis is only considered exploratory, we may inspect various such tables and see
whether the ranking of Paths helps us make sense of the data. Nevertheless, there is certainly
room for improvement here.

\begin{Ex}
The first few hits in the ranking by $z$ value all have exactly the same values. Why?
\end{Ex}

\section{Working with rlog-transformed data}

\subsection{The rlog transform}

Many common statistical methods for exploratory analysis of multidimensional
data, especially methods for clustering and ordination (e.\,g., principal-component
analysis and the like), work best for (at least approximately)
homoskedastic data; this means that the variance of an observable quantity (i.e., here,
the expression strength of a gene) does not depend on the mean. In RNA-Seq data, 
however, variance grows with the mean. 
For example, if one performs PCA directly on a matrix of normalized read counts, the result
typically depends only on the few most strongly expressed genes because they show the
largest absolute differences between samples. A simple and often used strategy to
avoid this is to take the logarithm of the normalized count values plus a small pseudocount; 
however, now the genes with low counts tend to dominate the results because, due to the strong Poisson
noise inherent to small count values, they show the strongest relative differences 
between samples.

As a solution, \deseqtwo{} offers the \emph{regularized-logarithm transformation},
or \emph{rlog} for short. For genes with high counts, the rlog transformation
differs not much from an ordinary log2 transformation. For genes with lower counts, however,
the values are shrunken towards the genes' averages across all samples. Using
an empirical Bayesian prior in the form of a \emph{ridge penality}, this is done such
that the rlog-transformed data are approximately homoskedastic.

Note that the rlog transformation is provided for applications other 
than differential testing. For differential testing we recommend the 
\Rfunction{DESeq} function applied to raw counts, as described earlier
in this vignette, which also takes into account the dependence of the 
variance of counts on the mean value during the dispersion estimation step.

The function \Rfunction{rlog} returns a \Rclass{SummarizedExperiment} object
which contains the rlog-transformed values in its \Rclass{assay} slot:

<<rld, cache=TRUE>>=
rld <- rlog( dds )
head( assay(rld) )
@ 

To show the effect of the transformation, we plot the first sample against the 
second, first simply using the \Rfunction{log2} function (after adding 1,
to avoid taking the log of zero), and then using the rlog-transformed values.

\incfig{figure/beginner_rldPlot}{.9\textwidth}{Scatter plot of sample 2 vs sample 1.}{
  Left: using an ordinary $log_2$ transformation. Right: Using the rlog transformation.}

<<beginner_rldPlot,fig.width=10,fig.height=5>>=
par( mfrow = c( 1, 2 ) )
plot( log2( 1+counts(dds, normalized=TRUE)[, 1:2] ), col="#00000020", pch=20, cex=0.3 )
plot( assay(rld)[, 1:2], col="#00000020", pch=20, cex=0.3 )
@

Note that, in order to make it easier to see where several points are plotted on top of each other,
we set the plotting color to a semi-transparent black (encoded as \Rfunction{\#00000020})
and changed the points to solid disks (\Rfunction{pch=20}) with reduced size
(\Rfunction{cex=0.3})\footnote{The function \Rfunction{heatscatter} from the package
  \CRANpkg{LSD} offers a colorful alternative.}.

In Figure~\ref{figure/beginner_rldPlot}, we can see how genes with low counts seem to be excessively variable
on the ordinary logarithmic scale, while the rlog transform compresses differences for genes
for which the data cannot provide good information anyway.

\subsection{Sample distances}

A useful first step in an RNA-Seq analysis is often to assess overall 
similarity between samples: Which samples are similar to each other, 
which are different? Does this fit to the expectation from the 
experiment's design?

\incfig{figure/beginner_sampleDistHeatmap}{.5\textwidth}{Heatmap of Euclidean sample distances after rlog transformation.}

We use the R function \Rfunction{dist} to calculate the Euclidean distance
between samples. To avoid that the distance measure is dominated by a few highly variable genes,
and have a roughly equal contribution from all genes, we use it on the rlog-transformed data:

<<euclDist>>=
sampleDists <- dist( t( assay(rld) ) )
sampleDists
@

Note the use of the function \Rfunction{t} to transpose the data matrix. We need
this because \Rfunction{dist} calculates distances between data \emph{rows} and
our samples constitute the columns.

We visualize the distances in a heatmap, using the function \Rfunction{heatmap.2} from
the \CRANpkg{gplots} package.

<<beginner_sampleDistHeatmap, dev="pdf">>=
sampleDistMatrix <- as.matrix( sampleDists )
rownames(sampleDistMatrix) <- paste( rld$treatment, 
   rld$patient, sep="-" )
colnames(sampleDistMatrix) <- NULL   
library( "gplots" )
library( "RColorBrewer" )
colours = colorRampPalette( rev(brewer.pal(9, "Blues")) )(255)
heatmap.2( sampleDistMatrix, trace="none", col=colours)
@ 

Note that we have changed the row names of the distance matrix to contain
treatment type and patient number instead of sample ID, so that we have
all this information in view when looking at the heatmap 
(Fig.~\ref{figure/beginner_sampleDistHeatmap}).

\incfig{figure/beginner_samplePCA}{.6\textwidth}{Principal components analysis (PCA)}{ of samples after rlog transformation.}

Another way to visualize sample-to-sample distances is a principal-components
analysis (PCA). In this ordination method, the data points (i.e., here, the samples)
are projected onto the 2D plane such that they spread out optimally (Fig.\ \ref{figure/beginner_samplePCA}).

<<beginner_samplePCA, dev="pdf", fig.width=4.3, fig.height=6>>=
plotPCA( rld, intgroup = c( "patient", "treatment" ) )
@

Here, we have used the function \Rfunction{plotPCA} which comes with \deseqtwo.
The two terms specified as \Rfunction{intgroup} are column names from our
sample data; they tell the function to use them to choose colours. 

From both visualizations, we see that the differences between patients is much larger 
than the difference between treatment and control samples of the same patient. This
shows why it was important to account for this paired design (``paired'', because each
treated sample is paired with one control sample from the \emph{same} patient). We did
so by using the design formula \texttt{\lowtilde{} patient + treatment} when setting up the
data object in the beginning. Had we used an un-paired analysis, by specifying
only \texttt{\lowtilde{} treatment}, we would not have found many hits, because then,
the patient-to-patient differences would have drowned out any treatment effects.

Here, we have performed this sample distance analysis towards the end of our analysis.
In practice, however, this is a step suitable to give a first overview on the data.
Hence, one will typically carry out this analysis as one of the first steps in an analysis.
To this end, you may also find the function \Rfunction{arrayQualityMetrics}, from the
package of the same name, useful. 

\subsection{Gene clustering}

\incfig{figure/beginner_geneHeatmap}{.5\textwidth}{Heatmap with gene clustering.}

In the heatmap of Fig.~\ref{figure/beginner_sampleDistHeatmap}, the dendrogram at the side shows
us a hierarchical clustering of the samples. Such a clustering can also be performed 
for the genes. 

Since the clustering is only relevant for genes that actually carry signal, one usually carries it out only for
a subset of most highly variable genes. Here, for demonstration, let us select the 35 genes
with the highest variance across samples:

<<topVarGenes>>=
library( "genefilter" )
topVarGenes <- head( order( rowVars( assay(rld) ), decreasing=TRUE ), 35 )
@

The heatmap becomes more interesting if we do not look at absolute expression strength
but rather at the amount by which each gene deviates in a specific sample from
the gene's average across all samples. Hence, we center and scale each genes' values across
samples, and plot a heatmap.

<<beginner_geneHeatmap, dev="pdf", fig.width=9, fig.height=9>>=
heatmap.2( assay(rld)[ topVarGenes, ], scale="row", 
     trace="none", dendrogram="column", 
     col = colorRampPalette( rev(brewer.pal(9, "RdBu")) )(255),
     ColSideColors = c( Control="gray", DPN="darkgreen", OHT="lightblue" )[
        colData(rld)$treatment ] )
@

\begin{Ex}
Which of the options to \Rfunction{heatmap.2} cause the centering and scaling of the genes? How does
the heatmap change if you do not scale it?
\end{Ex}

We can now see (Fig. \ref{figure/beginner_geneHeatmap}) blocks of genes which covary across patients.
Often, such a heatmap is insightful, even though here, seeing these variations
across patients is of limited value because we are rather interested in the effects
between the treatments from each patient. 

\begin{Ex}
Still, this heatmap shows genes that vary strongest between \emph{patients}. In our differential-expression
analysis, we found a list of genes that varied significantly between \emph{treatment} and control.
Display these genes (or maybe only those showing \emph{up}-regulation upon treatment) in a similar heatmap.
Can you confirm the test result from visual inspection of the heatmap?
\end{Ex}

\bibliography{RNA-Seq-Analysis-Lab}

\section{Session Info}

As last part of this document, we call the function \Rfunction{sessionInfo}, which
reports the version numbers of R and all the packages used in this session. It is
good practice to always keep such a record as it will help to trace down what has
happened in case that an R script ceases to work because a package has been
changed in a newer version. The session information should also always 
be included in any emails to the Bioconductor mailing list.

<<sessInfo, results="asis", echo=FALSE>>=
toLatex(sessionInfo())
@

\end{document}