Similarly to Pearson’s r, Kappa may be subject to confounding factors of the experimental design such as the total read count. Applying the Kappa procedure to multiple RNA-Seq samples, the authors concluded that replicates of the same RNA-Seq library from different “lanes” (compartments in an Illumina genome analyzer flow cell used to separate samples) were subject to a systematic bias, a finding that appeared to contradict previous observations by others. recently suggested a measure of concordance based on the Kappa statistic to compare RNA-Seq samples. Chambers et al.) but from additional shortcomings specific to count data.Īs an alternative, McIntyre et al. However, as a quasi standard in the RNA-Seq literature r may be problematic as it may suffer not only from the general pitfalls that have long been recognized (e.g. Pearson’s correlation coefficient r has been widely used to affirm that pairs of RNA-Seq datasets are faithful replicates and continues to be in use. Simple computation is a desirable but not essential attribute. An ideal measure for this task should be easy to compute and have three features: (1) Sensitivity: The measure is sensitive to actual differences (2) Calibration: There is a known baseline value that corresponds to success (3) Stability: The behavior is independent of the sequencing depth, total number of exons, and other experimental conditions not relevant to the question. Here we focus on a more targeted approach that is useful in both quality control and early analysis. Sophisticated statisitical tools for analysis of Next-Generation Sequencing data are beginning to appear, e.g. A common task in RNA-Seq statistical analysis is to determine whether two RNA-Seq datasets are faithful replicates and, if not, whether two datasets differ only slightly or very markedly. The statistical analysis of RNA-Seq variability has been the focus of several comprehensive studies and remains a topic of active investigation.
The development of effective statistical data analysis methods has been essential to the utility of RNA-Seq and has been a focus since the original reports on the technology. Massively parallel shotgun RNA-Sequencing (RNA-Seq) has become the technology of choice for transcriptome analysis because of its potential to yield extensive biological information with digital precision. SERE can therefore serve as a straightforward and reliable statistical procedure for the global assessment of pairs or large groups of RNA-Seq datasets by a single statistical parameter. For quantifying global sample differences SERE performs similarly to a measure based on the negative binomial distribution yet is simpler to compute. Cohen’s simple Kappa results are also ambiguous and are highly dependent on the choice of bins. On the contrary the interpretation of Pearson’s r is generally ambiguous and highly dependent on sequencing depth and the range of expression levels inherent to the sample (difference between lowest and highest bin count). Benchmarking shows that the interpretation of SERE is unambiguous regardless of the total read count or the range of expression differences among bins (exons or genes), a score of 1 indicating faithful replication (i.e., samples are affected only by Poisson variation of individual counts), a score of 0 indicating data duplication, and scores >1 corresponding to true global differences between RNA-Seq libraries. Here we present a single-parameter test procedure for count data, the Simple Error Ratio Estimate (SERE), that can determine whether two RNA-Seq libraries are faithful replicates or globally different. Pearson’s correlation coefficient r has been widely used in the RNA-Seq field even though its statistical characteristics may be poorly suited to the task. Assessing the reliability of experimental replicates (or global alterations corresponding to different experimental conditions) is a critical step in analyzing RNA-Seq data.