Genome-Wide Association Studies Explained Simply - Statistical Tests: P-Values and Multiple Testing

Welcome to part 3 on this video series on Genome-Wide Association Studies. In previous videos we have discussed the relationship between genetic variants and traits and how researchers use genotyping to identify the location of genetic variants associated with complex traits by taking advantage of linkage disequilibrium.

This video will discuss the role of P-values in genome-wide association studies as well as how researchers use a genome-wide p-value threshold to account for the multiple testing problem. 

First, we need to understand how researchers define a statistically significant association. GWAS uses P-values to measure the statistical significance of an association between a genetic variant and a trait. P-values measure the likelihood that an association at least as strong as the observed association would be found if there was in fact no real connection between the trait and the genetic variant. 

P-values are scored on a scale where the closer the score is to one the weaker the evidence is for a real association and the smaller the score the stronger the evidence for a real association. For example, a P-value of 0.01 means that finding a false positive - that is, finding an association where there is no real relationship between the trait and the genetic variant - where the association is of this strength would be expected once for every 100 trials in samples of the same size. 

Researchers use an arbitrarily agreed upon P-value threshold to decide how statistically significant the association needs to be to be confident that the association is real. Traditionally this had been agreed upon as P<=0.05 - that is a one in twenty chance of the association being a false positive. For GWAS however this threshold is far too lenient.

The reason that p<=0.05 had been acceptable historically in genetics but is not acceptable in modern genomics is that GWAS carry out way more than 20 tests. If every known independent common variant is tested for association there would be more than 3 million tests carried out per GWAS. A P-value threshold of p<=0.05 would therefore result in more than 150 thousand spurious associations being deemed statistically significant.

Researchers usually correct for the multiple testing problem by instead using a genome-wide significance threshold which is normally accepted as p < 5 × 10⁻⁸. This number is arrived at by dividing the traditionally accepted significance threshold of 0.05 by approximately the number of independent common SNPs across the human genome.

Correcting for multiple testing in this way solves the problem of too many false positives but there is always a trade off to be made between rejecting false positives and the power to detect real associations. If the size of the effect that a genetic variant has on the trait is small, and for complex traits like behaviour this is almost always the case, it is necessary for sample sizes - that is, the number of participants who have volunteered their genotype data - to be very large in order to be able to detect the effect at genome wide significance. For this reason, genomic researchers most often share databases of large amounts of genotyping data. The UK BioBank for example is a cohort of over 500,000 participants who have volunteered their genomic data to be used in GWAS and other types of studies.

If you want to find out how researchers use these data to find associations between genetic variants and traits subscribe to behavioural genomics and look out for the next video.