Inconsistencies in standard approximations for selection coefficients at loci affecting a polygenic character

Inconsistencies exist in the standard expansions used to approximate selection coefficients for alleles at a locus underlying a quantitative character. Allelic (marginal) fitnesses obtained from expansions based on average excesses differ from allelic fitnesses obtained from expansions based on genotypic values. Similarly, (Formula presented.) the mean population fitness based on summing over either allelic or genotypic fitnesses usually differs mean population fitness obtained by averaging over the unrestricted phenotypic distribution. A consistent value of (Formula presented.) requires no variation in genotypic values. If, as suggested by Nagylaki (1984), expansions are corrected for the decrease in phenotypic variance resulting from conditioning on the presence of a particular allele or genotype, inconsistencies still exist. Unless ∫ W(z)[V_zp″(z) + zp′(z) + p(z)] dz = 0, where p(z) is the phenotypic probability density function, V_z the phenotypic variance, W(_z) the fitness of phenotypic value z, the primes denote differentiation with respect to z, allelic fitnesses based on average effects differ from allelic fitnesses based on genotypic values. This condition must also be satisfied in order for either expansion to give a consistent (Formula presented.), as first shown by Nagylaki. For arbitrary W(z), this is satisfied if and only if phenotypes are normally distributed.