Joint Maximum Likelihood Estimation of Genetic and Mating Structure Using Open-Pollinated Progenies

SUMMARY Maximum likelihood estimation is described for parameters of genetic and mating structure under two plant mating system models, the mixed-mating model and the effective selfing model. Genetic structure is described by Wright's gene fixation index, which measures excess homozygosity at a single locus, and mating structure is described by selfing rates. The unit of observation is treated as the entire array of open-pollinated progeny genotypes at a single locus, descended from a maternal parent of unknown genotype. For the mixed-mating model, recursions are given for joint estimates of selffertilization rate s, fixation index of parents F, and allele frequencies of each sex. For the effective selfing model, recursions are given for joint estimates of effective selfing rate of inbred parents si, effective selfing rate of outbred parents so, fixation index F, and allele frequency among both sexes. Two alternate forms are given for each recursion. The first is the single-variable Newton-Raphson (Fisher scoring) method, where information divides the score. The second is the expectationmaximization (EM) method, which can be derived from the first method by increasing the information to that expected if the underlying variables of incomplete data could be directly observed. The EM form is more numerically stable, whereas the Newton-Raphson form gives faster convergence and allows negative estimates. Consideration of the progeny array as the unit of sampling allows asymptotic variances to be a function of progeny array size, for fixed total zygote sample size. The optimal number of progeny per array, which minimizes variance of estimates, is from 4 to 8 zygotes for F and the generational change of gene fixation, AF. It is near 4 for inbred and/or substructured populations and for more alleles/ more extreme allele frequencies, and near 8 for outbred populations. Optimal number is large for s, while in contrast, optimal number can be 4-5 for the effective selfing rate E (an average of si and so) because of variation of selfing rate among plants in substructured populations. Progeny sizes as low as 2 give information but a minimum size of 3 is recommended.