Sampling Error due to Choice of Split in Split-Half Reliability Coefficients

The formula and derivation are given for the variance of the re liability coefficient when all possible splits are sampled for a given group of examinees. When the number of test items is large, this variance is found to be larger than that of the Kuder-Richardson formula-20 reliability coef ficient by a multiplicative factor equal to the number of test items. The usual matched-halves reliability coefficient will likewise o r d i narily have a similarly large variance, since it likewise is based on an arbitrary split of the test items. An adaptation of the Jackson-Fergu son battery reliability coefficient, usedasan estimate of the matched-forms reliability of the test, avoids this relatively large vari IN OBTAINING a split-half reliability coeffi cient, the items of a test are first of all divided into two equally numerous sets (half-tests) and the correlation between scores on the half-tests is computed. It will be assumed in the present discussion of reliability that all such coefficients are by any one of the usual, math ematically equivalent formulas that do not as sume equal score variance for the two half-tests. A fuller discussion of these formulas is given by Cronbach[2]. One of these formulas ap pears as equation 1, below. Three kinds of split-half coefficients will be discussed in detail: 1. A random-halves reliability coefficient is ob tained when the choice of items for the half tests is made at random. It is an estimate of the random-forms reliability of the test. 2. A matched-halve s reliability coefficient is ob tained whenever the two half-tests are pur posely selected to be more nearly alike than random halves. It is an estimate of the matched-forms reliability of the test. A matched-halves coefficient tends to be higher than a random-halves coefficient, the amount of difference depending on the effectiveness of the matching. 3. An odd-even reliability coefficient is obtained when items are assigned to the two half-tests according to whether their serial number i n the test is odd or even. If the test items are arranged in effectively random order in the test, the odd-even coefficient is a random halves coefficient; if the items are arranged roughly in order of difficulty, or are grouped by type or by subject matter, it is a matched halves coefficient. It has been shown by Cronbach (2) that the Kuder-Richardson formula-20 reliability coeffi cient (r2 ) is the average of all split-half coeffi cients computed from all possible splits of a giv en test. Two conclusions are immediately obvi ous: 1. The random-halves coefficient and r20 are both estimates of the same parameter (the random-forms reliability of the test). 2. The variance due to of i tems-type-2 sampling [4]-?is larger for the random-halves coefficient than for r^ ; hence, the former coefficient is a relatively inefficient estimator. The degree of this in efficiency will be brought out in what follows. The variance of the random-halves coefficient for type-2 (which will be as sumed hereafter unless the contrary is specified) has not been worked out previously. The large sample formula is derived in the present article. It will be seen that when the number of test items (m) is large, the variance of the ran dom-halves coefficient is ordinarily of order 1/m2 whereas the variance of r2o is [4] of order 1/m3. Thus the relative efficiency of the former coefficient is of order 1/m?obvi ously very poor when is large. This low ef ficiency is not surprising when it is considered that the particular value obtained for the random halves reliability depends on an arbitrary choice among all the | m /[ (^m) ] 2 splits of the to ta 1 test into halves. Now the type-1 variance (represent ing fluctuations when examinees rather than items are sampled) will be of the same order of magnitude for both ?& and the random-halves coefficient. Since r20 is more efficient in type 2 sampling, it is clear that it is theoretic ally preferable to a random-halve s coefficient. In many cases the research worker will wish to estimate the matched-forms rather than the This content downloaded from 207.46.13.77 on Mon, 23 May 2016 06:07:38 UTC All use subject to http://about.jstor.org/terms 246 JOURNAL OF EXPERIMENTAL EDUCATION (Vol. 24 random-forms reliability of the test. The usual matched-halves coefficient is based on a less arbitrary split of the test items than is the ran dom-halves coefficient, but the split is never theless still arbitrary. Even if the total test con tains no more than one other item matching any single given item, the total number of pairs of matching half-tests thatcan be formed is 22~m , and each pair in general leads to a different and arbitrary estimate of the matched-forms relia bility. Some less arbitrary and more e?iicient meth od for estimating matched-forms reliability would be desirable. Such a method is briefly de scribed at the end of the present article. Derivation of the Standard Error of the Random-Halves Coefficient The formula for the random-halves coeffi cient is derived below. Some familiarity with the line of reasoning in [ 4] will be helpful, but not essential, to the reader who wishes to fol low the proof. The stepped-up split-half reliability coef ficient may be written [ cf. 2, Table I] :