This is the authors' abstract. We don't add key points for this paper.
This article describes 3 methods for obtaining comparison data for a program evaluation: 1) random assignment 2) matched-group comparison and 3) comparison with existing data. In particular the authors demonstrate how the 3rd method can be designed and implemented as a relatively inexpensive source of comparison data. The scientifically ideal method for evaluating a programs effectiveness is to use a true experimental design with random assignment to treatment and control groups. Because this method is often impractical evaluators may consider matched-group comparisons. In that case sufficient resources are necessary to preclude or control for preexisting differences between treatment and control groups and to minimize attrition from both. If sufficient resources are unavailable for matched-group comparisons it is often better to collect good data from the treatment group and to use already existing data from well-conducted studies for comparison. When preparing tables of comparison statistics the evaluator should follow 4 steps: 1) select 1 or more national or regional data sets 2) identify important demographic covariates 3) determine dependent variables of interest and 4) calculate cell statistics for the table. 4 steps for comparing data from program participants with those from a demographically stratified comparison group are 1) determine the demographic characteristics of the study sample; 2) obtain study sample statistic; 3) calculate comparison statistic weighted according to the study samples demographics; and 4) statistically compare the result from the study sample with that of the comparison group. Some cautions about this method include 1) since existing comparison data are likely to be from a period prior to the collection of study data outcomes may have been influenced by historical changes in the population of interest; 2) when data is summarized in tabular form some information is lost so analysts may want to base their comparisons on original data; 3) if a variable that is unaccounted for in the stratification of comparison tables is associated with the outcome variable and the sample differs on this factor from the comparison population the 2 groups may differ on unmeasured yet very relevant variables; and 4) the sample should be compared with the comparison data on antecedent and outcome variables.
Answers are grounded in this paper and cite their sources.
No takes yet. Share an insight, caveat, or question.
Card et al. (1989) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: