Operating Characteristics of Brief Screens for Dementia in a Multicultural Population

Operating characteristics of seven screens for dementia were compared across various groups for 795 persons who had received a criterion diagnostic evaluation. Area under the curve (AUC), based on receiver operating characteristics, was compared between and within scales as an indication of their efficiency. Differences in AUC were only 5% across all the scales for the entire sample, but increased to 11% across sociocultural groups and scales and to 20% across education groups and scales. Two scales (the Mini-Mental State Exam and the Short Portable Mental Status Questionnaire) misclassified most nondementias for the entire sample, and all scales misclassified most nondementias among persons with less than 5 years of education. Findings could support a recommendation that certain shorter scales be used because they perform as well as longer ones, are more consistent across cultural and educational groups, and can be more easily modified to improve performance in culturally diverse populations. Operating characteristics of seven screens for dementia were compared across various groups for 795 persons who had received a criterion diagnostic evaluation. Area under the curve (AUC), based on receiver operating characteristics, was compared between and within scales as an indication of their efficiency. Differences in AUC were only 5% across all the scales for the entire sample, but increased to 11% across sociocultural groups and scales and to 20% across education groups and scales. Two scales (the Mini-Mental State Exam and the Short Portable Mental Status Questionnaire) misclassified most nondementias for the entire sample, and all scales misclassified most nondementias among persons with less than 5 years of education. Findings could support a recommendation that certain shorter scales be used because they perform as well as longer ones, are more consistent across cultural and educational groups, and can be more easily modified to improve performance in culturally diverse populations. There are a number of widely used and well-documented screening techniques for Alzheimer's disease and related dementias (ADRD).1Kane R Kane R Assessing the Elderly: A Practical Guide to Measurement. Lexington Books, Lexington, MA1981Google Scholar, 2Baker FM Screening tests for cognitive impairment.Hosp Community Psychiatry. 1989; 40: 339-340PubMed Google Scholar These screens are currently used in research and clinical settings. Also, the National Advisory Panel on Alzheimer's Disease has recently recommended that the conventional measures of functional deficit (activities of daily living; ADL) used to assess patients to determine their eligibility for admission to and reimbursement for long-term care be modified to detect the unique characteristics of ADRD patients.3Advisory Panel on Alzheimer's Disease, 4th Report of the Advisory Panel on Alzheimer's Disease, 1992. DHHS Publication No. 93-3520. Supt. of Documents, U.S. Government Printing Office, Washington, DC1993Google Scholar Thus investigators, clinicians, and regulators confront the task of choosing among screening techniques for a given application. Most brief screening instruments for ADRD test selected aspects of cognitive status. These brief screens, usually taking not more than 5–10 minutes to administer, are typically systematic, structured questions and tasks and do not require information from a third party. They are intended to be sensitive and specific to the presence of early as well as established dementia, to be reliable and quantitative, to be able to be administered by health professionals or research assistants from a variety of disciplines, not to need more than a few hours of special training to administer, and to be readily scorable at the time of testing. Ideally, they are not substantially biased by such sociocultural influences as ethnicity and education. During recent years, brief screening instruments have been used in studies with a number of different ethnic groups, including Chinese, Finns, Koreans, Italians, Jews, Latinos, and African-Americans.4Escobar JI Burnam A Karno M et al.Use of the Mini-Mental State Examination (MMSE) in a community population of mixed ethnicity.J Nerv Ment Dis. 1986; 174: 607-614Crossref PubMed Scopus (350) Google Scholar, 5Folstein MF Folstein SE McHugh PR The Mini-Mental State: a practical method of grading the cognitive state of patients for the clinician.J Psychiatr Res. 1975; 12: 189-198Abstract Full Text PDF PubMed Scopus (69806) Google Scholar, 6Kua E The prevalence of dementia in elderly Chinese.Acta Psychiatr Scand. 1991; 83: 350-352Crossref PubMed Scopus (40) Google Scholar, 7Li G Shen YC Chen CH et al.A three-year follow-up study of age-related dementia in an urban area of Beijing.Acta Psychiatr Scand. 1991; 83: 99-104Crossref PubMed Scopus (150) Google Scholar, 8Salmon DP Riekkinen PJ Katzman R et al.Cross-cultural studies of dementia: a comparison of Mini-Mental State Examination performance in Finland and China.Arch Neurol. 1989; 46: 769-772Crossref PubMed Scopus (111) Google Scholar, 9Silverman JM Li G Schear S et al.A cross-cultural family history study of primary progressive dementia in relatives of nondemented elderly Chinese, Italians, Jews, and Puerto Ricans.Acta Psychiatr Scand. 1992; 85: 211-217Crossref PubMed Scopus (18) Google Scholar, 10Yu ES Liu WT Levy P et al.Cognitive impairment among elderly adults in Shanghai, China.J Gerontol. 1989; 44: S97-S106Crossref PubMed Scopus (126) Google Scholar Studies that include African-American or Latino as well as non-Latino whites have generally reported higher rates of cognitive impairment among the minority groups.4Escobar JI Burnam A Karno M et al.Use of the Mini-Mental State Examination (MMSE) in a community population of mixed ethnicity.J Nerv Ment Dis. 1986; 174: 607-614Crossref PubMed Scopus (350) Google Scholar, 5Folstein MF Folstein SE McHugh PR The Mini-Mental State: a practical method of grading the cognitive state of patients for the clinician.J Psychiatr Res. 1975; 12: 189-198Abstract Full Text PDF PubMed Scopus (69806) Google Scholar, 8Salmon DP Riekkinen PJ Katzman R et al.Cross-cultural studies of dementia: a comparison of Mini-Mental State Examination performance in Finland and China.Arch Neurol. 1989; 46: 769-772Crossref PubMed Scopus (111) Google Scholar, 11De la Monte SM Hutchins GM Moore GW Racial differences in the etiology of dementia and frequency of Alzheimer lesions in the brain.Journal of the National Medical Association. 1989; 81: 644-652PubMed Google Scholar, 12Murden RA McRae TD Kaner S et al.Mini-Mental State exam scores vary with education in Blacks and Whites.J Am Geriatr Soc. 1991; 39: 149-155Crossref PubMed Scopus (256) Google Scholar, 13Weissmann M Myers JK Tischler GL et al.Psychiatric disorders (DSM-III) and cognitive impairment among the elderly in a US urban community.Acta Psychiatr Scand. 1985; 71: 366-379Crossref PubMed Scopus (173) Google Scholar Most studies also reported higher rates among persons with less education.14Burvill PW The impact of criteria selection on prevalence rates. Fifth Congress of the International Federation of Psychiatric Epidemiology (Montreal, Canada; 1990).Psychiatric Journal of the University of Ottawa. 1990; 15: 194-199PubMed Google Scholar, 15Fratiglioni L Grut M Forsell Y et al.Prevalence of Alzheimer's disease and other dementias in an elderly urban population: relationship with age, sex, and education.Neurology. 1991; 41: 1886-1892Crossref PubMed Google Scholar, 16Uhlmann RF Larson EB Effect of education on the Mini-Mental State Examination as a screening test for dementia.J Am Geriatr Soc. 1991; 39: 876-880Crossref PubMed Scopus (226) Google Scholar, 17O'Connor DW Pollitt PA Hyde JB et al.Clinical issues relating to the diagnosis of mild dementia in a British community survey.Arch Neurol. 1991; 48: 530-534Crossref PubMed Scopus (11) Google Scholar Some studies have used multiple instruments with the same subjects in an effort to determine whether some work better than others with minority groups or persons with less education.18Davis PB Morris JC Grant E Brief screening tests vs. clinical staging in senile dementia of the Alzheimer type.J Am Geriatr Soc. 1990; 38: 129-135Crossref PubMed Scopus (79) Google Scholar, 19Fillenbaum G Heyman A Williams K et al.Sensitivity and specificity of standardized screens of cognitive impairment and dementia among elderly black and white community residents.J Clin Epidemiol. 1990; 43: 651-660Abstract Full Text PDF PubMed Scopus (226) Google Scholar The choice of a brief screen for dementia should be guided by criteria that are explicit, logical, and tied to a sound knowledge base. However, the scientific knowledge base about screens for dementia does not provide adequate information for making these decisions (whether they be by health care professionals, researchers, funding agencies, or various committees concerned with practice standards or regulation) within the context of other relevant information, such as functioning. These crucial decisions must often defer to issues of custom, critical mass of usage, collaborative compromise, preferences of funding bodies, or expert consensus. Recent studies have found that some of the items in conventional screens, including those used in this study, are racially or ethnically biased.20Teresi J Golden R Cross P et al.Item bias in cognitive screening measures: comparisons of elderly White, Afro-American, Hispanic, and high and low education groups.J Clin Epidemiol. 1994; Google Scholar However, these studies do not indicate the extent to which these items bias results in actual misclassification of persons as demented or not. Here we assess the relative accuracy of selected brief screens for ADRD in detecting dementia in a multicultural population. The data in this paper were gathered in the course of the North Manhattan Aging Project (NMAP), a registry for ADRD. This registry consists of a Reporting Component and a Survey Component: the data in this paper are from the Reporting Component and from the pilot study21Gurland BJ Wilder DE Cross P et al.Screening scales for dementia: toward reconciliation of conflicting cross-cultural findings.International Journal of Geriatric Psychiatry. 1992; 7: 105-113Crossref Scopus (95) Google Scholar conducted earlier to establish registry methods. Relative accuracy of screens for dementia is determined by comparing sensitivity—the proportion of all diagnosed dementias so identified by the screen, specificity—the proportion of all diagnosed nondementias identified, and receiver operating characteristic (ROC) curves22Metz CE Basic principles of ROC in Full Text PDF PubMed Scopus Google Scholar for different sociocultural and education groups within the same study ROC are with on the Y and specificity (the on the are based on and specificity at The area under the curve is the that a diagnosed dementia have a higher screening test than a diagnosed BJ The and of the area under a receiver operating characteristic (ROC) Scholar is a more indication of the accuracy of a than or specificity at The to and all of ADRD among persons years and in a a or Reporting and a of in this or Survey The is a of subjects for the studies as well as a on The area for study is in North that Latinos, non-Latino and non-Latino white subjects years and to by the of the Reporting Component of the the and for a criterion diagnosis are in a Wilder Cross P et rates of dementia by multiple prevalence in sociocultural Journal of Geriatric Psychiatry. Full Text Full Text PDF Scopus Google Scholar A was from widely used dementia screening the Mental Status M et for the of in the J Psychiatry. PubMed Scopus Google Scholar the Short Portable Mental Status E A for the of deficit in elderly Am Geriatr Soc. 1975; Scopus Google Scholar the and a paper this was the the diagnostic has been to with the of diagnostic the Golden R and to the of and dementia in the S K J Alzheimer's A of Scholar and the Mini-Mental State Examination R P et of a test of cognitive J Psychiatry. PubMed Scopus Google Scholar A the MF Folstein SE McHugh PR Mini-Mental State: a practical method for grading the cognitive state of patients for the clinician.J Psychiatr Res. 1975; 12: 189-198Abstract Full Text PDF PubMed Scopus Google Scholar based on of cognitive items in the and a of the items from the other scales in the from of data at the of this are also in the comparisons in this The was to between measures and a of the The minutes in A of the was by a in the of the The was by professionals with in were the and a few an than a was in and Puerto or were to subjects to the for This was based on criteria to information in an by the of and who were to the screen The BJ of scales for the and Gerontol. 39: PubMed Scopus Google Scholar, Y J et of dementia in a population: of a diagnosis of dementia and for the of Neurol. 1992; PubMed Scopus Google Scholar the diagnostic that a at an Alzheimer's disease research A criterion diagnosis of dementia the criteria for dementia, all the for the BJ Wilder DE Cross P et al.Screening scales for dementia: toward reconciliation of conflicting cross-cultural findings.International Journal of Geriatric Psychiatry. 1992; 7: 105-113Crossref Scopus (95) Google Scholar the cognitive screening was administered to subjects years of and including groups of Latinos, and non-Latino was to a but of dementia and the for the screening results for and relative rates of cognitive impairment for the cultural groups were These differences were by the However, to the high for the some scales had specificity than this of the the was selected for of all and a proportion of to the in the and of the Some subjects from the from the area and were in the from the However, all pilot study subjects have been from the for this paper so as to comparisons of the results from The subjects for this paper are 795 persons not also in the pilot study who were in the Reporting Component of the and given a diagnosis of or by the criterion characteristics of this of 795 persons are in and education for sociocultural Reporting subjects 795 diagnosed subjects from sociocultural in a Reporting subjects 795 diagnosed subjects from sociocultural The the differences among the ethnic groups the are the and they have the and the education. whites are the and they are and have more education than the other The African-American is between the other on all There is of items among the seven screening scales in the between of scales from to for the persons in the from to for the persons by the Reporting Component of the the prevalence in the years of the Reporting and from to for the 795 persons in the Reporting Component who were also and in all were for the and for the with the other scales than for the other scales with A of the was to be able to a high of for specificity to so low that the of a clinical of data has on comparisons across with at high Some studies to be certain that most persons by a screen have dementia, and their at However, this results in low and only of dementia with all the screens we have the of 795 persons from the Reporting Component of the who were not of the pilot sample, from to (MMSE) they were to provide the pilot sample, persons were by the clinical diagnostic with at specificity from to and for all scales in the Reporting Component of the than in the pilot the a of reported persons were to be at and had cognitive they not have dementia, the pilot who were more easily from persons diagnosed with the of the of at the from to the pilot with patients from to and and of for seven screening with at for criterion diagnosis of dementia in culturally mixed Reporting and of number of at which was was the same for study for all scales the which the high with in the Reporting Component was not used in the and the was was not used in the and the was was not used in the and the was was not used in the and the was Mental Status Mini-Mental State Short Portable Mental Status and number of at which was was the same for study for all scales the which the high with in the Reporting Component The was not used in the and the was in a Mental Status Mini-Mental State Short Portable Mental Status and with the in the of brief cognitive screening scales in a multicultural ROC curves22Metz CE Basic principles of ROC in Full Text PDF PubMed Scopus Google Scholar and the BJ The and of the area under a receiver operating characteristic (ROC) Scholar were for of the seven screening scales for the Reporting Component as a for ethnic non-Latino and and for educational less than 5 years, years, and or more years of education. performance of a is by a and a higher This is in which ROC for the for the educational A low number of on the screening results in high and low specificity and a high number of results in high specificity and low for all educational with specificity more with for the with years of education than for the with years, and more for the than for the with 5 years of education. A has been at the to the ROC for the groups at different specificity different ROC be but some differences and among the can be the as a AUC only from with the to with the the seven scales in the of this of the screen scales have been for ethnic or educational However, is that the AUC for the only increased to the recommended were Some subjects not provide adequate information for conventional screening The of these persons were in or other long-term and they were by the clinical were diagnosed with a persons data for the scales were in the study, and they were as at the of on This of 795 from the Reporting Component such were diagnosed with dementia, and AUC for was these were but of AUC for the scales was not AUC 11% ethnicity was from with the among non-Latino whites to for the among and AUC 20% with from with the test among the high to for the among those with years of Differences in AUC for scales across ethnic groups from for the to for the AUC differences for scales across education groups from for the to for the scales had their AUC with the non-Latino and all had their AUC with the Latino under the curve for seven cognitive screening scales and of differences in AUC across sociocultural and educational groups 795 diagnosed in AUC Reporting subjects ethnic non-Latino and and education 5 years of years of and years of Mental Status Mini-Mental State Short Portable Mental Status and in a Reporting subjects ethnic non-Latino and and education 5 years of years of and years of Mental Status Mini-Mental State Short Portable Mental Status and screening scores that were a different was to provide a more for specificity at the of of the ROC curve with the the was from a the and by actual This was for the as a and for the different ethnic and education for the criterion diagnosis at ROC all seven scales had their among the non-Latino and all had their among the Latino the specificity was for among the non-Latino whites from to among African-American subjects from to and the among Latino subjects was from to Differences in specificity across ethnic groups were for the and for the of seven screening scales for sociocultural and educational groups, with at from ROC with criterion diagnosis 795 diagnosed Reporting subjects receiver operating Mental Status Mini-Mental State Short Portable Mental and in a Reporting subjects ROC receiver operating Mental Status Mini-Mental State Short Portable Mental Status and for the criterion diagnosis at all seven scales had high specificity among subjects with years of scales specificity among subjects with years of and had their specificity among persons with 5 years of The scales and had higher specificity among high than the had among those with years of and the with the had better specificity than the with the education Differences in specificity across the educational groups, with at were for the and for the This comparison of widely used screens for cognitive impairment a screening was on by performance as a of a in with or with of information, was not within the of this the or of a was determined in of to a criterion diagnosis of dementia, to to of of dementia, and of the seven scales on the of the of to a criterion diagnosis of dementia were from of and specificity across the of by of AUC of ROC and the of specificity with at was determined across scales and sociocultural and educational differences between scales in AUC were found to differences within and between scales for sociocultural and educational groups and at different the ROC The in the pilot study to specificity across and within with at was by the to the screen and that be for and the of persons in the to the criteria for a diagnosis of ROC more comparisons for this to comparisons of with specificity high but low results from with groups that are and and with at earlier of the on the were substantially in with the from the pilot study but more and The of cognitive impairment between the studies and the in that criterion nondementias had less cognitive impairment in the pilot study than in the study, the of the of the on the scales. this study, AUC and specificity for all including to be culturally were for the most and non-Latino and for the and for the Latino with at specificity for all scales more across education than across sociocultural The the test to be as well as or better than some of the other scales. The and or in most of the of the scales had specificity so low that were more than among the with 5 years of education. Two of the most widely used screens, the and were the of the scales in most of the and had specificity so low that of diagnosed nondementias were for the entire The of the selection of subjects for to criterion diagnosis must be in these comparisons among scales. The established in was the for this The only for bias in of the be in that have been found by screen but that were because those not the that be However, of those who not the were also in the and of those were diagnosed of the other screens had to by the or by of the other bias to the also be in increased for other scales to is whether has been in the of screening for ADRD from within the cognitive the of the on is that brief cognitive screening techniques have not the of information in the the given by more recent of and of the also among the better than the and that the longer scales to vary more with ethnic and educational indicate that brief screens are as as longer for cognitive for clinical and A on the of more screens has been a in the of conventional techniques in and in certain sociocultural and educational for educational or such as those for the which not improve relative accuracy in detecting dementia compared with the other scales used in this study sample, The of screening must from the of information in the screening has been on tasks that are in the on the of the in such as and is but has been The on have been by such scales as the G M The between measures of dementia and of senile in the of J Psychiatry. PubMed Scopus Google Scholar the that was used in this the Wilder Cross P et rates of dementia by multiple prevalence in sociocultural Journal of Geriatric Psychiatry. Full Text Full Text PDF Scopus Google Scholar and the L et al.A clinical for the staging of J Psychiatry. PubMed Scopus Google Scholar However, these techniques have had or at making of cognitive and functional scales in in this has been of the relative of and performance as of has been less information on the of other such as and course of the for their to the in the The with screening techniques for ADRD with for practice standards on screening for ADRD to be in clinical care and with the need for but more in and health of this and of the research knowledge we a more and more screening for among culturally diverse and persons with or education. be to the of screening for dementia others on the of a that in some by the study study or of non-Latino white persons with years of these results indicate that does not which is all perform However, study or include persons from groups with low of all scales of high is the and were among the in all be to they not be used because they are so widely used and comparisons among studies and However, the shorter scales the of the as well as the longer ones, and less across cultural and educational groups, does to these shorter scales because of their and also because their shorter more easily the items that be to improve their performance in culturally diverse populations.