Proposal for a New Set of Classification Criteria for Adult-Onset Still Disease

Introduction Over the last 20 years, classification criteria have been developed for rheumatic diseases (1,7). Many of these diseases do not have unequivocal or pathognomonic features corresponding to a true gold standard. Criteria sets thus constitute a way to identify homogenous groups of patients and distinguish them from those with “confusable diseases” (7). Adultonset Still disease (AOSD) is a rheumatic condition for which reliable classification criteria are needed. The clinical presentation of AOSD is heterogeneous, and the spectrum of differential diagnoses is broad, including infectious, neoplastic, and autoimmune diseases (2,4,5,14–16,19,20,23,27). Several sets of classification criteria have been published already (3,4,8,10,20,28). All have been developed from retrospective data, and have classified criteria as minor and major. Only 1 study to date has tried to validate these classification criteria (13). In that multicenter study, including 65 recently diagnosed AOSD patients, Yamaguchi’s (28) criteria set demonstrated the highest sensitivity (92%), although Calabro’s (3) and Cush’s (4) sets each had a sensitivity of 80%. Since there was no control group, no validation of specificity is available. However, at this juncture, Yamaguchi’s classification criteria set is the most commonly used. Existing classification criteria sets for AOSD, including Yamaguchi’s, suffer from 2 important limitations. First, Yamaguchi’s set, as well as most others, contains exclusion criteria that must be ruled out before making a diagnosis of AOSD (3,8,10,20,28). This situation is unique among published classification criteria for rheumatic diseases, and represents a major problem for use in clinical research. The exclusion criteria correspond to systemic, infectious, or neoplastic diseases; however, only general recommendations are provided. Neither a precise list of diagnoses to be excluded nor suggested lab tests and imaging exams are specified. The certainty of having excluded other diagnoses is thus often doubtful, even after many complementary exams, often time consuming and costly, have been performed. This difficulty seems particularly acute for occult neoplasm or infections; moreover in these diseases, treatments are usually very different from AOSD treatment. There is hence a need for a more powerful and accurate criteria set, which could potentially remove the requirement for exclusion criteria. A second limitation of existing classification criteria sets for AOSD is that 2 novel diagnostic markers for AOSD have been more recently identified and are not yet incorporated in any criteria set. Serum ferritin is a protein with increased production during inflammatory response. Its level seems to be extremely high in AOSD, and several authors have proposed it as a diagnosis or activity marker for this condition (6,9,17,18,21,22). The dramatic decrease—less than 20% of total ferritin—of glycosylated ferritin, an isoform of serum ferritin, has also been proposed as an AOSD diagnostic marker (6,24). This glycosylated component may be potentially more useful in clinical practice since its level is relatively constant over time and appears independent of fluctuations in disease activity (6,25). Preliminary results indicate that these 2 tests may increase the specificity of a criteria set, thereby obviating the need for exclusion criteria (6). However, their diagnostic utility compared with that of other AOSD characteristics remains unclear. The present study aims to reassess the influence of the various clinical and laboratory parameters suggestive of AOSD, and to determine a more accurate set of classification criteria for AOSD. Materials and Methods Patients The cohort of patients is described elsewhere (6). In brief, we retrospectively reviewed the records of 207 adult patients who underwent ferritin and glycosylated ferritin testing at the Biochemistry A laboratory, Bichat Hospital, Paris, between 1993 and 1998. Medical information was extracted via a standardized questionnaire, comprising basic clinical or biologic items frequently observed in the course of AOSD or “confusable” diseases. Since the majority of patients had been admitted to hospital, records were comprehensive. Standard laboratory tests (hemogram, liver enzymes) were available for all patients. The majority had undergone tests to establish antinuclear antibodies (ANA) or rheumatoid factor (RF) at time of diagnosis. For the remainder, we performed these tests on frozen sera stored at −80 °C. Serum ferritin and glycosylated ferritin tests were available for all patients, except when serum ferritin was lower than 45 μMol/L, as this prohibits the measurement of glycosylated ferritin (6). Classification procedure Patients were initially classified into 1 of 2 groups: AOSD patients and controls. Inclusion in the AOSD group was based uniquely on clinical and laboratory information extracted from the patient’s medical record. For a patient to be assigned a diagnosis of AOSD, this diagnosis should be retained 1) at last follow-up by the medical team in charge of the patient and 2) at the time of the present study by the principal investigator (BF) on the basis of medical chart data. In cases of disagreement, the final classification decision was made by a senior team physician, either rheumatologist or internist. The control group included patients with definite non-AOSD diseases, whose diagnosis was established by a senior rheumatologist or internist on the basis of clinical, pathologic, or microbiologic data. Specifically, this group included patients with hepatic, systemic, infectious, or neoplastic diseases, as well as those with fever of unknown origin, provided that AOSD diagnosis had been completely ruled out. All these conditions represent diseases frequently confused with AOSD. Patients for whom several diagnoses, including AOSD, were still being considered were excluded due to inability to make a definite diagnosis. No formal AOSD classification criteria were applied in constructing AOSD and control groups. Levels of ferritin and glycosylated ferritin were not used as a basis for classification, since their diagnostic value was insufficiently established at the time of study. Moreover, since evaluation of these markers was part of the study objective, their inclusion in the classification process would have led to a problem of circular reasoning (1). Variables tested All variables were transformed into binary variables: clinical parameters were coded as present or absent, laboratory parameters as normal or abnormal according to a predefined threshold (given in parentheses below). The clinical variables tested in the statistical analysis were spiking fever (≥39 °C), arthralgia, maculopapular rash, transient erythema, pharyngitis, lymphadenopathy or splenomegaly, total leukocyte count (≥10,000/mm3), polymorphonuclear neutrophil (PMN) count (≥80%), elevated liver enzymes in serum (alanine aminotransferase, aspartate aminotransferase, and lactate dehydrogenase), negative ANA (≤1/80) or RF in serum, increased serum ferritin and glycosylated ferritin (≤20%). Serum ferritin was assessed using the 2 thresholds proposed in the literature, that is, greater than the upper normal value or greater than 5-fold the upper normal value. Interactions between serum ferritin and glycosylated ferritin were also evaluated. Statistical analysis A univariate analysis was performed for each variable in AOSD and control groups, using the chi-squared and Fisher exact tests. Classification criteria were selected using multiple logistic regression (SAS statistical software, SAS Institute, Cary, NC). All variables and interaction terms were included in the initial model. The final model was selected using the likelihood ratio test. The probability of AOSD can be expressed as:EQUATION where β0 is an intercept corresponding to the case where all the variables Xi are equal to 0, that is, where all criteria are absent (Xi = 1 when the variable is present), βi are the regression coefficients corresponding to each variable Xi, and P(AOSD | X1, . . . ,Xp>) is the probability for a patient having (X1, . . . ,Xp) variables to have AOSD. A patient was classified as having AOSD if he or she displayed P(AOSD | X1, . . . ,Xp) > 0.5 (1). The sensitivity and specificity of this classification rule was described using a receiver operating characteristic curve (ROC). Results Population characteristics Seventy-two patients with AOSD were identified. Of the 130 patients classified as controls, 24 were affected with infectious diseases, 21 with liver disorders, 11 with neoplasms, 67 with systemic diseases, and 7 with fever of unknown origin (Table 1). For 5 patients, diagnosis was uncertain; these patients were consequently excluded from the study.TABLE 1: Control group diagnosesThe main features of patients with AOSD are indicated in Table 2. Table 3 shows the frequency of the clinical and laboratory variables included in the initial regression model, for both AOSD and control groups. Sensitivity and specificity are also indicated in the table. Except for serum ferritin level greater than the upper normal value, significant differences in the frequency of these variables were noted in AOSD patients as compared to controls. Some variables were particularly frequent in AOSD patients (that is, >80%): spiking fever ≥39 °C, arthralgia, leukocyte count ≥10,000/mm3, negative RF and ANA. Others were less frequent, but more specific for AOSD (specificity >80%): transient erythema, pharyngitis, PMN ≥80%, serum ferritin greater than 5-fold the upper normal value, and the combination of increased serum ferritin and decreased glycosylated ferritin (≤20%). Finally, 2 variables, maculopapular erythema and glycosylated ferritin ≤20%, displayed a mixed pattern, with both sensitivity and specificity about 70%.TABLE 2: Main clinical and laboratory characteristics of patients with AOSDTABLE 3: Univariate analysis of the variables used in the initial model, for AOSD and control groupsRegression model The statistical analysis provided the estimation of each of the regression coefficients in the model. The final model includes 8 variables: spiking fever ≥39 °C, arthralgia, maculopapular rash, transient erythema, pharyngitis, leukocytes ≥10,000/mm3, PMN ≥80%, and glycosylated ferritin ≤20% (Table 4). The intercept β0 was estimated at −8.5427.TABLE 4: Multivariate analysis using the multiple logistic regression modelWe developed a new AOSD classification criteria set for use in clinical research by stratifying variables into major and minor criteria, based on the regression coefficient (Table 5). We established as major criteria those variables whose regression estimates indicated that they played the greatest explanatory role in the final model. These variables, whose regression estimates were between 2 and 3, were spiking fever ≥39 °C, arthralgia, transient erythema, pharyngitis, PMN count ≥80%, and glycosylated ferritin ≤20%. Maculopapular rash and leukocyte count ≥10,000/ mm3 were considered minor criteria; their estimates were about 1.5. AOSD diagnosis should be considered if a patient satisfies 4 major criteria, or 3 major criteria plus 2 minor criteria. As defined, the present set displayed a sensitivity of 80.6%, a specificity of 98.5%, and positive and negative predictive values of 96.7% and 90.1%, respectively. These values are higher than those for Yamaguchi’s set in the same population (Table 6). The accuracy of major and minor criteria, defined as the number of adequately classified subjects (true positive plus true negative) divided by the total number of subjects, was 92.1% for the present set, compared to 88.6% for Yamaguchi’s set (see Table 6). Only 2 patients were falsely classified as AOSD by the present set, and 14 AOSD diagnoses were missed. With Yamaguchi’s set, 8 controls were falsely classified as AOSD; these were mainly patients with other systemic diseases, but also 1 patient with a lymphoma. Moreover, 15 AOSD diagnoses were missed. Interestingly, the addition of only glycosylated ferritin to Yamaguchi’s set reduced the number of false positives to 1 patient; however, this combination missed 20 AOSD diagnoses (false negatives), which lessened its clinical utility.TABLE 5: Proposed new set of AOSD criteriaTABLE 6: Evaluation of AOSD classification criteriaAn ROC curve was used to evaluate the capacity of the model to discriminate between AOSD and control diseases (Figure 1), and to identify the best combination of sensitivity and specificity for clinical purposes. The area under the curve was 0.973, which indicates a high discriminative power for the present set.Fig. 1: Receiver operating characteristic (ROC) curve of the new set of criteria. The curve indicates the discriminative power of the logistic regression model. The rate of true positive corresponds to sensitivity and the rate of false positive to (1-specificity). The performance of the present criteria set is indicated by (°).Discussion In this study we propose an accurate classification criteria set for AOSD that does not require exclusion criteria. The present set is more specific than Yamaguchi’s set (28), and provides a more useful guide for helping physicians exclude “confusable” diseases. Its application resulted in only 2 false positive cases, compared with 8 for Yamaguchi’s set. These 2 patients had other systemic diseases, with treatment differing little from AOSD therapy. Although our set contains no exclusion criteria, no controls with infectious or neoplastic disease were classified as positive. Moreover, our results strongly support the diagnostic value of glycosylated ferritin in AOSD, as superior to serum ferritin values (6,24). Determining appropriate study designs for AOSD is challenging. Although prospective study designs are desirable, the low incidence—between 1 and 3 cases per million inhabitants per year (12,26)—and prevalence—10 cases per million inhabitants (26)—of AOSD limit the feasibility. Since AOSD is not considered a critical public health problem, but belongs to the group of orphan diseases, it is difficult to generate resources for a prospective study, which, by necessity, would be long and complex. The published criteria sets to date (3,4,8,10,20,28) are hence based on a retrospective design. This has well-known limitations, which we have done our best to minimize. The first concerns the possibility of bias in data recording; to minimize this bias, we ensured that patients’ records were comprehensive, and all were carefully screened by investigators familiar with AOSD, using a standardized procedure. The second limitation of a retrospective design concerns potential difficulties in the selection of controls. The proper design of a classification criteria study is the comparison of AOSD patients and patients with diseases mimicking some AOSD features (“confusable diagnoses”) (7). Our present work and the study by Yamaguchi and colleagues (28) are the only studies to use this methodology; the others did not include a control group (3,4,8,10,20). The definition of the control group presents 2 difficulties. First, the size of the control sample is highly debatable. We do have an estimate of AOSD frequency in the general population, but no reliable data are available on what diseases should be included in the “confusable” group. In the study by Yamaguchi et al (28), which is the most important and detailed in the literature, roughly 3 controls were included for 1 AOSD case, compared with only 2 in our work. The second problem concerns the nature of what control diagnoses to include. To solve these methodologic issues, we based the patient selection on the simple basis of ferritin and glycosylated ferritin testing. Serum ferritin level is commonly determined in patients when a diagnosis of AOSD is suspected, and glycosylated ferritin has been available for the same indication in France at the Bichat Hospital biochemistry laboratory since 1992. This enabled the identification of a large number of both AOSD cases and controls in a unique registry (ferritin testing registry), regardless of the origin and the final diagnosis of the patients. Since the laboratory receives samples from all over France, the determination of cases and controls avoided a potential referral bias. Moreover, the identification of patients as cases or controls was independent of serum ferritin and glycosylated ferritin levels to avoid any selection bias (circularity problem). In the Japanese series, Yamaguchi et al (28) used an expert committee to identify controls with infectious or systemic diseases, or with fever of unknown origin. Although the distribution of infectious, neoplastic, and systemic diseases was different in our study, the spectrum of diagnoses was similar to that in other studies (8,28). The classification procedure for AOSD patients was not based directly on Yamaguchi’s classification criteria set, considered the most accurate up to the present work, since the study objective was to build a new set of classification criteria. Thus, we defined a 3-step classification procedure based on expert physicians’ judgment: 1) diagnosis retained by clinicians in charge of the patients, 2) diagnosis retained at the time of source data review, and 3) diagnosis of a senior rheumatologist or internist when uncertainty persisted after the first 2 steps. In the literature, Yamaguchi et al (28) were the only other team to also use experts’ judgment. Such procedures are imperfect, but they constitute valuable approaches in the absence of a gold standard. The statistical analysis was conducted according to the methodology used to determine previous criteria sets for the classification of other rheumatic diseases (1,7). The logistic regression equation enabled us to weight the different criteria and thus to compare them according to their ability to discriminate AOSD cases from controls. The cardinal symptoms have of course been selected in the model. For cutaneous lesions, we recorded in the source data the descriptors of skin lesions rather than whether or not the lesions were typical of AOSD, as defined in Yamaguchi’s set. Defining typical lesions was too complex to apply to patients’ files, as dermatologists’ notes in these files often indicated uncertainty as to whether the lesions were typical. This explains why 2 dermatologic items have been included in the final model, attributing a higher value to the “transient” feature. Other classic signs of AOSD have been ruled out. For example, lymphadenopathy and/or splenomegaly was not included; 1 explanation may be the presence of patients with neoplastic (especially lymphoma) or infectious diseases (notably acquired immunodeficiency syndrome AIDS) among controls. Moreover, the increase of liver enzymes was not retained in the model as many patients with liver diseases (especially hepatitis) were included in controls. Negative ANA and RF status was also not helpful as only a few rheumatoid arthritis and lupus patients have been included in controls; these patients are often rapidly diagnosed, especially the latter who usually display low leukocyte counts. This emphasized the fact that the absence of these 2 autoantibodies is probably too restrictive to eliminate an autoimmune condition, as presented by Yamaguchi et al (28). Although serum ferritin has been proposed as an additional diagnostic marker for AOSD, there has been uncertainty as to its actual diagnostic value and the threshold to use in clinical practice (9,17,18,21,22). The present study confirms the lack of specificity of serum ferritin (6,11) and the better diagnostic value of glycosylated ferritin (6). Glycosylated ferritin level becomes 1 of the major criteria; yet similar to the other major criteria, it is not pathognomonic of AOSD. The present work is of course preliminary. The ROC curve constituted a first evaluation of our criteria set, but this analysis was performed on the population used to develop the model. Therefore, a true validation of our model requires application to another series of AOSD patients and controls. This further step is the only way to ensure the performance of our criteria set, that is, its sensitivity, specificity, and accuracy, and its external validity. Summary Adult-onset Still disease (AOSD) is a rare rheumatic for which several sets of classification criteria have been can be considered a gold in clinical research. The present study to a better classification criteria set, including more markers than those used. We retrospectively reviewed the medical of 207 patients who had ferritin and glycosylated ferritin at the biochemistry laboratory of Bichat Hospital, Paris, present at the time of diagnosis were and diagnosis was assigned according to a standardized procedure. Seventy-two AOSD patients and 130 controls, from systemic, neoplastic, infectious, or diseases, were criteria for the new classification set were selected using a multiple logistic spiking fever ≥39 °C, arthralgia, maculopapular rash, transient erythema, pharyngitis, leukocyte count ≥10,000/mm3, polymorphonuclear count ≥80%, glycosylated ferritin ≤20%. The selected variables were classified into minor rash and leukocytes and major criteria to the diagnosis of AOSD, 4 major criteria or 3 major 2 minor were needed. This set provided a sensitivity of and a specificity of 98.5%, which is higher than those with previous The discriminative power of the model has been with a receiver operating characteristic (ROC) which displayed an area under the curve of 0.973, to This new classification criteria set seems more powerful than the previous published exclusion criteria. validation in a different population, this set represent a for use in clinical research. We for and data and and for their on the We also the clinicians who provided patient Hospital, Hospital, A Hospital, Institute, Hospital, Hospital, Hospital, Hospital, and