Validity of MAST-CLA for diagnosis of arthropod allergy using receiver operating characteristic (ROC) analysis

Article information

Korean J Parasitol. 2007;45(3):239-243
Publication date (electronic) : 2007 September 20
doi : https://doi.org/10.3347/kjp.2007.45.3.239
1Department of Pediatrics, College of Medicine, Soonchunhyang University, Cheonan 330-090, Korea.
2Department of Clinical Parasitology, College of Medicine, Soonchunhyang University, Cheonan 330-090, Korea.
3Department of Preventive Medicine, College of Medicine, Soonchunhyang University, Cheonan 330-090, Korea.
4Department of Laboratory Medicine, College of Medicine, Soonchunhyang University, Cheonan 330-090, Korea.
5Department of Biochemistry, College of Medicine, Soonchunhyang University, Cheonan 330-090, Korea.
6Department of Chemistry, College of Natural Science, Soonchunhyang University, Asan 336-745, Korea.
Corresponding author (namhs@sch.ac.kr)
Received 2007 April 02; Accepted 2007 August 10.

Abstract

Many allergists are currently focusing on the development of new diagnostic tools, and are attempting to improve both the sensitivity and specificity. A multiple allergen simultaneous test-chemiluminescent assay (MAST-CLA) is one of the most popular diagnostic tools used in the Republic of Korea. However, there remains controversy among allergists with regard to the cut-off point for a positive result. The present study was conducted in order to determine the validity of MAST-CLA as compared with that of the skin prick test, with particular emphasis on arthropod allergens, on the basis of percentage agreement rates and κ-values, and also to suggest the optimal positive cut-off points using receiver operating characteristic (ROC) curves. The study was conducted with 97 subjects (54 men, 43 women). Optimal individual cut-off points were calculated as follows; class II for Dermatophagoides farinae, class I for Dermatophagoides pteronyssinus, and trace for a cockroach mix. These findings suggest that attempting to apply optimal individual cut-off points will be a good way of improving diagnostic tests, particularly MAST-CLA.

The prevalence of allergic disease is rapidly increasing. Over the years, studies have consistently been indicating that sensitization to house dust mite allergens is closely associated with asthma in children and adults (Illi et al., 2001; Jaakkola et al., 2006), but no ideal treatment has yet been developed. The skin prick test, a typical in vivo test, is being extensively utilized for the diagnosis of allergic diseases, and is considered the gold standard, although it also has several disadvantages, including invasiveness or false positive reactions (Kiniker, 1989). The radioallergosorbent test (RAST), a typical in vitro test using patient sera, has been extensively utilized by allergists around the world, but it has an important handicap in that it requires the use of radioisotopes. Additionally, enzymatic immunoassay tools, such as the multiple allergen simultaneous test-chemiluminescent assay (MAST-CLA) (Brown et al., 1985; Miller et al., 1984) and the Pharmacia CAP System (Van Durme and Stevens, 1989), are commonly used, and have evidenced good agreement rates of results with the skin prick test (Agata et al., 1993; Finnery et al., 1989; Orgino et al., 1993; Scolozzi et al., 1989; Scolozzi et al., 1992; Tang and Wu, 1989; Tang et al., 1994). MAST-CLA has the advantage that it can test for 35 common allergens simultaneously, and thus it is suitable as a screening test. This is why it is currently one of the most popular diagnostic tools used in the Republic of Korea (Hwang et al., 2001; Lee et al., 1995; Yang et al., 1998). However, a controversy rages among allergists as to the cut-off points for a positive result (Hwang et al., 2001; Lee et al., 1995; Yang et al., 1998), and this has ramifications with regard both to specificity and sensitivity. Therefore, the lack of a consistent cut-off standard significantly ameliorates the reliability of MAST-CLA.

The present study was conducted to determine the validity of MAST-CLA as compared to the skin prick test for arthropod allergens, i.e., Dermatophagoides farinae, Dermatophagoides pteronyssinus, and a cockroach mix, using agreement rates, and to determine the optimal individual cut-off points for improved reliable test results, using receiver operating characteristic (ROC) curves.

A total of 97 subjects (54 men and 43 women, all are same class volunteer students of a medical school) in an age range of 24.7 ± 2.7 (mean ± SD) years old, none of whom had ever been assessed for allergic diseases, were evaluated. Three allergens derived from D. farinae, D. pteronyssinus and a cockroach mix (Bencard, Worthing, West Sussex, UK) were utilized as follows in routine laboratory tests conducted at the Soonchunhyang Cheonan Hospital (Table 1). Histamine and normal saline solution were employed as positive and negative controls. All allergens and controls were applied at the medial side of the forearm, and then pricked using a 26-gauge needle. After 15 min of pricking, the size of each of the indurations was measured using the means of the longest and shortest diameters. In cases in which the size of the indurations was equal to or larger than the positive control (histamine), the results were considered to be positive. The Korean inhalant panel of the MAST-CLA (MAST-Immunosystems, Mountain view, California, USA) was used for comparison. The panel actually contains the profile of 35 allergens, but 3 allergens corresponding to the skin prick test were analyzed (Table 2). The MAST-CLA was conducted in accordance with the method described by Brown et al. (1985). All MAST-CLA results are graded into 6 classes (0, trace, I, II, III, and IV).

Results of MAST-CLA by classes about arthropod allergens

Consistency rates between MAST-CLA and skin prick test against arthropod allergens

The statistical analysis was conducted with the SPSS software (SPSS v. 12.0, SPSS Inc., Chicago, Illinois, USA). Percentage agreement and κ-values were calculated in order to evaluate the consistency of the skin prick test and the MAST-CLA (class I regarded as a positive cut-off point). The sensitivity and specificity of each MAST-CLA point (class of trace, I, II, III, and IV) was calculated against the skin prick test. The ROC analysis was utilized to evaluate the adequacy of MAST-CLA and to determine the optimal cut-off points. A P-value below 0.05 was considered to be statistically significant.

The positive rates for 3 different allergens were 49.5% for D. farinae, 50.5% for D. pteronyssinus and 30.9% for the cockroach mix from the skin prick test. Also, from the MAST-CLA, the positive rates of D. farinae, D. pteronyssinus and the cockroach mix were 47.4%, 42.3%, and 15.4%, respectively (Table 1). The overall percentage agreement rates and κ-values are provided in Table 2. All of the values were statistically significant (P < 0.05).

For the purpose of the ROC analysis, the sensitivities and specificities for each of the allergens were calculated by changing the cut-off points of MAST-CLA (Table 3). The optimal individual cut-off points for 3 allergens were obtained by ROC analysis (P < 0.05): class II for D. farinae, class I for D. pteronyssinus, and "trace" for the cockroach mix (Fig. 1).

Validity of MAST-CLA by different cut-off points compared to skin prick test

Fig. 1

ROC curves of MAST-CLA by different cut-off points compare to skin prick test about arthropod allergens. *: optimal cut-off point.

The skin prick test is generally considered to be a gold standard for use in the diagnosis of allergic diseases. MAST-CLA has been popularly applied as a diagnostic tool for simplicity, convenience, and safety. However, there have still been some arguments regarding the specificity and sensitivity of MAST-CLA due to an insufficient quantity of scientific studies (Yang et al., 1998; Lim et al., 2001; Nepper-Christensen et al., 2003). In fact, this problem has resulted in a controversy regarding the definition of a positive cut-off point, and many allergists have utilized a single unique cut-off point (Park et al., 1991; Agata et al., 1993; Lee et al., 1995; Yang et al., 1998; Hwang et al., 2001; Lim et al., 2001; Nepper-Christensen et al., 2003). In the Republic of Korea, a unique cut-off point for MAST-CLA, which was first proposed by Park et al. (1991) as class II, has been adopted by many Korean allergists. However, this has also caused a problem of low positive rates as compared with the results from other countries (Lee et al. 1995; Yang et al., 1998). Recently, Lim et al. (2001) reported that if a cut-off point of class I is adopted, the MAST-CLA positive rates will be increased significantly. The other problem is that there is only personal preference that currently guides allergists in their selection of a unique cut-off point for every allergen in the MAST-CLA, due to insufficient data regarding the proper points. In the present study, a ROC analysis has allowed us to suggest a meaningful and individual MAST-CLA cut-off point of 3 arthropod allergens, based on a scientific analysis.

ROC analysis can be utilized to determine the relationship between the sensitivity and specificity of a clinical test for allergy in accordance with different cut-off points. Unfortunately, changing the cut-off point to try and increase either sensitivity or specificity will usually result in a reduction in the other measures; thus, we need to determine the optimal cut-off point (Metz, 1978; Zweig and Campbell, 1993). The ROC curve is a plot of the sensitivity (proportion of individuals predicted to have a future adverse event among those who actually have an event) versus the 1-specificity (proportion of individuals predicted to have an event among those who remain event-free) for the range of cut-off points that can be utilized to predict whether or not a patient will experience an event. The area under the ROC curve is frequently utilized as an index of how well a scoring system is able to classify patients into 1 of the 2 medical alternatives (an area of 1 is perfect, 0.5 is non-informative) such as, death or survival, disease or non-disease (Streiner and Norman, 1995).

The ROC curve analysis allowed us to determine the optimal MAST-CLA cut-off points as compared to the skin prick test. The percentage agreements of the MAST-CLA in comparison with the skin prick test were significantly increased by these newly obtained individual optimal cut-off points. This finding can be considered to be, in part, a solution to the specificity and sensitivity problems of this test. We were able to acquire limited data only for 3 arthropod allergens in MAST-CLA. However, the panel contains 32 other allergens. Therefore, we suggest that further evaluations be conducted in order to determine the optimal individual cut-off points for each allergen in the MAST-CLA panel, via ROC analysis.

References

1. Agata H, Yomo A, Hanashiro Y, Muraki T, Kondo N, Orii T. Comparison of the MAST chemiluminescent assay system with RAST and skin tests in allergic children. Ann Allergy 1993. 70153–157. 7679259.
2. Brown CR, Higgins KW, Frazer K, Schoelz LK, Dyminski JW, Marinkovich VA, Miller SP, Burd JF. Simultaneous determination of total IgE and allergen-specific IgE in serum by the MAST chemiluminescent assay system. Clin Chem 1985. 311500–1505. 4028399.
3. Finnery JP, Summerell S, Holgate ST. Relationship between skin-prick tests, the multiple allergosorbent test and symptoms of allergic disease. Clin Exp Allergy 1989. 1951–56. 2649214.
4. Hwang KY, Park JS, Ahn HC, Nam HS. Prevalence of arthropod antibodies in Korean patients with allergic rhinitis. Korean J Parasitol 2001. 39197–199. 11441508.
5. Illi S, von Mutius E, Lau S, Nickel R, Niggeman B, Sommerfeld C, Wahn U. The pattern of atopic sensitization is associated with the development of asthma in childhood. J Allergy Clin Immunol 2001. 108709–714. 11692093.
6. Jaakkola MS, Ieromnimon A, Jaakkola JJ. Are atopy and specific IgE to mites and molds important for adult asthma? J Allergy Clin Immunol 2006. 117642–648. 16522465.
7. Kiniker WT. Is the choice of allergy skin testing versus in vitro determination of specific IgE no longer a scientific issue? Ann Allergy 1989. 62373–374. 2719346.
8. Lee SR, Lee HR, Keum DG. Evaluation of the MAST-CLA allergy System (Korea IgE panel) for diagnosis of atopic allergy. Korean J Clin Pathol 1995. 15469–477.
9. Lim HS, Yoon JK, Lee HH. Allergen patterns using MAST CLA test in Korean pediatric patients. Korean J Clin Pathol 2001. 21292–297.
10. Metz CE. Basic principles of ROC analysis. Seminar Nucl Med 1978. 8283–298.
11. Miller SP, Marinkovich VA, Riege DH, Sell WJ, Baker DL, Eldredge NT, Dyminski JW, Hale RG, Peisach JM, Burd JF. Application of the MAST Immunodiagnostic System to the determination of allergen-specific IgE. Clin Chem 1984. 301467–1472. 6380812.
12. Nepper-Christensen S, Backer V, DuBuske LM, Nolte H. In vitro diagnostic evaluation of patients with inhalant allergies: Summary of probability outcomes comparing results of CLA- and CAP-specific Immunoglobulin E test systems. Allergy and Asthma Proc 2003. 24253–258. 12974191.
13. Orgino S, Bessho K, Harada T, Irifune M. Evaluation of allergen-specific IgE antibodies by MAST for the diagnosis of nasal allergy. Rhinology 1991. 3127–31.
14. Park HS, Kim JW, Chung DH, Kim YJ. Relationships between skin prick test, radioallergosorbent test, and chemiluminescent assay in allergic patients. Korean J Allergy 1991. 11347–354.
15. Scolozzi R, Boccafogli A, Vincentini L, Baraldi A, Bagni B. Correlation of MAST chemiluminescent assay (CLA) with RAST and skin prick tests for diagnosis of inhalant allergic disease. Ann Allergy 1989. 62193.
16. Scolozzi R, Vincentini L, Boccafogli A, Camerani A, Pradella R, Cavallini A, Bagni B. Comparative evaluation of RAST and MAST-CLA for six allergens for the diagnosis of inhalant allergic disease in 232 patients. Clin Exp Allergy 1992. 22227–231. 1571816.
17. Streiner DL, Norman GR. Health Measurement Scales 1995. 2nd edth ed. Oxford, UK: Oxford University Press;
18. Tang RB, Wu KG. Total serum IgE, allergy skin testing, and the radioallergosorbent test for the diagnosis of allergy in asthmatic children. Ann Allergy 1989. 62432–435. 2719352.
19. Tang RB, Wu KG, Hwang BT. Comparison between skin testing and in vitro testing for diagnosis of allergen in asthmatic children. Chin Med J (Taipei) 1994. 54246–250.
20. Van Durme P, Stevens E. The "Pharmacia CAP System": a more sensitive tool for detection of specific IgE in the serum: a clinical evaluation. Allergologie 1989. 1241.
21. Yang SE, Oh HB, Hong SJ, Moon DH, Chi HS. Analysis of MAST chemiluminescent assay (MAST CLA) results performed in asan medical center - Suggestion for the improvement of MAST CLA performance-. Korean J Clin Pathol 1998. 18660–666.
22. Zweig MH, Campbell G. Receiver operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clin Chem 1993. 39561–577. 8472349.

Notes

This study was supported by a grant from Soonchunhyang University Research Fund (2005).

Article information Continued

Fig. 1

ROC curves of MAST-CLA by different cut-off points compare to skin prick test about arthropod allergens. *: optimal cut-off point.

Table 1.

Results of MAST-CLA by classes about arthropod allergens

Allergen MAST-CLA (%)
0 trace Class I Class II Class III Class IV
Dfa) 39 (40.2) 12 (12.4) 4 (4.1) 4 (4.1) 3 (3.1) 35 (36.1)
Dpa) 42 (43.3) 14 (14.4) 4 (4.1) 4 (4.1) 2 (2.1) 31 (32.0)
Cockroach mix 67 (69.1) 15 (15.5) 10 (10.3) 4 (4.1) - 1 (1.0)
a)

Df: Dermatophagoides farinae, Dp: Dermatophagoides pteronyssinus.

Table 2.

Consistency rates between MAST-CLA and skin prick test against arthropod allergens

Allergens Skin prick test MAST-CLA
Overall percentage agreement Consistency (κ-value)
- +
Dpa) - 45 4 0.897 0.794b)
+ 6 42
Dpa) - 46 2 0.876 0.753b)
+ 10 39
Cockroach mix - 61 6 0.722 0.244b)
+ 21 9
a)

Df: D. farinae, Dp: D. pteronyssinus.

b)

P < 0.05.

Table 3.

Validity of MAST-CLA by different cut-off points compared to skin prick test

Allergen Trace
Class I
Class II
Class III
Class IV
SENb) SPEb) SEN SPE SEN SPE SEN SPE SEN SPE
Dfa) 0.90 0.69 0.88 0.92 0.85 0.98 0.77 0.98 0.71 0.98
Dpa) 0.86 0.73 0.80 0.96 0.73 0.98 0.65 0.98 0.61 0.98
Cockroach mix 0.50 0.78 0.30 0.91 0.10 0.97 0.03 1.00 0.03 1.00
a)

Df: D. farinae, Dp: D. pteronyssinus.

b)

SEN: sensitivity, SPE: specificity.