Analytical Chemistry Chemical Cytometry Quantitates Superoxide

More documents

Recommendations

Info

Anal. Chem. 2010, 82, 7000–7007 Visualization and Recovery of the (Bio)chemical Interesting Variables in Data Analysis with Support Vector Machine Classification Patrick W. T. Krooshof, † Bülent Üstün, ‡ Geert J. Postma, † and Lutgarde M. C. Buydens* ,† Radboud University Nijmegen, Institute for Molecules and Materials, Analytical Chemistry, P.O. Box 9010, 6500 GL Nijmegen, The Netherlands, and Analytical Sciences Chemicals, Quality Unit API/Biotech, MSD Oss, P.O. Box 20, 5340 BH Oss, The Netherlands Support vector machines (SVMs) have become a popular technique in the chemometrics and bioinformatics field, and other fields, for the classification of complex data sets. Especially because SVMs are able to model nonlinear relationships, the usage of this technique has increased substantially. This modeling is obtained by mapping the data in a higher-dimensional feature space. The disadvantage of such a transformation is, however, that information about the contribution of the original variables in the classification is lost. In this paper we introduce an innovative method which can retrieve the information about the variables of complex data sets. We apply the proposed method to several benchmark data sets and a metabolomics data set to illustrate that we can determine the contribution of the original variables in SVM classifications. The corresponding visualization of the contribution of the variables can assist in a better understanding of the underlying chemical or biological process. In the past decade support vector machines (SVMs) have become a popular technique in pattern recognition and regression estimation. Applications of SVMs are among the fields of bioinformatics, 1,2 medicine, 3-6 drug discovery, 7-9 text categorizing, 10 gene expression analysis, 11-13 face recognition, 14 spam * To whom correspondence should be addressed. Phone: +31 24 3653180. Fax: +31 24 3652653. E-mail: l.buydens@science.ru.nl. † Radboud University Nijmegen. ‡ Analytical Sciences Chemicals. (1) Yang, Z. R. Briefings Bioinf. 2004, 5 (4), 328–338. (2) Ramo, P.; Sacher, R.; Snijder, B.; Begemann, B.; Pelkmans, L. Bioinformatics. 2009, 25, 3028–3030. (3) Akay, M. F. Expert Syst. Appl 2009, 36 (2), 3240–3247. (4) Magnin, B.; et al. Neuroradiology. 2009, 51 (2), 73–83. (5) Luts, J.; Heerschap, A.; Suykens, J. A. K.; Van Huffel, S. Artif. Intell. Med. 2007, 40 (2), 87–102. (6) Conforti, D.; Guido, R. Comput. Oper. Res. 2010, 37, 1389–1394. (7) Burbidge, R.; Trotter, M.; Buxton, B.; Holden, S. Comput. Chem. 2001, 26, 5–14. (8) Warmuth, M. K.; et al. J. Chem. Inf. Comput. Sci. 2003, 43, 667–673. (9) Zernov, V. V.; Balakin, K. V.; Ivaschenko, A. A.; Savchuk, N. P.; Pletnev, I. V. J. Chem. Inf. Comput. Sci. 2003, 43, 2048–2056. (10) Leopold, E.; Kindermann, J. Mach. Learn. 2002, 46, 423–444. (11) Furey, T. S.; et al. Bioinformatics. 2000, 16 (10), 906–914. (12) Clarke, R.; et al. Nat. Rev. Cancer. 2008, 8, 37–49. (13) Noble, W. S. Nat. Biotechnol. 2006, 24 (12), 1565–1567. (14) Guo, G.; Li, S. Z.; Chan, K. L. Image Visualization Comput. 2001, 19, 631– 638. 7000 Analytical Chemistry, Vol. 82, No. 16, August 15, 2010 categorizing, 15,16 financial forecasting, 17,18 and many others. Especially because of the possibility to model complex nonlinear relationships the application of SVMs has grown substantially. 19-22 By transforming the original input space into a high dimensional feature space, the nonlinear relationships can be presented in a linear form. This transformation is performed by using a specific kernel function. 6,23,24 Several kernel functions are proposed in the literature for this purpose and include variance-covariance based linear and polynomial kernels, the Euclidean distance based radial basis function (RBF) and the Pearson VII Universal Kernel (PUK) functions. 23-25 The transformation by a kernel function has also been introduced in other algorithms, such as Kernel Principal Component Analysis, 26 Kernel Partial Least Squares, 27,28 and Kernel Fisher Discrimination. 29 However, the disadvantage of using such a kernel function is that the correlation between the obtained SVM model and the original input space is lost. Therefore it is not possible to determine which variables (e.g., spectral ranges) contribute to the final SVM results and a direct interpretation of the SVM model is not straightforward. 30,31 This seriously hampers the ultimate (bio)chemical interpretation of the resulting classification model. In this manuscript we propose a novel method to overcome this problem and reveal the importance of the original variables. (15) Drucker, H.; Wu, D.; Vapnik, V. N. IEEE Trans. Neural Networks 1999, 10 (5), 1048–1054. (16) Guzella, T. S.; Caminhas, W. M. Expert Syst. Appl. 2009, 36 (7), 10206– 10222. (17) Tay, F. E. H.; Cao, L. Omega. 2001, 29, 309–317. (18) Kim, K. J. Neurocomputing. 2003, 55, 307–319. (19) Vapnik, V. Estimation of Dependence Based on Empirical Data; Springer Verlag: New York, 1982. (20) Vapnik, V. the Nature of Statistical Learning Theory; Springer Verlag: New York, 1995. (21) Cortes, C.; Vapnik, V. Mach. Learn. 1995, 20, 273–297. (22) Vapnik, V. Statistical Learning Theory; John Wiley and Sons: New York, 1998. (23) Cristianini, N.; Shawe-Taylor, J. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods; Cambridge University Press.: Cambridge, 2000. (24) Schölkopf, B.; Smola, A. J. Learning with Kernels; MIT Press.: Cambridge, 2002. (25) Üstün, B.; Melssen, W. J.; Buydens, L. M. C. Chemom. Intell. Lab. Syst. 2006, 81, 29–40. (26) Schölkopf, B.; Smola, A. J.; Müller, K. R. Neural Comput. 1998, 10, 1299– 1319. (27) Walczak, B.; Massart, D. L. Anal. Chim. Acta 1996, 331, 177–185. (28) Rosipal, R.; Trejo, L. J. J. Mach. Learn. Res. 2001, 2, 97–123. (29) Mika, S.; Rãtsch, G.; Wetson, J.; Schölkopf, B.; Müller, K. R. Proc. NNSP′99; 1999, 41–48. 10.1021/ac101338y © 2010 American Chemical Society Published on Web 07/20/2010
Another approach to handle this limitation is automatic relevance determination, 32,33 which was developed as a feature selection procedure in SVM models. Even though this method selects the variables that are important for the model, the relation between the variables and, for example, the class separation is not visualized. Generally, researchers report the high performance obtained by using a SVM model for classification and regression estimation, but do not comment on the relationship between the input variables and the modeled output data. SVM is often used as a black box approach. In this paper we present an innovative approach to open this black box for the SVM classifier and give insight in the transformation by the kernel function to make the SVM model more transparent. This approach is based on the nonlinear biplot principles described by Gower and Harding in 1988 34 and is used to visualize and determine the importance and influence of the input variables to the final SVM classifier. The resulting information can then be used to reduce the number of input variables to improve the performance or to reduce the complexity of the model. Furthermore, the visualization of the contribution of the original variables can assist in a better understanding of the underlying chemical or biological process. We will present the proposed approach, and illustrate and validate the methodology by applying it for classification problems: two benchmark data sets and a relevant metabolomics data set obtained from magnetic resonance spectroscopic images to diagnose human brain tumors. The effectiveness of the method is verified by a comparison of the classification performance obtained by using the entire set of input variables and a selection of variables, determined by our approach. EXPERIMENTAL SECTION Theory and Computational Strategy. As the theory of SVMs is described extensively in the literature 13,21-24 and the use of a kernel transformation results in the loss of information about the input variables, we will focus in the next section briefly on the concepts of the kernel function. Subsequently, we will explain the basic steps of the nonlinear biplot technique, as described by Gower and Harding, 34 to discuss the proposed method to determine the contribution of the input variables in SVM classifications. The full theory and algorithm of the proposed method can be found in the Supporting Information (SI). Kernel Transformations. In the various kernel-based methods a specific mapping function is used to project the original input data in a higher-dimensional feature space. 24 The data in this new feature space can subsequently be used as a new input for pattern recognition. The advantage of such an approach is that the method can deal with complex nonlinear problems. The typical (twodimensional) example to illustrate this approach is shown in the inset of Figure 1. This data set (X) contains two classes that we would like to separate. (30) Üstün, B.; Melssen, W. J.; Buydens, L. M. C. Anal. Chim. Acta 2007, 595, 299–309. (31) Devos, O.; Ruckebusch, C.; Durand, A.; Duponchel, L.; Huvenne, J. P. Chemom. Intell. Lab. Syst. 2009, 96, 27–33. (32) Van Gestel, T.; Suykens, J. A. K.; De Moor, B.; Vandewalle, J. Proc. Eur. Symp. Artif. Neural Networks. 2001, 13–18. (33) MacKay, D. J. C. In Neural Networks and Machine Learning, NATO Asi Series. Series F, Computer and Systems Sciences 168; Bishop, C. M., Ed.; Springer: Berlin, 1998; pp 133-165. (34) Gower, J. C.; Harding, S. A. Biometrika 1988, 75, 445–455. Figure 1. The synthetic benchmark data set. Mapping of a twodimensional data set which contains two nonlinearly separable classes (outer and inner circles) into a three-dimensional feature space. The transformation is performed by applying the nonlinear mapping function φ(x.1, x.2) ) (x.1 2 ,�2x.1x.2, x.2 2 ), which makes the two classes linearly separable. The inner circle represents one class and the outer circle represents the other class. From the figure it is obvious that we are not able to separate the classes by a linear model. However, if we project this data in a higher dimensional space by using a mapping function, we are able to obtain the three-dimensional feature space which is represented in Figure 1. In this feature space the two classes can be linearly separated by a plane between the circles. In this case the transformation is performed by the mapping function φ. It has been shown that, since the (nonlinear) mapping function is in general unknown beforehand and is difficult to determine, the feature space can be constructed implicitly by invoking a generic kernel function (see, e.g., refs 23-25, 35). Such a kernel function is a function (K) which operates on two vectors, such that K(x i. , x j. ) ) 〈�(x i. ), �(x j. )〉 (1) where xi. and xj. are two objects in the data set and φ represents the actual nonlinear mapping function. The use of a kernel function makes it unnecessary to know the actual underlying feature map in order to be able to construct a linear model in the feature space. The application of such a kernel function will result in a square symmetric matrix: the kernel matrix K. This matrix is a weighted dissimilarity matrix, of which each position represents a dissimilarity (distance) measure between two objects. The specific kernel function and the optimal parameter settings of the function are determined in combination with the applied classification algorithm (e.g., SVM) by optimizing the classification performance. However, because the kernel function transforms the original input space into a feature space with a higher dimension, information about the original variables is not preserved. Nonlinear Biplots. To explain the concepts of nonlinear biplots, we will first describe the classical biplots technique which is used extensively in (chemometric) data analysis. (35) Gunn, S. R. Support Vector Machines for Classification and Regression. Technical Report; Image Speech and Intelligent Systems Research Group, University of Southampton: Southampton, 1997. Analytical Chemistry, Vol. 82, No. 16, August 15, 2010 7001
Page 1 and 2:
Anal. Chem. 2010, 82, 6745-6750 Let
Page 3 and 4:
purchased from Invitrogen-Molecular
Page 5 and 6:
Figure 3. Electropherograms of TPP-
Page 7 and 8:
Anal. Chem. 2010, 82, 6751-6755 Res
Page 9 and 10:
(pH 8.0) with cysteine and cystamin
Page 11 and 12:
Figure 4. Ion mobility mass spectra
Page 13 and 14:
GOx glucose + O298 gluconic acid +
Page 15 and 16:
coater at 2000 rpm for 20 s, and th
Page 17 and 18:
Figure 5. Comparison of cyclic volt
Page 19 and 20:
in channels with either no grooves
Page 21 and 22:
indicators of atmospheric processin
Page 23 and 24:
Figure 1. GC/MS total ion chromatog
Page 25 and 26:
Table 2. Concentrations and Stable
Page 27 and 28:
on a substrate are preferred. 20-24
Page 29 and 30:
Figure 4. SERS analysis of NAADP co
Page 31 and 32:
Anal. Chem. 2010, 82, 6775-6781 Hig
Page 33 and 34:
tion, 2 µL of proprionaldehyde wer
Page 35 and 36:
Figure 4. Analysis of 2a by HPLC-MS
Page 37 and 38:
eaction of 1a with PBH can be condu
Page 39 and 40:
Numerous references had demonstrate
Page 41 and 42:
Figure 1. TEM images of the prepare
Page 43 and 44:
Figure 4. Schematic representation
Page 45 and 46:
esult in a big SPR signal change wi
Page 47 and 48:
also reduces chemical noise, which
Page 49 and 50:
Table 1. Extraction Yields, Liquid
Page 51 and 52:
scanning of AMPP amides of the anal
Page 53 and 54:
Anal. Chem. 2010, 82, 6797-6806 δ
Page 55 and 56:
Figure 1. Schematic view of the pre
Page 57 and 58:
from the specimen and enclosed in a
Page 59 and 60:
Figure 4. (A) IRMS mass-44 chromato
Page 61 and 62:
Table 3. Ambient Measurement Result
Page 63 and 64:
Anal. Chem. 2010, 82, 6807-6813 Dir
Page 65 and 66:
Polymerase (1 U per sample). Reacti
Page 67 and 68:
of which was constant for all ampli
Page 69 and 70:
analytically useful signals at less
Page 71 and 72:
carbon black and RP-C18 for the ext
Page 73 and 74:
solution in an equal volume, and 1
Page 75 and 76:
Table 2. Concentrations and Ratios
Page 77 and 78:
Anal. Chem. 2010, 82, 6821-6829 Mac
Page 79 and 80:
mg/mL protein, followed by separati
Page 81 and 82:
Figure 2. Productivity of SEQUST an
Page 83 and 84:
Figure 4. High-resolution MS/MS spe
Page 85 and 86:
Figure 6. Characterization of PSMs
Page 87 and 88:
containing T-T mismatches. 23 Based
Page 89 and 90:
Figure 1. Extinction spectra of sol
Page 91 and 92:
Figure 3. (A) The value of Ex650 nm
Page 93 and 94:
Figure 5. Extinction spectra and co
Page 95 and 96:
wished to explore the dehydration o
Page 97 and 98:
constant medium for separation; we
Page 99 and 100:
Figure 2. Standards of (Pi)n, n ) 1
Page 101 and 102:
Figure 5. Quantitative calibration
Page 103 and 104:
Anal. Chem. 2010, 82, 6847-6853 Met
Page 105 and 106:
Figure 1. 226 Ra spectrum by liquid
Page 107 and 108:
Table 1. Counting Properties and De
Page 109 and 110:
Table 3. Analysis of 226 Ra in Sedi
Page 111 and 112:
mercial microarray scanner and fabr
Page 113 and 114:
Figure 2. Optical transmission meas
Page 115 and 116:
Figure 5. Volcano plots detailing t
Page 117 and 118:
data as well. The 41 genes in Table
Page 119 and 120:
corresponding compound if its chemi
Page 121 and 122:
Figure 1. Schematic illustration of
Page 123 and 124:
Table 1. Absolute Quantification Re
Page 125 and 126:
ment. Therefore, the long-time drea
Page 127 and 128:
Figure 1. Chemically actuated micro
Page 129 and 130:
Figure 2. Influence of a surfactant
Page 131 and 132:
Figure 5. Device to eject and mix s
Page 133 and 134:
Anal. Chem. 2010, 82, 6877-6886 Imm
Page 135 and 136:
NaCl, phosphate buffer saline (PBS)
Page 137 and 138:
Figure 2. Product ion mass spectra
Page 139 and 140:
-70 °C resolved the problem, givin
Page 141 and 142:
Table 1. Intraday Precision and Acc
Page 143 and 144:
Anal. Chem. 2010, 82, 6887-6894 Fer
Page 145 and 146:
Figure 1. Infrared spectra of (A) u
Page 147 and 148:
Figure 3. Cyclic voltammograms obta
Page 149 and 150:
Figure 6. Calculated charge from ch
Page 151 and 152:
Anal. Chem. 2010, 82, 6895-6903 Ele
Page 153 and 154:
Table 1. Chemical Structure, pKa Va
Page 155 and 156:
pH with a tilted baseline (Figure 1
Page 157 and 158:
Table 2. Linearity and Detection Li
Page 159 and 160:
ascorbic acid (AA), uric acid (UA),
Page 161 and 162:
the multielement capabilities, the
Page 163 and 164:
RESULTS AND DISCUSSION Sulfur Detec
Page 165 and 166:
Table 2. Molecular Properties and C
Page 167 and 168:
Anal. Chem. 2010, 82, 6911-6918 Dir
Page 169 and 170:
dilution and hybridization buffer.
Page 171 and 172:
solution under appropriate incubati
Page 173 and 174:
Figure 4. Standardization curve for
Page 175 and 176:
Anal. Chem. 2010, 82, 6919-6925 Ele
Page 177 and 178:
the ×10 objective, to have a large
Page 179 and 180:
Figure 2. With a suitable removal o
Page 181 and 182:
the NB signal in a much better foot
Page 183 and 184:
Scheme 1. Reactions of Selenium Rea
Page 185 and 186:
Figure 2. (a) ESI-MS spectrum showi
Page 187 and 188:
Figure 4. (a) ESI-MS spectrum showi
Page 189 and 190:
Anal. Chem. 2010, 82, 6933-6939 Dif
Page 191 and 192:
trode 28 by a finite element using
Page 193 and 194:
Figure 4. Comparison between simula
Page 195 and 196:
Figure 6. Comparison between simula
Page 197 and 198:
educed in the vicinity of double bo
Page 199 and 200:
Figure 2. Normalized product ion ab
Page 201 and 202:
Figure 4. EID (a) and IRMPD (b) of
Page 203 and 204:
Anal. Chem. 2010, 82, 6947-6957 Ide
Page 205 and 206: Figure 1. Schematic flowchart showi
Page 207 and 208: difference, ppm compound Table 1. I
Page 209 and 210: isoforms, its successful use, in th
Page 211 and 212: Figure 5. Extracted ion current ESI
Page 213 and 214: m/z 1172.935 was observed for Ser14
Page 215 and 216: linked products via affinity tags.
Page 217 and 218: Scheme 2. Fragmentation Mechanism o
Page 219 and 220: Figure 1. (A) ESI-LTQ-CID-MS 2 prod
Page 221 and 222: Figure 3. (A) ESI-LTQ-CID-MS 2 prod
Page 223 and 224: Figure 5. (A) MALDI-TOF/TOF product
Page 225 and 226: Anal. Chem. 2010, 82, 6969-6975 Ana
Page 227 and 228: Figure 3. Equilibrium response as a
Page 229 and 230: Figure 6. The average measured resp
Page 231 and 232: for the fill time, and we find that
Page 233 and 234: nucleotide tails. 3-5 Thus, the amo
Page 235 and 236: allow the use of higher aptamer con
Page 237 and 238: quent ligation of the aptamers afte
Page 239 and 240: Anal. Chem. 2010, 82, 6983-6990 Imp
Page 241 and 242: Figure 2. Configuration editor wind
Page 243 and 244: Figure 4. Dependencies between volu
Page 245 and 246: Since the time for liquid expulsion
Page 247 and 248: Anal. Chem. 2010, 82, 6991-6999 Int
Page 249 and 250: Figure 1. Representation of the Car
Page 251 and 252: Table 2. Capillary Electrophoresis
Page 253 and 254: Figure 4. (A) Typical raw electroph
Page 255: field. DNA profiles were delivered
Page 259 and 260: (principal components, PCs) in whic
Page 261 and 262: Figure 5. Relevant score plots and
Page 263 and 264: CONCLUSIONS In this paper we have i
Page 265 and 266: electroactive-species loaded liposo
Page 267 and 268: Figure 2. Qdot-based FLFTS response
Page 269 and 270: Figure 6. Fluorescence imaging of Q
Page 271 and 272: Anal. Chem. 2010, 82, 7015-7020 Sel
Page 273 and 274: Table 1. Effect of 1 D-LC Condition
Page 275 and 276: Figure 3. Peak-production rate vers
Page 277 and 278: Anal. Chem. 2010, 82, 7021-7026 Cel
Page 279 and 280: luciferase (T7 control vector) was
Page 281 and 282: Figure 3. Inhibitory effects of luc
Page 283 and 284: Anal. Chem. 2010, 82, 7027-7034 Pat
Page 285 and 286: Electrochemistry. Electrochemical m
Page 287 and 288: Figure 2. SEM images of Au substrat
Page 289 and 290: Table 3. Film Thickness Measurement
Page 291 and 292: Anal. Chem. 2010, 82, 7035-7043 Qua
Page 293 and 294: Figure 1. (A) FL spectra of BSPOTPE
Page 295 and 296: Figure 4. (A) Variation in the FL i
Page 297 and 298: Figure 6. (A) FL spectra of BSPOTPE
Page 299 and 300: Scheme 1. Proposed Mechanism for Fl
Page 301 and 302: one or more drawbacks including poo
Page 303 and 304: Figure 2. Free fluorophores do not
Page 305 and 306: Anal. Chem. 2010, 82, 7049-7052 Dev
Page 307 and 308:
Figure 2. Comparative analysis of d
show all

Analytical Chemistry Chemical Cytometry Quantitates Superoxide

Create successful ePaper yourself

Delete template?

Save as template?