<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Neuroinform.</journal-id>
<journal-title>Frontiers in Neuroinformatics</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Neuroinform.</abbrev-journal-title>
<issn pub-type="epub">1662-5196</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fninf.2022.893788</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neuroscience</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Machine Learning Techniques for the Diagnosis of Schizophrenia Based on Event-Related Potentials</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Santos Febles</surname> <given-names>Elsa</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/1713655/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Ontivero Ortega</surname> <given-names>Marlis</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff3"><sup>3</sup></xref>
</contrib>
<contrib contrib-type="author">
<name><surname>Vald&#x000E9;s Sosa</surname> <given-names>Michell</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/74869/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Sahli</surname> <given-names>Hichem</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<xref ref-type="aff" rid="aff4"><sup>4</sup></xref>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Cuban Neuroscience Center</institution>, <addr-line>Havana</addr-line>, <country>Cuba</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel (VUB)</institution>, <addr-line>Brussels</addr-line>, <country>Belgium</country></aff>
<aff id="aff3"><sup>3</sup><institution>Department of Data Analysis, Faculty of Psychology and Educational Sciences, Ghent University</institution>, <addr-line>Ghent</addr-line>, <country>Belgium</country></aff>
<aff id="aff4"><sup>4</sup><institution>Interuniversity Microelectronics Centre (IMEC)</institution>, <addr-line>Leuven</addr-line>, <country>Belgium</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Antonio Fern&#x000E1;ndez-Caballero, University of Castilla-La Mancha, Spain</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Qunxi Dong, Beijing Institute of Technology, China; Tanu Wadhera, Thapar Institute of Engineering &#x00026; Technology, India</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Elsa Santos Febles <email>elsa&#x00040;cneuro.edu.cu</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>08</day>
<month>07</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="collection">
<year>2022</year>
</pub-date>
<volume>16</volume>
<elocation-id>893788</elocation-id>
<history>
<date date-type="received">
<day>10</day>
<month>03</month>
<year>2022</year>
</date>
<date date-type="accepted">
<day>09</day>
<month>06</month>
<year>2022</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2022 Santos Febles, Ontivero Ortega, Vald&#x000E9;s Sosa and Sahli.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Santos Febles, Ontivero Ortega, Vald&#x000E9;s Sosa and Sahli</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license></permissions>
<abstract>
<sec>
<title>Antecedent</title>
<p>The event-related potential (ERP) components P300 and mismatch negativity (MMN) have been linked to cognitive deficits in patients with schizophrenia. The diagnosis of schizophrenia could be improved by applying machine learning procedures to these objective neurophysiological biomarkers. Several studies have attempted to achieve this goal, but no study has examined Multiple Kernel Learning (MKL) classifiers. This algorithm finds optimally a combination of kernel functions, integrating them in a meaningful manner, and thus could improve diagnosis.</p></sec>
<sec>
<title>Objective</title>
<p>This study aimed to examine the efficacy of the MKL classifier and the Boruta feature selection method for schizophrenia patients (SZ) and healthy controls (HC) single-subject classification.</p></sec>
<sec>
<title>Methods</title>
<p>A cohort of 54 SZ and 54 HC participants were studied. Three sets of features related to ERP signals were calculated as follows: peak related features, peak to peak related features, and signal related features. The Boruta algorithm was used to evaluate the impact of feature selection on classification performance. An MKL algorithm was applied to address schizophrenia detection.</p></sec>
<sec>
<title>Results</title>
<p>A classification accuracy of 83% using the whole dataset, and 86% after applying Boruta feature selection was obtained. The variables that contributed most to the classification were mainly related to the latency and amplitude of the auditory P300 paradigm.</p></sec>
<sec>
<title>Conclusion</title>
<p>This study showed that MKL can be useful in distinguishing between schizophrenic patients and controls when using ERP measures. Moreover, the use of the Boruta algorithm provides an improvement in classification accuracy and computational cost.</p></sec></abstract>
<kwd-group>
<kwd>multiple kernel learning</kwd>
<kwd>schizophrenia</kwd>
<kwd>Boruta</kwd>
<kwd>feature selection</kwd>
<kwd>event related potential</kwd>
<kwd>machine learning</kwd>
</kwd-group>
<counts>
<fig-count count="5"/>
<table-count count="4"/>
<equation-count count="6"/>
<ref-count count="62"/>
<page-count count="11"/>
<word-count count="7123"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="s1">
<title>Introduction</title>
<p>Schizophrenia is a severe and persistent debilitating psychiatric disorder with a prevalence of about 1% of the world population (McGrath et al., <xref ref-type="bibr" rid="B40">2004</xref>). Although psychotic symptoms such as hallucinations and delusions are frequently present, impaired information processing is probably the most common symptom (Javitt et al., <xref ref-type="bibr" rid="B22">1993</xref>). This deficit is reflected mainly by deficits in attention and working memory tasks when compared with healthy controls (Li et al., <xref ref-type="bibr" rid="B35">2018</xref>). The diagnosis of schizophrenia is made by psychiatrists by ascertaining the presence of predefined symptoms (or their precursors) with personal interviews. However, in some cases this diagnosis is unclear, or patients are misdiagnosed with Schizophrenia (Coulter et al., <xref ref-type="bibr" rid="B11">2019</xref>). Thus, finding biomarkers for the prediction of individuals with schizophrenia would be desirable to enable choosing the optimal treatment (pharmacologic or non-pharmacologic). Analysis of the electroencephalogram (EEG) during information processing tasks could provide objective complementary measures to support the subjective human-based decision process (Sabeti et al., <xref ref-type="bibr" rid="B47">2009</xref>; Koukkou et al., <xref ref-type="bibr" rid="B28">2018</xref>).</p>
<p>EEG is a non-invasive and low-cost technique used to measure electrical brain activity along with multiple scalp locations. EEG signals have been widely adopted to study mental disorders, such as dementia, epileptic seizures, cognitive dysfunction, among others, as well as schizophrenia (Loo et al., <xref ref-type="bibr" rid="B38">2016</xref>; Olbrich et al., <xref ref-type="bibr" rid="B45">2016</xref>; Horvath et al., <xref ref-type="bibr" rid="B18">2018</xref>). Electrophysiological data reflects the spontaneous activity of myriad brain parcels, but also can include responses to afferent stimuli (Cong et al., <xref ref-type="bibr" rid="B10">2015</xref>). Event-related potentials (ERPs) are electrical responses that are time-locked to a specific stimulus or event and can be used to assess brain dynamics during information processing in specific tasks (Woodman, <xref ref-type="bibr" rid="B59">2010</xref>). When a subject is presented with a series of standard stimuli, interspersed with infrequent deviant stimuli, the Mismatch Negativity (MMN) (Lee et al., <xref ref-type="bibr" rid="B34">2017</xref>) and the P300 (Li et al., <xref ref-type="bibr" rid="B35">2018</xref>) components are generated. This task is known as the oddball paradigm and is used to study schizophrenia since consistent deficits in the P300 and MNN have been reported in this disease (Bramon et al., <xref ref-type="bibr" rid="B7">2004</xref>; Javitt et al., <xref ref-type="bibr" rid="B23">2017</xref>). Although MMN and P300 are usually produced by an infrequent unexpected event in a sequence of auditory stimuli, P300 can also be obtained with visual stimuli. The MMN is of shorter latency and does not require attention to the stimulus (N&#x000E4;&#x000E4;t&#x000E4;nen et al., <xref ref-type="bibr" rid="B42">2004</xref>), whereas the P300 is of longer latency and requires attention to the stimulus (Huang et al., <xref ref-type="bibr" rid="B19">2015</xref>).</p>
<p>Several studies have reported significant differences in the latency and amplitude of MMN and P300 between controls and patients, suggesting that these features are possible markers of the prodromal phase of schizophrenia (Atkinson et al., <xref ref-type="bibr" rid="B3">2012</xref>; Loo et al., <xref ref-type="bibr" rid="B38">2016</xref>) as well as potential endophenotypes for schizophrenia (Earls et al., <xref ref-type="bibr" rid="B14">2016</xref>). Analysis of a large dataset of auditory P300 ERP (649 controls and 587 patients) confirmed the reliability of this reduced amplitude, with a large effect size (Turetsky et al., <xref ref-type="bibr" rid="B56">2015</xref>). However, these findings of statistically significant differences in a group analysis do not imply that EEG is useful for the prediction of individual schizophrenia cases (Lo et al., <xref ref-type="bibr" rid="B37">2015</xref>), which requires applying a prediction paradigm using Machine Learning.</p>
<p>Machine learning techniques have potential value for assisting the diagnosis of brain disorders (Burgos and Colliot, <xref ref-type="bibr" rid="B8">2020</xref>). Recent works are based on EEG signals for the diagnosis of epilepsy (Tanu, <xref ref-type="bibr" rid="B55">2018</xref>), Alzheimer&#x00027;s disease and dementia (Joshi and Nanavati, <xref ref-type="bibr" rid="B24">2021</xref>), and Parkinson (Mait&#x000ED;n et al., <xref ref-type="bibr" rid="B39">2020</xref>), among other disorders. Particularly, ERP measures combined with machine learning techniques are being tested for the classification of schizophrenia. The most common features used are based on the amplitude and latency of different components [e.g., N100 and P300 (Neuhaus et al., <xref ref-type="bibr" rid="B43">2013</xref>), P50 and N100 (Iyer et al., <xref ref-type="bibr" rid="B21">2012</xref>; Neuhaus et al., <xref ref-type="bibr" rid="B44">2014</xref>)], with several classifiers tested. Neuhaus et al. (<xref ref-type="bibr" rid="B43">2013</xref>) used visual and auditory oddball paradigms and a k-nearest neighbor (KNN) classifier and obtained a classification accuracy of 72.4 %. The same author with a bigger sample size and a Naive Bayes (NB) classifier achieved a 77.7% of accuracy (Iyer et al., <xref ref-type="bibr" rid="B21">2012</xref>). Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>) evaluated the performance of several classifiers extracting features from auditory/visual P300 and MMN. The results using NB and Decision Tree (without and with AdaBoost) achieved accuracies of about 80%. Recently, Barros et al. (<xref ref-type="bibr" rid="B4">2021</xref>) published a critical review that summarizes machine learning-based classification studies to detect SZs based on EEG signals, conducted since 2016. These authors reported that Support Vector Machines (SVM) were the most commonly used classification algorithm, probably due to their computational efficiency. This kernel-based learning method also achieved the best performance in most studies. Nevertheless, to the best of our knowledge, none of the studies focused on ERP for SZ classification have used multiple kernels, employing instead only one specific kernel function.</p>
<p>The multiple kernel learning (MKL) method learns a weighted combination of different kernel functions and can benefit from information coming from multiple sources (Wani and Raza, <xref ref-type="bibr" rid="B58">2018</xref>). A recent survey of artificial intelligence methods for the classification and detection of Schizophrenia (Lai et al., <xref ref-type="bibr" rid="B32">2021</xref>), shows that MKL has been applied to both structural and functional Magnetic Resonance Images (MRI), increasing performance accuracy (Ula&#x0015F; et al., <xref ref-type="bibr" rid="B57">2012</xref>; Castro et al., <xref ref-type="bibr" rid="B9">2014</xref>; Iwabuchi and Palaniyappan, <xref ref-type="bibr" rid="B20">2017</xref>). Nevertheless, in this review MKL algorithms applied to electrophysiological data have been not reported, although a recent study used EEG dynamic functional connectivity networks to classify SZ based on MKL (Dimitriadis, <xref ref-type="bibr" rid="B13">2019</xref>). To our knowledge, ERP data has not been used to classify SZ using MKL despite its use for other purposes such as brain-computer interfaces (Li et al., <xref ref-type="bibr" rid="B36">2014</xref>; Yoon and Kim, <xref ref-type="bibr" rid="B60">2017</xref>).</p>
<p>Here, we explore the efficacy of MKL for the classification of schizophrenia based on ERP measures extracted from auditory and visual P300 and MMN. Using the same dataset provided by Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>), we extended the set of predictor variables beyond the latency and amplitude of the ERP components, by including additional morphological features (based on time) together with some features extracted from the frequency domain. Due to the huge number of features, the Boruta method (Kursa and Rudnicki, <xref ref-type="bibr" rid="B31">2010</xref>) was applied, which is a wrapper Random Forest (RF) based feature selection algorithm, to estimate the impact of a subset of important and relevant feature variables in the classification accuracy.</p></sec>
<sec sec-type="materials and methods" id="s2">
<title>Materials and Methods</title>
<sec>
<title>Dataset</title>
<p>The study (Laton et al., <xref ref-type="bibr" rid="B33">2014</xref>) was carried out on data from 54 SZ patients and 54 HC, matched for age and gender. Patients were classified by a semi-structured interview (OPCRIT v4.0) and all participants gave written informed consent. Detailed demographic data can be found in <xref ref-type="table" rid="T1">Table 1</xref>. EEGs were recorded using a 64-channel and the international 10/10 system, with a sampling frequency of 256 Hz. Three paradigms were used: auditory/visual P300 and MMN. <xref ref-type="table" rid="T2">Table 2</xref> shows a brief description of the paradigms and procedures. We refer to Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>) for the study details.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Demographic data (Laton et al., <xref ref-type="bibr" rid="B33">2014</xref>).</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th valign="top" align="center"><bold>Patients</bold></th>
<th valign="top" align="center"><bold>Controls</bold></th>
<th valign="top" align="center"><bold><italic>P</italic> (<italic>t</italic>-test)</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Number of participants</td>
<td valign="top" align="center">54</td>
<td valign="top" align="center">54</td>
<td/>
</tr>
<tr>
<td valign="top" align="left">Male</td>
<td valign="top" align="center">36</td>
<td valign="top" align="center">36</td>
<td/>
</tr>
<tr>
<td valign="top" align="left">Age (years): mean &#x000B1; std</td>
<td valign="top" align="center">40.5 &#x000B1; 10.1</td>
<td valign="top" align="center">37.6 &#x000B1; 14.1</td>
<td valign="top" align="center">0.22</td>
</tr>
<tr>
<td valign="top" align="left">Age (years): range</td>
<td valign="top" align="center">[22.4, 60.5]</td>
<td valign="top" align="center">[15.1, 64.4]</td>
<td/>
</tr>
<tr>
<td valign="top" align="left">Education (years): mean &#x000B1; std</td>
<td valign="top" align="center">12.6 &#x000B1; 1.80</td>
<td valign="top" align="center">14.8 &#x000B1; 2.11</td>
<td valign="top" align="center">4.84 &#x000D7;10&#x02013;5</td>
</tr>
<tr>
<td valign="top" align="left">Disease duration (years): mean &#x000B1; std</td>
<td valign="top" align="center">14.8 &#x000B1; 9.04</td>
<td valign="top" align="center">&#x02013;</td>
<td/>
</tr>
<tr>
<td valign="top" align="left">Disease duration (years): range</td>
<td valign="top" align="center">[1, 40]</td>
<td valign="top" align="center">&#x02013;</td>
<td/>
</tr>
</tbody>
</table>
</table-wrap>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Paradigms and procedures (Laton et al., <xref ref-type="bibr" rid="B33">2014</xref>).</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th valign="top" align="left"><bold>Auditory P300</bold></th>
<th valign="top" align="left"><bold>Visual P300</bold></th>
<th/>
</tr>
<tr>
<th/>
<th valign="top" align="left"><bold>Tone</bold></th>
<th valign="top" align="left"><bold>Figure</bold></th>
<th valign="top" align="center"><bold>Distribution (%)</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Target</td>
<td valign="top" align="left">1500 Hz 70 dB</td>
<td valign="top" align="left">Square, side 106 pixels</td>
<td valign="top" align="center">10</td>
</tr>
<tr>
<td valign="top" align="left">Distractor</td>
<td valign="top" align="left">500 Hz 70 dB</td>
<td valign="top" align="left">Circle, diameter 176 pixels</td>
<td valign="top" align="center">10</td>
</tr>
<tr>
<td valign="top" align="left">Standard</td>
<td valign="top" align="left">1000 Hz 70 dB</td>
<td valign="top" align="left">Square, side 158 pixels</td>
<td valign="top" align="center">80</td>
</tr> <tr>
<td valign="top" align="left" colspan="4">Inter-stimulus interval was randomized between 1 and 1.5 s. 400 stimuli per test. 100 ms per stimuli. Total test time of 540 s.</td>
</tr> <tr>
<td/>
<td valign="top" align="left" colspan="3"><bold>MMN</bold></td>
</tr>
<tr>
<td/>
<td valign="top" align="left"><bold>Tone</bold></td>
<td valign="top" align="left"><bold>Duration</bold></td>
<td valign="top" align="center"><bold>Distribution</bold></td>
</tr> <tr>
<td valign="top" align="left">Duration deviant</td>
<td valign="top" align="left">1000 Hz 70 dB</td>
<td valign="top" align="left">250 ms</td>
<td valign="top" align="center">5%</td>
</tr>
<tr>
<td valign="top" align="left">Frequency deviant</td>
<td valign="top" align="left">1500 Hz 70 dB</td>
<td valign="top" align="left">100 ms</td>
<td valign="top" align="center">5%</td>
</tr>
<tr>
<td valign="top" align="left">Standard</td>
<td valign="top" align="left">1000 Hz 70 dB</td>
<td valign="top" align="left">100 ms</td>
<td valign="top" align="center">90%</td>
</tr> <tr>
<td valign="top" align="left" colspan="4">Inter-stimulus interval of 300 ms, 1800 tones per test. Total test time of 733 s.</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>The signals were filtered using bandpass Butterworth filters with cut-offs at 0.1 and 30 Hz. Epochs were extracted using time windows between &#x02212;200 and 800 ms for the P300 paradigms (257 discrete data points) and between &#x02212;100 and 500 ms for the MMN (155 discrete data points). Subsequently, baseline correction, re-referencing to linked ears, and artifact rejection were performed. Finally, epochs were averaged into stimulus-specific responses for each individual, and low-pass filter and baseline correction were re-applied. More details can be found in Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>).</p></sec>
<sec>
<title>Feature Extraction</title>
<p>The initial set of measurement data consists of averaged signals of 62 channels for each specific response to the three paradigms. This leads to a large amount of data, so it is necessary to transform the initial raw data into a set of features, or signal characteristics that better represent the underlying problem. The process of transforming the signals into numerical features has been carried out on the waveform of ERPs emerged as the averaging of the electrical responses corresponding to the set of stimuli implicated in the detection of rare events (Target and Distractor for P300, Duration and Deviant for MNN), which are more prominent at midline scalp electrode locations Fz, Cz, and Pz (B&#x000E9;nar et al., <xref ref-type="bibr" rid="B5">2007</xref>). As stated, SZ typically exhibits smaller amplitudes in these components compared to HC (Li et al., <xref ref-type="bibr" rid="B35">2018</xref>). Additionally, several studies demonstrated P300 and MMN component differences between SZs and HCs at midline electrodes (Hirayasu et al., <xref ref-type="bibr" rid="B17">1998</xref>; Graber et al., <xref ref-type="bibr" rid="B16">2019</xref>), thus only these channels were considered (see <xref ref-type="fig" rid="F1">Figure 1</xref>).</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Averaged evoked potential signals used for feature extraction.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fninf-16-893788-g0001.tif"/>
</fig>
<p>The set of features can be divided into three categories: peak related features, peak to peak related features, and signal related features. The formal definitions of the used features are given in <xref ref-type="supplementary-material" rid="SM1">Annex 1</xref>. Some of these features were previously used by other authors to calculate features related to the ERP signal (Kalatzis et al., <xref ref-type="bibr" rid="B25">2004</xref>; Abootalebi et al., <xref ref-type="bibr" rid="B1">2009</xref>). Four peaks for the P300 paradigms (N100, P200, N200, and P300) and two peaks for the MMN paradigm (N200, P300) were considered (see <xref ref-type="fig" rid="F2">Figure 2</xref>). Consequently, the number of features extracted for classification purposes was 726 (282 features for auditory P300 paradigm, 282 for visual P300 paradigm, and 162 for MMN paradigm).</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>Principal components of P300 tasks (N100, P200, N200, P300) and MMN task (P200, P300).</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fninf-16-893788-g0002.tif"/>
</fig>
<sec>
<title>Peak Related Features</title>
<p>Peaks were estimated using the same algorithm described in Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>). Four intervals were established around the average latency of the respective peak, measured on the grand averages. The algorithm considered <italic>Amplitude</italic> as the largest absolute value in each interval and <italic>Latency</italic> as the time where the peak appears in the respective time window (interval). The other features were: <italic>Absolute Amplitude, Latency/Amplitude ratio, Absolute Latency/Amplitude ratio, Average Absolute Signal Slope</italic>, and <italic>Slope sign alterations</italic>.</p></sec>
<sec>
<title>Peak to Peak Related Features</title>
<p>Three features were calculated considering the relationship between adjacent selected peaks: the absolute difference between the amplitude of the peak and the next peak in latency order; the difference in latencies of these two peaks; and the slope of the signal in this time window.</p></sec>
<sec>
<title>Signal Related Features</title>
<p>Features considering the area under the curve were calculated: the sum of the positive signal values (<italic>Positive Area</italic>); the sum of the negative signal values (<italic>Negative Area</italic>); the <italic>Total Area</italic>, and <italic>Absolute Total Area</italic>. Two more features related to the whole signal were calculated: the number of times that the amplitude value of the signal crosses the zero y-axis between two adjacent peaks (<italic>Zero Crossing</italic>); and the relation of the number of crosses per time interval (<italic>Zero Cross Density</italic>).</p>
<p>Additionally, frequency domain features were extracted using a Power Spectral Density (PSD) analysis: the frequency with the largest energy content in the signal spectrum (<italic>Mode frequency</italic>) spectrum; the frequency that separates the power spectrum into two equal energy areas (<italic>Median frequency</italic>); and an estimate of the central tendency of the derivate power distributions (<italic>Mean frequency</italic>).</p></sec></sec>
<sec>
<title>Feature Scaling</title>
<p>Mapping the feature values of a dataset into the same range is crucial for those algorithms that exploit distances or similarities (Ahsan et al., <xref ref-type="bibr" rid="B2">2021</xref>). The feature values were z-scored, for standardizing them on the same scale by dividing the feature&#x00027;s deviation by the standard deviation in a data set. This improved the numerical stability of the model. Standardization also maintains useful information about outliers and makes the algorithm less sensitive to them (Sahu et al., <xref ref-type="bibr" rid="B48">2020</xref>). The standardized values were then normalized, rescaling them all to values between 0 and 1 using the sigmoid transformation function.</p></sec>
<sec>
<title>Feature Selection</title>
<p>After the feature scaling process, feature selection was applied. This is useful for constructing the smallest subset of features from the original set maintaining as much as possible the original meaning of the data. This technique of dimensionality reduction removes redundant and irrelevant features. The main purpose of this process is to reduce the training time and amount of memory required for the algorithm to work, thus reducing the computational cost when developing a predictive model (Zebari et al., <xref ref-type="bibr" rid="B62">2020</xref>). In some cases, it also improves the performance of the model, although this is not always guaranteed (Benouini et al., <xref ref-type="bibr" rid="B6">2020</xref>).</p>
<p>There are several methods available for performing feature selection in the setting of random forest classification (Speiser et al., <xref ref-type="bibr" rid="B54">2019</xref>). RFs are a collection of classification and regression trees, which are simple models using binary splits on predictor variables to determine outcome predictions. Thus, they provide variable importance measures to rank predictors according to their predictive power.</p>
<sec>
<title>Boruta Algorithm</title>
<p>Boruta is a feature selection algorithm that uses a wrapper method based on the RF classifier to measure the importance of variables. RF makes it relatively fast due to its simple heuristic feature selection procedure (Kursa, <xref ref-type="bibr" rid="B29">2017</xref>).</p>
<p>In the Boruta algorithm, the original feature set is extended by adding shadow variables (Kursa and Rudnicki, <xref ref-type="bibr" rid="B31">2010</xref>). A shadow variable is created by shuffling the values of the original feature. Several RFs are run. In each run, the set of predictor variables is doubled by adding a copy of each variable. An RF is trained on the extended data set to obtain the variable importance values. For each real variable, a statistical test is performed comparing its importance with the maximum value of all the shadow variables. If a variable systematically falls below the shadow ones, its contribution to the model is doubtful and is therefore eliminated. The shadow variables are removed, and the process continues until all variables are accepted, rejected, or a limit number of iterations is reached in which case some variables may be left undecided. This limit corresponds to the maximal number of RF runs.</p>
<p>In this work, we made use of the R package &#x0201C;Boruta&#x0201D; (Kursa and Rudnicki, <xref ref-type="bibr" rid="B30">2020</xref>), and set the number of maximum RF to 500.</p></sec></sec>
<sec>
<title>MKL Classifier Algorithm</title>
<p>Kernel-based SVM employs a kernel <bold><italic>k</italic></bold>(<bold><italic>x</italic></bold><sub><bold><italic>i</italic></bold></sub><bold>,<italic>x</italic></bold><sub><bold><italic>j</italic></bold></sub>) as a function of the similarity between two instances <bold><italic>x</italic></bold><sub><bold><italic>i</italic></bold></sub> and <bold><italic>x</italic></bold><sub><bold><italic>j</italic></bold></sub>. Given a binary classification and <bold><italic>N</italic> </bold>labeled training instances (<bold><italic>x</italic></bold><sub><bold><italic>i</italic></bold></sub><bold>,<italic>y</italic></bold><sub><bold><italic>i</italic></bold></sub>) (<bold><italic>y</italic></bold><sub><bold><italic>i</italic></bold></sub><bold>&#x003F5;<italic>&#x000B1;1</italic></bold>) a result of training an SVM is learning the weights (&#x003B1;<sub><italic>i</italic></sub>) in the decision function:</p>
<disp-formula id="E1"><label>(1)</label><mml:math id="M1"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle mathvariant="bold-italic"><mml:mtext>f</mml:mtext></mml:mstyle><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mtext>sign</mml:mtext></mml:mstyle><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>i</mml:mtext></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>N</mml:mtext></mml:mstyle></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>&#x003B1;</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>i</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>y</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>i</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mtext>k</mml:mtext></mml:mstyle><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>i</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>&#x0002B;</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mtext>b</mml:mtext></mml:mstyle></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>The three commonly used kernels are: linear kernel <bold>(<italic>K</italic></bold><sub><bold><italic>L</italic></bold></sub>), polynomial kernel <bold>(<italic>K</italic></bold><sub><bold><italic>P</italic></bold></sub>), and Gaussian kernel (<bold><italic>K</italic></bold><sub><italic>G</italic></sub>):</p>
<disp-formula id="E2"><label>(2)</label><mml:math id="M2"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>K</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>L</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>i</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>j</mml:mtext></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>i</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>x</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>j</mml:mtext></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E3"><label>(3)</label><mml:math id="M3"><mml:mrow><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>K</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>P</mml:mi></mml:mstyle></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:msub><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02329;</mml:mo><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:msub><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:msub><mml:mo>&#x0232A;</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>q</mml:mi></mml:mstyle></mml:msup></mml:mrow></mml:math></disp-formula>
<disp-formula id="E4"><label>(4)</label><mml:math id="M4"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>K</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>G</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mo class="qopname">exp</mml:mo><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>-</mml:mo></mml:mstyle><mml:mfrac><mml:mrow><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>-</mml:mo></mml:mstyle><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>|</mml:mo><mml:msup><mml:mrow><mml:mo>|</mml:mo></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mn>2</mml:mn></mml:mstyle></mml:mrow></mml:msup></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>s</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mn>2</mml:mn></mml:mstyle></mml:mrow></mml:msup></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo></mml:mstyle><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>with parameter <bold><italic>q</italic> </bold>the polynomial degree and parameter <bold><italic>s</italic> </bold>determine the width for Gaussian distribution.</p>
<p>Multiple kernel learning can be a linear or nonlinear combination of <bold><italic>M</italic> </bold>sub-kernel functions <bold>(</bold><inline-formula><mml:math id="M5"><mml:mrow><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>k</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:msubsup><mml:mo>,</mml:mo><mml:mo>&#x000A0;</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02026;</mml:mo><mml:mo>&#x000A0;</mml:mo><mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>k</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>M</mml:mi></mml:mstyle></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>M</mml:mi></mml:mstyle></mml:msubsup><mml:mo>,</mml:mo><mml:mo>&#x000A0;</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:math></inline-formula>, where <inline-formula><mml:math id="M6"><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:msubsup><mml:mrow><mml:mrow><mml:mo>{</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msubsup></mml:mrow><mml:mo>}</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>M</mml:mi></mml:mstyle></mml:mrow></mml:msubsup></mml:math></inline-formula>, <inline-formula><mml:math id="M7"><mml:msubsup><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msubsup><mml:mstyle mathvariant="bold-italic"><mml:mo>&#x02208;</mml:mo></mml:mstyle><mml:msup><mml:mrow><mml:mi>&#x0211D;</mml:mi></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>D</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow></mml:msup></mml:math></inline-formula>, <bold><italic>D</italic></bold><sub><bold><italic>m</italic></bold></sub> denotes the dimensionality of the <bold><italic>m</italic></bold><sup><bold><italic>th</italic></bold></sup> feature representation. The methods aim to construct an optimal kernel model where the kernel is a linear combination of <bold><italic>M</italic></bold> fixed base kernels. Learning the kernel then consists of learning the weighting coefficients <bold>&#x003B2;<italic>&#x0003D;[&#x003B2;<sub>1</sub></italic></bold>,<bold>&#x003B2;<italic><sub>2</sub></italic>..</bold>,<bold>&#x003B2;<italic><sub>m</sub></italic></bold>] for each base kernel, rather than optimizing the kernel parameters of a single kernel.</p>
<disp-formula id="E5"><label>(5)</label><mml:math id="M8"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>K</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>o</mml:mi><mml:mi>p</mml:mi><mml:mi>t</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>M</mml:mi></mml:mstyle></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003B2;</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>K</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msubsup><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo></mml:mstyle><mml:msubsup><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mtext>&#x000A0;&#x000A0;&#x000A0;</mml:mtext></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003B2;</mml:mi></mml:mstyle><mml:mtext>&#x000A0;</mml:mtext><mml:mstyle mathvariant="bold-italic"><mml:mo>&#x0003E;</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>0</mml:mn></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>M</mml:mi></mml:mstyle></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003B2;</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Plugged into the SVM decision function leads to the following decision function:</p>
<disp-formula id="E6"><label>(6)</label><mml:math id="M9"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle mathvariant="bold-italic"><mml:mi>f</mml:mi></mml:mstyle><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>s</mml:mi><mml:mi>i</mml:mi><mml:mi>g</mml:mi><mml:mi>n</mml:mi></mml:mstyle><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>N</mml:mi></mml:mstyle></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003B1;</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>y</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mo>=</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mn>1</mml:mn></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>M</mml:mi></mml:mstyle></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003B2;</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mi>k</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>j</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>i</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mstyle mathvariant="bold-italic"><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mo>&#x0002B;</mml:mo></mml:mstyle><mml:mstyle mathvariant="bold-italic"><mml:mi>b</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>There are several MKL algorithms. We used MKL available in the SHOGUN toolbox (Sonnenburg et al., <xref ref-type="bibr" rid="B53">2010</xref>). In this implementation, the kernel functions and corresponding kernel parameters are known before training, thus, only the parameters used to combine the set of kernel functions are optimized during training. The MKL learning method can help to find which kernel or combination of kernels corresponds to a better notion of similarity for the same representation of data. Nevertheless, by using inputs coming from different representations (that have different measures of similarities corresponding to different kernels), and combining them, the learning methods can find the best kernel for data representation or the combination that includes the discriminative information the data could carry. These approaches depend on the regularization chosen for the restrictions on the kernel&#x00027;s weights. Regularized L1-norm induces sparsity on the kernel&#x00027;s coefficients obtained with a considerably large fraction of zero entries focusing on the best kernels. Using the L2-norm the solution will be non-sparse, distributing the weights over all kernels (Kloft et al., <xref ref-type="bibr" rid="B26">2011</xref>). Additionally, it has been demonstrated that the L2 MKL yields better performance on most of the benchmark data sets (Yu et al., <xref ref-type="bibr" rid="B61">2010</xref>).</p>
<p>The two-step training method used here updates the combination function and the base learner parameters in an alternating manner. The algorithm was then based on wrapping linear programs around SVMs. The outer loop optimization is related to the Semi-Infinite Linear Program (SILP) that optimizes the non-smooth dual problem formulated by Sonnenburg et al. (<xref ref-type="bibr" rid="B51">2005</xref>, <xref ref-type="bibr" rid="B52">2006</xref>). In this approach, the optimization target function follows the structural risk minimization framework and tries to minimize the sum of a regularization term that corresponds to the model complexity and an error term that correspond to the system performance. The optimization problem modeled as a SILP problem has lower computational complexity compared to those modeled with a semidefinite programming (SDP) problem and a quadratically constrained quadratic programming (QCQP) used in one-step methods.</p>
<p>In this work, the input data was mapped into different feature spaces trying to group variables with common aspects or sources: type of paradigm (P300a, P330v, MMN), channels (Fz, Cz, Pz), or type of feature (Latency &#x00026; Amplitude, Morphological, Frequency) as shown in <xref ref-type="fig" rid="F3">Figure 3</xref>. The 726 features were rearranged into three sources of data considering the common aspects. Thus, three different views of the data can be used to create three models to be compared. The experiments explored combinations of three kernels (one per source of data). For example, in the case of Channels the criteria used were grouping all the features from the global feature set that belong to the Cz channel and feeding a kernel to look for a notion of similarity and do the same with the other two kernels for Fz and Pz channels. The kernels were iteratively selected from a grid search of linear, polynomial, and RBF kernels with different parameters. We used a non-sparse MKL with L2-norm for thoroughly combining complementary information of the heterogeneous data sources.</p>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p>Grouping input data (726 features) in three possible kernel combinations according to the feature space approach.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fninf-16-893788-g0003.tif"/>
</fig></sec>
<sec>
<title>Nested Cross-Validation</title>
<p>To explore the feature selection impact, nested cross-validation (NCV) was applied. The NCV is characterized by having an inner loop responsible for model selection/hyperparameter tuning and an outer loop is for error estimation. The entire data was divided randomly into <bold><italic>k</italic> </bold>subsets or folds with stratification, the same proportion of patients and controls as in the complete dataset. The <bold><italic>k-1</italic> </bold>subsets are used for feature selection and the remaining subset for testing the model after feature selection. As in the <italic>k</italic>-fold cross-validation method, this process was repeated <bold><italic>k</italic> </bold>times (outer loop), each time leaving out one of the subsets reserved for testing and the rest for feature selection using the Boruta algorithm in an inner loop (see <xref ref-type="fig" rid="F4">Figure 4</xref>).</p>
<fig id="F4" position="float">
<label>Figure 4</label>
<caption><p>Feature selection steps applying nested cross-validation.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fninf-16-893788-g0004.tif"/>
</fig>
<p>Each subset obtained after feature selection was used for model hyperparameter tuning in the inner loop. One of the approaches commonly used in practice for the selection of hyperparameters is to try several combinations of them and evaluate theirs out of sample performance. The tuned parameters in the MKL classifier were:</p>
<list list-type="bullet">
<list-item><p>Regularization parameter <bold><italic>C</italic></bold>, we evaluated C with {0.5, 1, 1.5, 5, 10}, and selected the best value considering a tradeoff between misclassification and model simplicity</p></list-item>
<list-item><p>Type of kernel (linear, RBF, and polynomial)</p></list-item>
<list-item><p>In the case of RBF kernels the Sigma (<bold>&#x003C3;</bold>), we explored the following values 10, 5, 1, 0.25, 0.5, 0.75, to determine the width for Gaussian distribution.</p></list-item>
</list>
<p>The parameters configuration selected to train the final model was the one that reached the highest average accuracy on the inner loop. The whole dataset used for tuning parameters was then trained and tested with its corresponding test set in the outer loop. The classifiers&#x00027; performance was obtained by averaging the accuracy of the <bold><italic>k</italic> </bold>trained models.</p></sec></sec>
<sec sec-type="results" id="s3">
<title>Results</title>
<sec>
<title>Feature Selection</title>
<p>The Boruta algorithms yielded an average of 32 (in a range of 26&#x02013;42) attributes selected per <bold><italic>k</italic> </bold>iteration in a 10-fold cross-validation (see <xref ref-type="fig" rid="F5">Figure 5A</xref>). The median computation time was around 2.6 min (std 0.04), with 0.005 min per RF run. A total of 76 attributes were selected at least in one CV iteration. <xref ref-type="fig" rid="F5">Figure 5B</xref> shows the number of attributes that were selected in <italic>n</italic> of the 10 CV iterations. The distribution of variables per paradigm is also shown. About 80% of the 76 attributes selected were related to amplitude, latency, or the correlation between them. Attributes related to the frequency domain were rarely selected. Only seven features were identified as important every time the Boruta algorithm was used. <xref ref-type="table" rid="T3">Table 3</xref> describes these seven features according to the paradigm, type of stimulus, channel, and type of feature.</p>
<fig id="F5" position="float">
<label>Figure 5</label>
<caption><p>Distribution of feature selection in 10 fold cross-validation. <bold>(A)</bold> Amount of attributes selected per k iteration of the 10 fold CV and the distribution per paradigm in the 10 subsets of features selected, <bold>(B)</bold> Frequency of selection of all the attributes that were selected at least once in the ten Boruta applications. The bottom number means how many features were selected at least in <italic>n</italic> CV iterations (n on top).</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fninf-16-893788-g0005.tif"/>
</fig>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Features selected by the Boruta feature selection method.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Paradigm</bold></th>
<th valign="top" align="left"><bold>Stimulus</bold></th>
<th valign="top" align="left"><bold>Channel</bold></th>
<th valign="top" align="left"><bold>Peak</bold></th>
<th valign="top" align="left"><bold>Feature</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">P300v</td>
<td valign="top" align="left">Target</td>
<td valign="top" align="left">Pz</td>
<td valign="top" align="left">P2</td>
<td valign="top" align="left">Latency</td>
</tr>
<tr>
<td valign="top" align="left">P300a</td>
<td valign="top" align="left">Distractor</td>
<td valign="top" align="left">Cz</td>
<td valign="top" align="left">N1</td>
<td valign="top" align="left">absRatio</td>
</tr>
<tr>
<td valign="top" align="left">P300a</td>
<td valign="top" align="left">Distractor</td>
<td valign="top" align="left">Fz</td>
<td valign="top" align="left">P2</td>
<td valign="top" align="left">absRatio</td>
</tr>
<tr>
<td valign="top" align="left">P300a</td>
<td valign="top" align="left">Distractor</td>
<td valign="top" align="left">Fz</td>
<td valign="top" align="left">P2</td>
<td valign="top" align="left">absAmplitude</td>
</tr>
<tr>
<td valign="top" align="left">P300a</td>
<td valign="top" align="left">Target</td>
<td valign="top" align="left">Cz</td>
<td valign="top" align="left">N1</td>
<td valign="top" align="left">absRatio</td>
</tr>
<tr>
<td valign="top" align="left">P300a</td>
<td valign="top" align="left">Target</td>
<td valign="top" align="left">Cz</td>
<td valign="top" align="left">N2</td>
<td valign="top" align="left">Latency</td>
</tr>
<tr>
<td valign="top" align="left">P300a</td>
<td valign="top" align="left">Target</td>
<td valign="top" align="left">Cz</td>
<td valign="top" align="left">P2</td>
<td valign="top" align="left">Latency</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec>
<title>Classifier Performance</title>
<p>To compare the performance of the MKL algorithms, four metrics derived from the confusion matrix were used, namely accuracy (<italic>Acc</italic>), area under a receiver operating characteristic (ROC) curve (<italic>Auc</italic>), sensibility (<italic>Sen</italic>) which evaluates true positive rates, and specificity (<italic>Spe</italic>) to evaluate the false positive rate (Kohl, <xref ref-type="bibr" rid="B27">2012</xref>). The performances of the MKL classifier, with and without feature selection are summarized in <xref ref-type="table" rid="T4">Table 4</xref>.</p>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Performance of MKL algorithm with and without feature selection.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>MKL Kernels</bold></th>
<th valign="top" align="center" style="border-bottom: thin solid #000000;" colspan="4"><bold>Without FS</bold></th>
<th valign="top" align="center" style="border-bottom: thin solid #000000;" colspan="4"><bold>With FS</bold></th>
</tr>
<tr>
<th/>
<th valign="top" align="center"><bold>ACC(%)</bold></th>
<th valign="top" align="center"><bold>SEN(%)</bold></th>
<th valign="top" align="center"><bold>SPE(%)</bold></th>
<th valign="top" align="center"><bold>AUC</bold></th>
<th valign="top" align="center"><bold>ACC(%)</bold></th>
<th valign="top" align="center"><bold>SEN(%)</bold></th>
<th valign="top" align="center"><bold>SPE(%)</bold></th>
<th valign="top" align="center"><bold>AUC</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Paradigm</td>
<td valign="top" align="center">83</td>
<td valign="top" align="center">80</td>
<td valign="top" align="center">88</td>
<td valign="top" align="center">0.88</td>
<td valign="top" align="center">86</td>
<td valign="top" align="center">86</td>
<td valign="top" align="center">87</td>
<td valign="top" align="center">0.92</td>
</tr>
<tr>
<td valign="top" align="left">Channels</td>
<td valign="top" align="center">80</td>
<td valign="top" align="center">74</td>
<td valign="top" align="center">87</td>
<td valign="top" align="center">0.82</td>
<td valign="top" align="center">84</td>
<td valign="top" align="center">85</td>
<td valign="top" align="center">86</td>
<td valign="top" align="center">0.91</td>
</tr>
<tr>
<td valign="top" align="left">Type of Features</td>
<td valign="top" align="center">82</td>
<td valign="top" align="center">78</td>
<td valign="top" align="center">85</td>
<td valign="top" align="center">0.87</td>
<td valign="top" align="center">86</td>
<td valign="top" align="center">86</td>
<td valign="top" align="center">86</td>
<td valign="top" align="center">0.92</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec></sec>
<sec sec-type="discussion" id="s4">
<title>Discussion</title>
<p>In this work, we explored the use of MKL classification for distinguishing SZs from HCs based on ERP data. Using all features, the best classification accuracy (83%) was achieved when kernels were built by grouping features according to paradigms. Moreover, when MKL was combined with the Boruta features selection method, a classification accuracy of 86% was obtained. With this feature selection algorithm, the large number of predictor variables was reduced significantly (96%) with a lower computation time. Therefore, the training time of MKL was also reduced [0.18 (<italic>std</italic> 0.03) seconds per inner cross-validation loop], thus solving one of its main shortcomings: its high computational cost, especially when many features are used (de Carvalho, <xref ref-type="bibr" rid="B12">2019</xref>).</p>
<p>The feature selection algorithm results showed that the variables that contributed most to the discrimination were related to the auditory P300 paradigm. This corresponds with the general finding that auditory P300 measures are more effective in differentiating SZs from HCs than those obtained from the visual stimuli (Park et al., <xref ref-type="bibr" rid="B46">2005</xref>). The selected features were peak related, mainly related to amplitude, latency, and their combination. To a lesser extent, peak to peak related features were included in the selection. However, only three Signal related features were occasionally included. Thus, features from the frequency domain did not contribute much to the improvement of the classification.</p>
<p>Our results are in line with prior works. A previous study (Santos-Mayo et al., <xref ref-type="bibr" rid="B49">2017</xref>) proposed a system to help diagnose schizophrenia by analyzing P300 signals during an auditory oddball task. The authors extracted time and frequency domain features similar to ours but using different collections of signals from electrodes in different regions of the scalp. Our results are comparable to theirs when the electrode groupings were used, but they obtained larger AUC values (more than 0.95) for their Left and Right hemisphere electrode groupings. We did not explore these locations. However, their dataset was unbalanced and small, which possibly limits the reliability of their findings. Other authors using also P300 amplitude and latency values as features (Shim et al., <xref ref-type="bibr" rid="B50">2016</xref>) reported classification accuracies of 81% using an SVM classifier. When they combined these features with a selection of source-level density measures they increased their accuracy to 88%, a result similar to ours. Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>) also extracted latency and amplitude features of responses to three different odd-ball tasks and applied several classification algorithms. They achieved an average accuracy of 77% (std = 3.5). Their best result (about 85%), corresponded to an RF classifier, comparable to our results. Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>) also found that in a ranking of the 20 main variables contributing to the classification, 14 were extracted from the P300 auditory oddball paradigm. This suggests that, out of the three ERP paradigms used, the auditory P300 contributes most to the classification which is congruent with our findings.</p>
<p>One limitation of our study is that we did not use the spatial distributions of the ERPs over the scalp. Further research should include features using ERP scalp topographic maps (STM). This would take advantage of the differences in STM between schizophrenia and normal control groups reported by different authors (Morstyn et al., <xref ref-type="bibr" rid="B41">1983</xref>; Frantseva et al., <xref ref-type="bibr" rid="B15">2014</xref>). This is a pure image processing approach. Another track is to use independent component analysis (ICA) to split up the multi-channel ERP data into several independent spatiotemporal components. ICA separates the mixed signals into unmixed signals which are statistically independent. These approaches could generate features for a classifier. Another limitation of the present study is the small sample size which is usual in psychiatric cohorts from one site. We addressed this limitation using cross-validation strategies. However, training with larger data sets (possibly from multiple sites) would yield a more stable and reliable estimate of future performance and guarantee better generalization.</p></sec>
<sec sec-type="conclusions" id="s5">
<title>Conclusion</title>
<p>Using Multiple Kernel Learning (MKL) classifiers on features defined for ERP obtained in oddball paradigms, it was possible to distinguish SZs from HCs with a classification accuracy up to 86%. Accuracy improved when the Boruta feature selection was applied. The auditory P300 provided the most informative features. Future work should explore new ERP features including topographic information.</p></sec>
<sec sec-type="data-availability" id="s6">
<title>Data Availability Statement</title>
<p>The data analyzed in this study is subject to third party restrictions, which were used under license for this study. Requests to access these datasets should be directed to Laton et al. (<xref ref-type="bibr" rid="B33">2014</xref>).</p></sec>
<sec id="s7">
<title>Ethics Statement</title>
<p>The provenance of data in the study (which involved human participants), and the adherence to adequate ethical standards, were reviewed and their use approved by Ethics Committee of Cuban Neuroscience Center. The patients/participants provided their written informed consent to participate in this study, as stated in the original article where they were described.</p></sec>
<sec id="s8">
<title>Author Contributions</title>
<p>ES wrote and edited the manuscript, developed the theory, and carried out the experiments and the interpretation of the results. MO reviewed the manuscript. MV critically revised the manuscript. HS conceived the present idea and supervised it. All the authors discussed and agreed with the main focus and ideas of this article and contributed to the methodology and analysis.</p></sec>
<sec sec-type="funding-information" id="s9">
<title>Funding</title>
<p>This work was supported by the VLIR-UOS project A Cuban National School of Neurotechnology for Cognitive Aging (NSNCA), Grant number CU2017TEA436A103.</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of Interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p></sec>
<sec sec-type="disclaimer" id="s10">
<title>Publisher&#x00027;s Note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p></sec>
</body>
<back>
<ack><p>The authors would like to thank teams of Cuban Neuroscience Center and the Department of Electronics and Informatics (ETRO) of Vrije Universiteit Brussel (VUB) for supporting this research project.</p>
</ack><sec sec-type="supplementary-material" id="s11">
<title>Supplementary Material</title>
<p>The Supplementary Material for this article can be found online at: <ext-link ext-link-type="uri" xlink:href="https://www.frontiersin.org/articles/10.3389/fninf.2022.893788/full#supplementary-material">https://www.frontiersin.org/articles/10.3389/fninf.2022.893788/full#supplementary-material</ext-link></p>
<supplementary-material xlink:href="Table_1.pdf" id="SM1" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink"/></sec>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Abootalebi</surname> <given-names>V.</given-names></name> <name><surname>Moradi</surname> <given-names>M. H.</given-names></name> <name><surname>Khalilzadeh</surname> <given-names>M. A.</given-names></name></person-group> (<year>2009</year>). <article-title>A new approach for EEG feature extraction in P300-based lie detection</article-title>. <source>Comput. Methods Programs Biomed.</source> <volume>94</volume>, <fpage>48</fpage>&#x02013;<lpage>57</lpage>. <pub-id pub-id-type="doi">10.1016/j.cmpb.2008.10.001</pub-id><pub-id pub-id-type="pmid">19041154</pub-id></citation></ref>
<ref id="B2">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Ahsan</surname> <given-names>M. M.</given-names></name> <name><surname>Mahmud</surname> <given-names>M. A. P.</given-names></name> <name><surname>Saha</surname> <given-names>P. K.</given-names></name> <name><surname>Gupta</surname> <given-names>K. D.</given-names></name> <name><surname>Siddique</surname> <given-names>Z.</given-names></name></person-group> (<year>2021</year>). <article-title>Effect of data scaling methods on machine learning algorithms and model performance</article-title>. <source>Technologies</source>. 9, 52. <pub-id pub-id-type="doi">10.3390/technologies9030052</pub-id></citation>
</ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Atkinson</surname> <given-names>R. J.</given-names></name> <name><surname>Michie</surname> <given-names>P. T.</given-names></name> <name><surname>Schall</surname> <given-names>U.</given-names></name></person-group> (<year>2012</year>). <article-title>Duration mismatch negativity and P3a in first-episode psychosis and individuals at ultra-high risk of psychosis</article-title>. <source>Biol. Psychiatry</source>. <volume>71</volume>, <fpage>98</fpage>&#x02013;<lpage>104</lpage>. <pub-id pub-id-type="doi">10.1016/j.biopsych.2011.08.023</pub-id><pub-id pub-id-type="pmid">22000060</pub-id></citation></ref>
<ref id="B4">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barros</surname> <given-names>C.</given-names></name> <name><surname>Silva</surname> <given-names>C. A.</given-names></name> <name><surname>Pinheiro</surname> <given-names>A. P.</given-names></name></person-group> (<year>2021</year>). <article-title>Advanced EEG-based learning approaches to predict schizophrenia: Promises and pitfalls</article-title>. <source>Artif. Intell. Med.</source> <volume>114</volume>, <fpage>102039</fpage>. <pub-id pub-id-type="doi">10.1016/j.artmed.2021.102039</pub-id><pub-id pub-id-type="pmid">33875158</pub-id></citation></ref>
<ref id="B5">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>B&#x000E9;nar</surname> <given-names>C. G.</given-names></name> <name><surname>Sch&#x000F6;n</surname> <given-names>D.</given-names></name> <name><surname>Grimault</surname> <given-names>S.</given-names></name> <name><surname>Nazarian</surname> <given-names>B.</given-names></name> <name><surname>Burle</surname> <given-names>B.</given-names></name> <name><surname>Roth</surname> <given-names>M.</given-names></name></person-group> (<year>2007</year>). <article-title>Single-trial analysis of oddball event-related potentials in simultaneous EEG-fMRI</article-title>. <source>Hum. Brain Mapp.</source> <volume>28</volume>, <fpage>602</fpage>&#x02013;<lpage>613</lpage>. <pub-id pub-id-type="doi">10.1002/hbm.20289</pub-id><pub-id pub-id-type="pmid">17295312</pub-id></citation></ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Benouini</surname> <given-names>R.</given-names></name> <name><surname>Batioua</surname> <given-names>I.</given-names></name> <name><surname>Ezghari</surname> <given-names>S.</given-names></name> <name><surname>Zenkouar</surname> <given-names>K.</given-names></name> <name><surname>Zahi</surname> <given-names>A.</given-names></name></person-group> (<year>2020</year>). <article-title>Fast feature selection algorithm for neighborhood rough set model based on Bucket and Trie structures</article-title>. <source>Granul. Comput.</source> <volume>5</volume>, <fpage>329</fpage>&#x02013;<lpage>347</lpage>. <pub-id pub-id-type="doi">10.1007/s41066-019-00162-w</pub-id></citation>
</ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bramon</surname> <given-names>E.</given-names></name> <name><surname>Rabe-Hesketh</surname> <given-names>S.</given-names></name> <name><surname>Sham</surname> <given-names>P.</given-names></name> <name><surname>Murray</surname> <given-names>R. M.</given-names></name> <name><surname>Frangou</surname> <given-names>S.</given-names></name></person-group> (<year>2004</year>). <article-title>Meta-analysis of the P300 and P50 waveforms in schizophrenia</article-title>. <source>Schizophr. Res.</source> <volume>70</volume>, <fpage>315</fpage>&#x02013;<lpage>329</lpage>. <pub-id pub-id-type="doi">10.1016/j.schres.2004.01.004</pub-id><pub-id pub-id-type="pmid">15329307</pub-id></citation></ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Burgos</surname> <given-names>N.</given-names></name> <name><surname>Colliot</surname> <given-names>O.</given-names></name></person-group> (<year>2020</year>). <article-title>Machine learning for classification and prediction of brain diseases: recent advances and upcoming challenges</article-title>. <source>Curr. Opin. Neurol.</source> <volume>33</volume>, <fpage>439</fpage>&#x02013;<lpage>450</lpage>. <pub-id pub-id-type="doi">10.1097/WCO.0000000000000838</pub-id><pub-id pub-id-type="pmid">32657885</pub-id></citation></ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Castro</surname> <given-names>E.</given-names></name> <name><surname>G&#x000F3;mez-Verdejo</surname> <given-names>V.</given-names></name> <name><surname>Mart&#x000ED;nez-Ram&#x000F3;n</surname> <given-names>M.</given-names></name> <name><surname>Kiehl</surname> <given-names>K. A.</given-names></name> <name><surname>Calhoun</surname> <given-names>V. D.</given-names></name></person-group> (<year>2014</year>). <article-title>A multiple kernel learning approach to perform classification of groups from complex-valued fMRI data analysis: application to schizophrenia</article-title>. <source>Neuroimage</source> <volume>87</volume>, <fpage>1</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2013.10.065</pub-id><pub-id pub-id-type="pmid">24225489</pub-id></citation></ref>
<ref id="B10">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Cong</surname> <given-names>F.</given-names></name> <name><surname>Ristaniemi</surname> <given-names>T.</given-names></name> <name><surname>Lyytinen</surname> <given-names>H.</given-names></name></person-group> (<year>2015</year>). <article-title>&#x0201C;Advanced signal processing on brain event-related potentials: Filtering ERPs in time,&#x0201D;</article-title> in <source>Frequency and Space Domains Sequentially and Simultaneously, Vol. 13</source>. World Scientific Publishing Co. <pub-id pub-id-type="doi">10.1142/9306</pub-id></citation>
</ref>
<ref id="B11">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Coulter</surname> <given-names>C.</given-names></name> <name><surname>Baker</surname> <given-names>K. K.</given-names></name> <name><surname>Margolis</surname> <given-names>R. L.</given-names></name></person-group> (<year>2019</year>). <article-title>Specialized consultation for suspected recent-onset schizophrenia: Diagnostic clarity and the distorting impact of anxiety and reported auditory hallucinations</article-title>. <source>J. Psychiatr. Pract.</source> <volume>25</volume>, <fpage>76</fpage>&#x02013;<lpage>81</lpage>. <pub-id pub-id-type="doi">10.1097/PRA.0000000000000363</pub-id><pub-id pub-id-type="pmid">30849055</pub-id></citation></ref>
<ref id="B12">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>de Carvalho</surname> <given-names>J. A. A. L</given-names></name></person-group> (<year>2019</year>). Is Multiple Kernel Learning Better than Other Classifier Methods? Available online at: <ext-link ext-link-type="uri" xlink:href="https://repositorio-aberto.up.pt/bitstream/10216/126534/2/387808.pdf">https://repositorio-aberto.up.pt/bitstream/10216/126534/2/387808.pdf</ext-link></citation>
</ref>
<ref id="B13">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Dimitriadis</surname> <given-names>S. I</given-names></name></person-group> (<year>2019</year>). <article-title>Multiplexity and graph signal processing of EEG dynamic functional connectivity networks as connectomic biomarkers for schizophrenia patients: a whole brain breakdown</article-title>. <source>bioRxiv</source> (New York, NY: Cold Spring Harbor Laboratory)</citation>
</ref>
<ref id="B14">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Earls</surname> <given-names>H. A.</given-names></name> <name><surname>Curran</surname> <given-names>T.</given-names></name> <name><surname>Mittal</surname> <given-names>V.</given-names></name></person-group> (<year>2016</year>). <article-title>A meta-analytic review of auditory event-related potential components as endophenotypes for schizophrenia: perspectives from first-degree relatives</article-title>. <source>Schizophr. Bull.</source> <volume>42</volume>, <fpage>1504</fpage>&#x02013;<lpage>1516</lpage>. <pub-id pub-id-type="doi">10.1093/schbul/sbw047</pub-id><pub-id pub-id-type="pmid">27217271</pub-id></citation></ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Frantseva</surname> <given-names>M.</given-names></name> <name><surname>Cui</surname> <given-names>J.</given-names></name> <name><surname>Farzan</surname> <given-names>F.</given-names></name> <name><surname>Chinta</surname> <given-names>L. V.</given-names></name> <name><surname>Perez Velazquez</surname> <given-names>J. L.</given-names></name> <name><surname>Daskalakis</surname> <given-names>Z. J.</given-names></name></person-group> (<year>2014</year>). <article-title>Disrupted cortical conductivity in schizophrenia: TMS-EEG study</article-title>. <source>Cereb. Cortex</source> <volume>24</volume>, <fpage>211</fpage>&#x02013;<lpage>221</lpage>. <pub-id pub-id-type="doi">10.1093/cercor/bhs304</pub-id><pub-id pub-id-type="pmid">23042743</pub-id></citation></ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Graber</surname> <given-names>K.</given-names></name> <name><surname>Bosquet Enlow</surname> <given-names>M.</given-names></name> <name><surname>Duffy</surname> <given-names>F. H.</given-names></name> <name><surname>D&#x00027;Angelo</surname> <given-names>E.</given-names></name> <name><surname>Sideridis</surname> <given-names>G.</given-names></name> <name><surname>Hyde</surname> <given-names>D. E.</given-names></name></person-group> (<year>2019</year>). <article-title>P300 amplitude attenuation in high risk and early onset psychosis youth</article-title>. <source>Schizophr. Res.</source> <volume>210</volume>, <fpage>228</fpage>&#x02013;<lpage>238</lpage>. <pub-id pub-id-type="doi">10.1016/j.schres.2018.12.029</pub-id><pub-id pub-id-type="pmid">30685392</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hirayasu</surname> <given-names>Y.</given-names></name> <name><surname>Potts</surname> <given-names>G. F.</given-names></name> <name><surname>O&#x00027;Donnell</surname> <given-names>B. F.</given-names></name> <name><surname>Kwon</surname> <given-names>J. S.</given-names></name> <name><surname>Arakaki</surname> <given-names>H.</given-names></name> <name><surname>Akdag</surname> <given-names>S. J.</given-names></name></person-group> (<year>1998</year>). <article-title>Auditory mismatch negativity in schizophrenia: topographic evaluation with a high-density recording montage</article-title>. <source>Am. J. Psychiatry</source>. <volume>155</volume>, <fpage>1281</fpage>&#x02013;<lpage>1284</lpage>. <pub-id pub-id-type="doi">10.1176/ajp.155.9.1281</pub-id><pub-id pub-id-type="pmid">9734556</pub-id></citation></ref>
<ref id="B18">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Horvath</surname> <given-names>A.</given-names></name> <name><surname>Szucs</surname> <given-names>A.</given-names></name> <name><surname>Csukly</surname> <given-names>G.</given-names></name> <name><surname>Sakovics</surname> <given-names>A.</given-names></name> <name><surname>Stefanics</surname> <given-names>G.</given-names></name> <name><surname>Kamondi</surname> <given-names>A.</given-names></name></person-group> (<year>2018</year>). <article-title>EEG and ERP biomarkers of Alzheimer&#x00027;s disease: a critical review</article-title>. <source>Front. Biosci. Landmark</source> <volume>23</volume>, <fpage>183</fpage>&#x02013;<lpage>220</lpage>. <pub-id pub-id-type="doi">10.2741/4587</pub-id><pub-id pub-id-type="pmid">28930543</pub-id></citation></ref>
<ref id="B19">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Huang</surname> <given-names>W.-J.</given-names></name> <name><surname>Chen</surname> <given-names>W.-W.</given-names></name> <name><surname>Zhang</surname> <given-names>X.</given-names></name></person-group> (<year>2015</year>). <article-title>The neurophysiology of P 300&#x02014;an integrated review</article-title>. <source>Eur. Rev. Med. Pharmacol. Sci.</source> <volume>19</volume>, <fpage>1480</fpage>&#x02013;<lpage>1488</lpage>.</citation>
</ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Iwabuchi</surname> <given-names>S. J.</given-names></name> <name><surname>Palaniyappan</surname> <given-names>L.</given-names></name></person-group> (<year>2017</year>). <article-title>Abnormalities in the effective connectivity of visuothalamic circuitry in schizophrenia</article-title>. <source>Psychol. Med.</source> <volume>47</volume>, <fpage>1300</fpage>&#x02013;<lpage>1310</lpage>. <pub-id pub-id-type="doi">10.1017/S0033291716003469</pub-id><pub-id pub-id-type="pmid">28077184</pub-id></citation></ref>
<ref id="B21">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Iyer</surname> <given-names>D.</given-names></name> <name><surname>Boutros</surname> <given-names>N. N.</given-names></name> <name><surname>Zouridakis</surname> <given-names>G.</given-names></name></person-group> (<year>2012</year>). <article-title>Clinical Neurophysiology Single-trial analysis of auditory evoked potentials improves separation of normal and schizophrenia subjects</article-title>. <source>Clin. Neurophysiol.</source> <volume>123</volume>, <fpage>1810</fpage>&#x02013;<lpage>1820</lpage>. <pub-id pub-id-type="doi">10.1016/j.clinph.2011.12.021</pub-id><pub-id pub-id-type="pmid">22356936</pub-id></citation></ref>
<ref id="B22">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Javitt</surname> <given-names>D. C.</given-names></name> <name><surname>Doneshka</surname> <given-names>P.</given-names></name> <name><surname>Zylberman</surname> <given-names>I.</given-names></name> <name><surname>Ritter</surname> <given-names>W.</given-names></name> <name><surname>Vaughan</surname> <given-names>H. G.</given-names></name></person-group> (<year>1993</year>). <article-title>Impairment of early cortical processing in schizophrenia: An event-related potential confirmation study</article-title>. <source>Biol. Psychiatry</source>. <volume>33</volume>, <fpage>513</fpage>&#x02013;<lpage>519</lpage>. <pub-id pub-id-type="doi">10.1016/0006-3223(93)90005-X</pub-id><pub-id pub-id-type="pmid">8513035</pub-id></citation></ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Javitt</surname> <given-names>D. C.</given-names></name> <name><surname>Lee</surname> <given-names>M.</given-names></name> <name><surname>Kantrowitz</surname> <given-names>J. T.</given-names></name> <name><surname>Martinez</surname> <given-names>A.</given-names></name></person-group> (<year>2017</year>). <article-title>Mismatch negativity as a biomarker of theta band oscillatory dysfunction in schizophrenia</article-title>. <source>Schizophr. Res.</source> <volume>191</volume>:<fpage>51</fpage>&#x02013;<lpage>60</lpage> <pub-id pub-id-type="doi">10.1016/j.schres.2017.06.023</pub-id><pub-id pub-id-type="pmid">28666633</pub-id></citation></ref>
<ref id="B24">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Joshi</surname> <given-names>V.</given-names></name> <name><surname>Nanavati</surname> <given-names>N.</given-names></name></person-group> (<year>2021</year>). <article-title>A review of EEG signal analysis for diagnosis of neurological disorders using machine learning</article-title>. <source>J. Biomed. Photonics Eng.</source> <volume>7</volume>, <fpage>1</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.18287/10.18287/JBPE21.07.040201</pub-id></citation>
</ref>
<ref id="B25">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kalatzis</surname> <given-names>I.</given-names></name> <name><surname>Piliouras</surname> <given-names>N.</given-names></name> <name><surname>Ventouras</surname> <given-names>E.</given-names></name> <name><surname>Papageorgiou</surname> <given-names>C. C.</given-names></name> <name><surname>Rabavilas</surname> <given-names>A. D.</given-names></name> <name><surname>Cavouras</surname> <given-names>D.</given-names></name></person-group> (<year>2004</year>). <article-title>Design and implementation of an SVM-based computer classification system for discriminating depressive patients from healthy controls using the P600 component of ERP signals</article-title>. <source>Comput. Methods Programs Biomed.</source> <volume>75</volume>, <fpage>11</fpage>&#x02013;<lpage>22</lpage>. <pub-id pub-id-type="doi">10.1016/j.cmpb.2003.09.003</pub-id><pub-id pub-id-type="pmid">15158043</pub-id></citation></ref>
<ref id="B26">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kloft</surname> <given-names>M.</given-names></name> <name><surname>Brefeld</surname> <given-names>U.</given-names></name> <name><surname>Sonnenburg</surname> <given-names>S.</given-names></name> <name><surname>Zien</surname> <given-names>A.</given-names></name></person-group> (<year>2011</year>). <article-title>&#x02113;p-norm multiple kernel learning</article-title>. <source>J. Mach. Learn. Res.</source> <volume>12</volume>, <fpage>953</fpage>&#x02013;<lpage>997</lpage>.</citation>
</ref>
<ref id="B27">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kohl</surname> <given-names>M</given-names></name></person-group> (<year>2012</year>). <article-title>Performance measures in binary classification</article-title>. <source>Int. J. Stat. Med. Res.</source> <volume>49</volume>, <fpage>79</fpage>&#x02013;<lpage>81</lpage>. <pub-id pub-id-type="doi">10.6000/1929-6029.2012.01.01.08</pub-id></citation>
</ref>
<ref id="B28">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Koukkou</surname> <given-names>M.</given-names></name> <name><surname>Koenig</surname> <given-names>T.</given-names></name> <name><surname>Banninger</surname> <given-names>A.</given-names></name> <name><surname>Rieger</surname> <given-names>K.</given-names></name> <name><surname>Hern&#x000E1;ndez</surname> <given-names>L. D.</given-names></name> <name><surname>Higuchi</surname> <given-names>Y.</given-names></name></person-group> (<year>2018</year>). <article-title>&#x0201C;Neurobiology of schizophrenia: electrophysiological indices,&#x0201D;</article-title> in <source>Advances in Psychiatry</source> (<publisher-loc>Springer, Cham</publisher-loc>), <fpage>433</fpage>&#x02013;<lpage>459</lpage>.<pub-id pub-id-type="pmid">33613338</pub-id></citation></ref>
<ref id="B29">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Kursa</surname> <given-names>M. B</given-names></name></person-group> (<year>2017</year>). <article-title>Efficient all relevant feature selection with random ferns</article-title>. <source>Lect. Notes Comput. Sci.</source> (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics<italic>) 10352 LNAI</italic>, 302&#x02013;311. <pub-id pub-id-type="doi">10.1007/978-3-319-60438-1_30</pub-id></citation>
</ref>
<ref id="B30">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Kursa</surname> <given-names>M. B.</given-names></name> <name><surname>Rudnicki</surname> <given-names>W. R.</given-names></name></person-group> (<year>2020</year>). <source>Package &#x02018;Boruta&#x00027;, 1&#x02013;17</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://cran.microsoft.com/snapshot/2020-04-16/web/packages/Boruta/Boruta.pdf">https://cran.microsoft.com/snapshot/2020-04-16/web/packages/Boruta/Boruta.pdf</ext-link></citation>
</ref>
<ref id="B31">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kursa</surname> <given-names>M. B.</given-names></name> <name><surname>Rudnicki</surname> <given-names>W. R.</given-names></name></person-group> (<year>2010</year>). <article-title>Feature selection with the boruta package</article-title>. <source>J. Stat. Softw.</source> <volume>36</volume>, <fpage>1</fpage>&#x02013;<lpage>13</lpage>. <pub-id pub-id-type="doi">10.18637/jss.v036.i11</pub-id></citation>
</ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lai</surname> <given-names>J. W.</given-names></name> <name><surname>Ang</surname> <given-names>C. K. E.</given-names></name> <name><surname>Rajendra Acharya</surname> <given-names>U.</given-names></name> <name><surname>Cheong</surname> <given-names>K. H.</given-names></name></person-group> (<year>2021</year>). <article-title>Schizophrenia: A survey of artificial intelligence techniques applied to detection and classification</article-title>. <source>Int. J. Environ. Res. Public Health</source> <volume>18</volume>, <fpage>1</fpage>&#x02013;<lpage>20</lpage>. <pub-id pub-id-type="doi">10.3390/ijerph18116099</pub-id><pub-id pub-id-type="pmid">34198829</pub-id></citation></ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Laton</surname> <given-names>J.</given-names></name> <name><surname>Van Schependom</surname> <given-names>J.</given-names></name> <name><surname>Gielen</surname> <given-names>J.</given-names></name> <name><surname>Decoster</surname> <given-names>J.</given-names></name> <name><surname>Moons</surname> <given-names>T.</given-names></name> <name><surname>De Keyser</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>Single-subject classification of schizophrenia patients based on a combination of oddball and mismatch evoked potential paradigms</article-title>. <source>J. Neurol. Sci.</source> <volume>347</volume>, <fpage>262</fpage>&#x02013;<lpage>267</lpage>. <pub-id pub-id-type="doi">10.1016/j.jns.2014.10.015</pub-id><pub-id pub-id-type="pmid">25454645</pub-id></citation></ref>
<ref id="B34">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>M.</given-names></name> <name><surname>Sehatpour</surname> <given-names>P.</given-names></name> <name><surname>Hoptman</surname> <given-names>M. J.</given-names></name> <name><surname>Lakatos</surname> <given-names>P.</given-names></name> <name><surname>Dias</surname> <given-names>E. C.</given-names></name> <name><surname>Kantrowitz</surname> <given-names>J. T.</given-names></name></person-group> (<year>2017</year>). <article-title>Neural mechanisms of mismatch negativity dysfunction in schizophrenia</article-title>. <source>Mol. Psychiatry</source> <volume>22</volume>, <fpage>1585</fpage>&#x02013;<lpage>1593</lpage>. <pub-id pub-id-type="doi">10.1038/mp.2017.3</pub-id><pub-id pub-id-type="pmid">28167837</pub-id></citation></ref>
<ref id="B35">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Li</surname> <given-names>F.</given-names></name> <name><surname>Wang</surname> <given-names>J.</given-names></name> <name><surname>Jiang</surname> <given-names>Y.</given-names></name> <name><surname>Si</surname> <given-names>Y.</given-names></name> <name><surname>Peng</surname> <given-names>W.</given-names></name> <name><surname>Song</surname> <given-names>L.</given-names></name></person-group> (<year>2018</year>). <article-title>Top-down disconnectivity in schizophrenia during P300 tasks</article-title>. <source>Front. Comput. Neurosci.</source> <volume>12</volume>, <fpage>1</fpage>&#x02013;<lpage>10</lpage>. <pub-id pub-id-type="doi">10.3389/fncom.2018.00033</pub-id><pub-id pub-id-type="pmid">29875646</pub-id></citation></ref>
<ref id="B36">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Li</surname> <given-names>X.</given-names></name> <name><surname>Chen</surname> <given-names>X.</given-names></name> <name><surname>Yan</surname> <given-names>Y.</given-names></name> <name><surname>Wei</surname> <given-names>W.</given-names></name> <name><surname>Wang</surname> <given-names>Z. J.</given-names></name></person-group> (<year>2014</year>). <article-title>Classification of EEG signals using a multiple kernel learning support vector machine</article-title>. <source>Sensors (Switzerland)</source> <volume>14</volume>, <fpage>12784</fpage>&#x02013;<lpage>12802</lpage>. <pub-id pub-id-type="doi">10.3390/s140712784</pub-id><pub-id pub-id-type="pmid">25036334</pub-id></citation></ref>
<ref id="B37">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lo</surname> <given-names>A.</given-names></name> <name><surname>Chernoff</surname> <given-names>H.</given-names></name> <name><surname>Zheng</surname> <given-names>T.</given-names></name> <name><surname>Lo</surname> <given-names>S. H.</given-names></name></person-group> (<year>2015</year>). <article-title>Why significant variables aren&#x00027;t automatically good predictors</article-title>. <source>Proc. Natl. Acad. Sci. U. S. A.</source> <volume>112</volume>, <fpage>13892</fpage>&#x02013;<lpage>13897</lpage>. <pub-id pub-id-type="doi">10.1073/pnas.1518285112</pub-id><pub-id pub-id-type="pmid">26504198</pub-id></citation></ref>
<ref id="B38">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Loo</surname> <given-names>S. K.</given-names></name> <name><surname>Lenartowicz</surname> <given-names>A.</given-names></name> <name><surname>Makeig</surname> <given-names>S.</given-names></name></person-group> (<year>2016</year>). <article-title>Research Review: Use of EEG biomarkers in child psychiatry research - Current state and future directions</article-title>. <source>J. Child Psychol. Psychiatry Allied Discip.</source> <volume>57</volume>, <fpage>4</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.1111/jcpp.12435</pub-id><pub-id pub-id-type="pmid">26099166</pub-id></citation></ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mait&#x000ED;n</surname> <given-names>A. M.</given-names></name> <name><surname>Garc&#x000ED;a-Tejedor</surname> <given-names>A. J.</given-names></name> <name><surname>Mu&#x000F1;oz</surname> <given-names>J. P. R.</given-names></name></person-group> (<year>2020</year>). <article-title>Machine learning approaches for detecting parkinson&#x00027;s disease from eeg analysis: A systematic review</article-title>. <source>Appl. Sci.</source> <volume>10</volume>, <fpage>1</fpage>&#x02013;<lpage>21</lpage>. <pub-id pub-id-type="doi">10.3390/app10238662</pub-id></citation>
</ref>
<ref id="B40">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>McGrath</surname> <given-names>J.</given-names></name> <name><surname>Saha</surname> <given-names>S.</given-names></name> <name><surname>Welham</surname> <given-names>J.</given-names></name> <name><surname>El Saadi</surname> <given-names>O.</given-names></name> <name><surname>MacCauley</surname> <given-names>C.</given-names></name> <name><surname>Chant</surname> <given-names>D.</given-names></name></person-group> (<year>2004</year>). <article-title>A systematic review of the incidence of schizophrenia: the distribution of rates and the influence of sex, urbanicity, migrant status and methodology</article-title>. <source>BMC Med.</source> <volume>2</volume>, <fpage>1</fpage>&#x02013;<lpage>22</lpage>. <pub-id pub-id-type="doi">10.1186/1741-7015-2-13</pub-id><pub-id pub-id-type="pmid">15115547</pub-id></citation></ref>
<ref id="B41">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morstyn</surname> <given-names>R.</given-names></name> <name><surname>Duffy</surname> <given-names>F. H.</given-names></name> <name><surname>Mccarley</surname> <given-names>R. W.</given-names></name></person-group> (<year>1983</year>). <article-title>Altered P300 topography in schizophrenia</article-title>. <source>Arch. Gen. Psychiatry</source> <volume>40</volume>, <fpage>729</fpage>&#x02013;<lpage>734</lpage>. <pub-id pub-id-type="doi">10.1001/archpsyc.1983.01790060027003</pub-id><pub-id pub-id-type="pmid">6860074</pub-id></citation></ref>
<ref id="B42">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>N&#x000E4;&#x000E4;t&#x000E4;nen</surname> <given-names>R.</given-names></name> <name><surname>Pakarinen</surname> <given-names>S.</given-names></name> <name><surname>Rinne</surname> <given-names>T.</given-names></name> <name><surname>Takegata</surname> <given-names>R.</given-names></name></person-group> (<year>2004</year>). <article-title>The mismatch negativity (MMN): towards the optimal paradigm</article-title>. <source>Clin. Neurophysiol.</source> <volume>115</volume>, <fpage>140</fpage>&#x02013;<lpage>144</lpage>. <pub-id pub-id-type="doi">10.1016/j.clinph.2003.04.001</pub-id><pub-id pub-id-type="pmid">14706481</pub-id></citation></ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Neuhaus</surname> <given-names>A. H.</given-names></name> <name><surname>Popescu</surname> <given-names>F. C.</given-names></name> <name><surname>Bates</surname> <given-names>J. A.</given-names></name> <name><surname>Goldberg</surname> <given-names>T. E.</given-names></name> <name><surname>Malhotra</surname> <given-names>A. K.</given-names></name></person-group> (<year>2013</year>). <article-title>Single-subject classification of schizophrenia using event-related potentials obtained during auditory and visual oddball paradigms</article-title>. <source>Eur. Arch. Psychiatry Clin. Neurosci.</source> <volume>263</volume>, <fpage>241</fpage>&#x02013;<lpage>247</lpage>. <pub-id pub-id-type="doi">10.1007/s00406-012-0326-7</pub-id><pub-id pub-id-type="pmid">22584805</pub-id></citation></ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Neuhaus</surname> <given-names>A. H.</given-names></name> <name><surname>Popescu</surname> <given-names>F. C.</given-names></name> <name><surname>Rentzsch</surname> <given-names>J.</given-names></name> <name><surname>Gallinat</surname> <given-names>J.</given-names></name></person-group> (<year>2014</year>). <article-title>Critical evaluation of auditory event-related potential deficits in schizophrenia: evidence from large-scale single-subject pattern classification</article-title>. <source>Schizophr. Bull.</source> <volume>40</volume>, <fpage>1062</fpage>&#x02013;<lpage>1071</lpage>. <pub-id pub-id-type="doi">10.1093/schbul/sbt151</pub-id><pub-id pub-id-type="pmid">24150041</pub-id></citation></ref>
<ref id="B45">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Olbrich</surname> <given-names>S.</given-names></name> <name><surname>Van Dinteren</surname> <given-names>R.</given-names></name> <name><surname>Arns</surname> <given-names>M.</given-names></name></person-group> (<year>2016</year>). <article-title>Personalized medicine: review and perspectives of promising baseline eeg biomarkers in major depressive disorder and attention deficit hyperactivity disorder</article-title>. <source>Neuropsychobiology</source> <volume>72</volume>, <fpage>229</fpage>&#x02013;<lpage>240</lpage>. <pub-id pub-id-type="doi">10.1159/000437435</pub-id><pub-id pub-id-type="pmid">26901357</pub-id></citation></ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Park</surname> <given-names>E. J.</given-names></name> <name><surname>Jin</surname> <given-names>Y. T.</given-names></name> <name><surname>Kang</surname> <given-names>C. Y.</given-names></name> <name><surname>Nam</surname> <given-names>J. H.</given-names></name> <name><surname>Lee</surname> <given-names>Y. H.</given-names></name> <name><surname>Yum</surname> <given-names>M. K.</given-names></name> <etal/></person-group>. (<year>2005</year>). <article-title>Auditory and visual P300 in patients with schizophrenia and controls: stimulus modality effect size differences</article-title>. <source>Clin. Psychopharmacol. Neurosci.</source> <volume>3</volume>, <fpage>22</fpage>&#x02013;<lpage>32</lpage>.</citation>
</ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sabeti</surname> <given-names>M.</given-names></name> <name><surname>Katebi</surname> <given-names>S.</given-names></name> <name><surname>Boostani</surname> <given-names>R.</given-names></name></person-group> (<year>2009</year>). <article-title>Entropy and complexity measures for EEG signal classification of schizophrenic and control participants</article-title>. <source>Artif. Intell. Med.</source> <volume>47</volume>, <fpage>263</fpage>&#x02013;<lpage>274</lpage>. <pub-id pub-id-type="doi">10.1016/j.artmed.2009.03.003</pub-id><pub-id pub-id-type="pmid">19403281</pub-id></citation></ref>
<ref id="B48">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Sahu</surname> <given-names>A.</given-names></name> <name><surname>Mao</surname> <given-names>Z.</given-names></name> <name><surname>Davis</surname> <given-names>K.</given-names></name> <name><surname>Goulart</surname> <given-names>A. E.</given-names></name></person-group> (<year>2020</year>). <article-title>Data Processing and Model Selection for Machine Learning-based Network Intrusion Detection</article-title>. <source>2020 IEEE Int. Work. Tech. Comm. Commun. Qual. Reliab. CQR</source> 2020. <pub-id pub-id-type="doi">10.1109/CQR47547.2020.9101394</pub-id></citation>
</ref>
<ref id="B49">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Santos-Mayo</surname> <given-names>L.</given-names></name> <name><surname>San-Jose-Revuelta</surname> <given-names>L. M.</given-names></name> <name><surname>Arribas</surname> <given-names>J. I.</given-names></name></person-group> (<year>2017</year>). <article-title>A computer-aided diagnosis system with EEG based on the p3b wave during an auditory odd-ball task in schizophrenia</article-title>. <source>IEEE Trans. Biomed. Eng.</source> <volume>64</volume>, <fpage>395</fpage>&#x02013;<lpage>407</lpage>. <pub-id pub-id-type="doi">10.1109/TBME.2016.2558824</pub-id><pub-id pub-id-type="pmid">28113193</pub-id></citation></ref>
<ref id="B50">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shim</surname> <given-names>M.</given-names></name> <name><surname>Hwang</surname> <given-names>H. J.</given-names></name> <name><surname>Kim</surname> <given-names>D. W.</given-names></name> <name><surname>Lee</surname> <given-names>S. H.</given-names></name> <name><surname>Im</surname> <given-names>C. H.</given-names></name></person-group> (<year>2016</year>). <article-title>Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features</article-title>. <source>Schizophr. Res</source>. <volume>176</volume>, <fpage>314</fpage>&#x02013;<lpage>319</lpage>. <pub-id pub-id-type="doi">10.1016/j.schres.2016.05.007</pub-id><pub-id pub-id-type="pmid">27427557</pub-id></citation></ref>
<ref id="B51">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Sonnenburg</surname> <given-names>S.</given-names></name> <name><surname>R&#x000E4;tsch</surname> <given-names>G.</given-names></name> <name><surname>Sch&#x000E4;fer</surname> <given-names>C.</given-names></name></person-group> (<year>2005</year>). <source>A general and efficient multiple kernel learning algorithm.</source> <italic>Adv. Neural Inf. Process. Syst</italic>., 1273&#x02013;1280. Available online at: <ext-link ext-link-type="uri" xlink:href="https://proceedings.neurips.cc/paper/2005/file/b4944963b5c83d545c3d3022bcf03282-Paper.pdf">https://proceedings.neurips.cc/paper/2005/file/b4944963b5c83d545c3d3022bcf03282-Paper.pdf</ext-link></citation>
</ref>
<ref id="B52">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sonnenburg</surname> <given-names>S.</given-names></name> <name><surname>R&#x000E4;tsch</surname> <given-names>G.</given-names></name> <name><surname>Sch&#x000E4;fer</surname> <given-names>C.</given-names></name> <name><surname>Sch&#x000F6;lkop</surname> <given-names>B.</given-names></name></person-group> (<year>2006</year>). <article-title>Large Scale Multiple Kernel Learning</article-title>. <source>J. Mach. Learn. Res.</source> <volume>7</volume>, <fpage>1531</fpage>&#x02013;<lpage>1565</lpage>.<pub-id pub-id-type="pmid">34234824</pub-id></citation></ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sonnenburg</surname> <given-names>S.</given-names></name> <name><surname>R&#x000E4;tsch</surname> <given-names>G.</given-names></name> <name><surname>Henschel</surname> <given-names>S.</given-names></name> <name><surname>Widmer</surname> <given-names>C.</given-names></name> <name><surname>Behr</surname> <given-names>J.</given-names></name> <name><surname>Zien</surname> <given-names>A.</given-names></name></person-group> (<year>2010</year>). <article-title>The Shogun machine learning toolbox</article-title>. <source>J. Mach. Learn. Res.</source> <volume>11</volume>, <fpage>1799</fpage>&#x02013;<lpage>1802</lpage>.</citation>
</ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Speiser</surname> <given-names>J. L.</given-names></name> <name><surname>Miller</surname> <given-names>M. E.</given-names></name> <name><surname>Tooze</surname> <given-names>J.</given-names></name> <name><surname>Ip</surname> <given-names>E.</given-names></name></person-group> (<year>2019</year>). <article-title>A comparison of random forest variable selection methods for classification prediction modeling</article-title>. <source>Expert Syst. Appl.</source> <volume>134</volume>, <fpage>93</fpage>&#x02013;<lpage>101</lpage>. <pub-id pub-id-type="doi">10.1016/j.eswa.2019.05.028</pub-id><pub-id pub-id-type="pmid">32968335</pub-id></citation></ref>
<ref id="B55">
<citation citation-type="journal"><person-group person-group-type="author"><collab>Tanu and Kakkar, D</collab></person-group> (<year>2018</year>). <article-title>A Study on Machine Learning Based Generalized Automated Seizure Detection System</article-title>. <source>Proc. 8th Int. Conf. Conflu. 2018 Cloud Comput. Data Sci. Eng. Conflu.</source> <volume>2018</volume>, <fpage>769</fpage>&#x02013;<lpage>774</lpage>. <pub-id pub-id-type="doi">10.1109/CONFLUENCE.2018.8442438</pub-id><pub-id pub-id-type="pmid">27295638</pub-id></citation></ref>
<ref id="B56">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Turetsky</surname> <given-names>B. I.</given-names></name> <name><surname>Dress</surname> <given-names>E. M.</given-names></name> <name><surname>Braff</surname> <given-names>D. L.</given-names></name> <name><surname>Calkins</surname> <given-names>M. E.</given-names></name> <name><surname>Green</surname> <given-names>M. F.</given-names></name> <name><surname>Greenwood</surname> <given-names>T. A.</given-names></name></person-group> (<year>2015</year>). <article-title>The utility of P300 as a schizophrenia endophenotype and predictive biomarker: Clinical and socio-demographic modulators in COGS-2</article-title>. <source>Schizophr. Res.</source> <volume>163</volume>, <fpage>53</fpage>&#x02013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1016/j.schres.2014.09.024</pub-id><pub-id pub-id-type="pmid">25306203</pub-id></citation></ref>
<ref id="B57">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ula&#x0015F;</surname> <given-names>A.</given-names></name> <name><surname>Castellani</surname> <given-names>U.</given-names></name> <name><surname>Murino</surname> <given-names>V.</given-names></name> <name><surname>Bellani</surname> <given-names>M.</given-names></name> <name><surname>Tansella</surname> <given-names>M.</given-names></name> <name><surname>Brambilla</surname> <given-names>P.</given-names></name></person-group> (<year>2012</year>). <article-title>Biomarker evaluation by multiple kernel learning for schizophrenia detection</article-title>. <source>Proc. - 2012 2nd Int. Work. Pattern Recognit. NeuroImaging, PRNI</source> <volume>2012</volume>, <fpage>89</fpage>&#x02013;<lpage>92</lpage>. <pub-id pub-id-type="doi">10.1109/PRNI.2012.12</pub-id></citation>
</ref>
<ref id="B58">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Wani</surname> <given-names>N.</given-names></name> <name><surname>Raza</surname> <given-names>K.</given-names></name></person-group> (<year>2018</year>). <article-title>&#x0201C;Multiple kernel-learning approach for medical image analysis,&#x0201D;</article-title> in <source>Soft Computing Based Medical Image Analysis</source> (<ext-link ext-link-type="uri" xlink:href="https://www.google.com/search?q=Amsterdam&#x00026;stick=H4sIAAAAAAAAAOPgE-LUz9U3MDJLSYpXYgcxs40LtLSyk63084vSE_MyqxJLMvPzUDhWGamJKYWliUUlqUXFi1g5HXOLgayUxNwdrIy72Jk4GAD1-FzrVgAAAA&#x00026;sa=X&#x00026;ved=2ahUKEwizzv7KqLv4AhWwc98KHeFKAmkQmxMoAXoECF0QAw">Amsterdam</ext-link>, Netherlands: Elsevier Inc.), <fpage>31</fpage>&#x02013;<lpage>47</lpage>. <pub-id pub-id-type="doi">10.1016/B978-0-12-813087-2.00002-6</pub-id></citation>
</ref>
<ref id="B59">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Woodman</surname> <given-names>G. F</given-names></name></person-group> (<year>2010</year>). <article-title>A brief introduction to the use of event-related potentials (ERPs) in studies of perception and attention</article-title>. <source>Atten. Percept. Psychophysiol.</source> <volume>72</volume>, <fpage>1</fpage>&#x02013;<lpage>29</lpage>. <pub-id pub-id-type="doi">10.3758/BF03196680</pub-id></citation>
</ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yoon</surname> <given-names>K.</given-names></name> <name><surname>Kim</surname> <given-names>K.</given-names></name></person-group> (<year>2017</year>). <article-title>Multiple kernel learning based on three discriminant features for a P300 speller BCI</article-title>. <source>Neurocomputing</source>. <volume>237</volume>, <fpage>133</fpage>&#x02013;<lpage>144</lpage>. <pub-id pub-id-type="doi">10.1016/j.neucom.2016.09.053</pub-id></citation>
</ref>
<ref id="B61">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Yu</surname> <given-names>S.</given-names></name> <name><surname>Falck</surname> <given-names>T.</given-names></name> <name><surname>Daemen</surname> <given-names>A.</given-names></name> <name><surname>Tranchevent</surname> <given-names>L. C.</given-names></name> <name><surname>Suykens</surname> <given-names>J. A. K.</given-names></name> <name><surname>De Moor</surname> <given-names>B.</given-names></name> <etal/></person-group>. (<year>2010</year>). <article-title>L2-norm multiple kernel learning and its application to biomedical data fusion</article-title>. <source>BMC Bioinformatics</source>. 11, 309. <pub-id pub-id-type="doi">10.1186/1471-2105-11-309</pub-id><pub-id pub-id-type="pmid">20529363</pub-id></citation></ref>
<ref id="B62">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zebari</surname> <given-names>R.</given-names></name> <name><surname>Abdulazeez</surname> <given-names>A.</given-names></name> <name><surname>Zeebaree</surname> <given-names>D.</given-names></name> <name><surname>Zebari</surname> <given-names>D.</given-names></name> <name><surname>Saeed</surname> <given-names>J.</given-names></name></person-group> (<year>2020</year>). <article-title>A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction</article-title>. <source>J. Appl. Sci. Technol. Trends</source> <volume>1</volume>, <fpage>56</fpage>&#x02013;<lpage>70</lpage>. <pub-id pub-id-type="doi">10.38094/jastt1224</pub-id></citation>
</ref>
</ref-list>
</back>
</article>