A comparison of two sleep spindle detection methods based on all night averages: individually adjusted vs. fixed frequencies

Sleep spindles are frequently studied for their relationship with state and trait cognitive variables, and they are thought to play an important role in sleep-related memory consolidation. Due to their frequent occurrence in NREM sleep, the detection of sleep spindles is only feasible using automatic algorithms, of which a large number is available. We compared subject averages of the spindle parameters computed by a fixed frequency (FixF) (11–13 Hz for slow spindles, 13–15 Hz for fast spindles) automatic detection algorithm and the individual adjustment method (IAM), which uses individual frequency bands for sleep spindle detection. Fast spindle duration and amplitude are strongly correlated in the two algorithms, but there is little overlap in fast spindle density and slow spindle parameters in general. The agreement between fixed and manually determined sleep spindle frequencies is limited, especially in case of slow spindles. This is the most likely reason for the poor agreement between the two detection methods in case of slow spindle parameters. Our results suggest that while various algorithms may reliably detect fast spindles, a more sophisticated algorithm primed to individual spindle frequencies is necessary for the detection of slow spindles as well as individual variations in the number of spindles in general.


INTRODUCTION
Sleep spindles are oscillations emerging from interacting thalamocortical, corticothalamic, and reticular networks in NREM sleep (Steriade and Deschenes, 1984;Amzica and Steriade, 2000;Steriade, 2000;Fogel and Smith, 2011), which are thought to play an important role in sleep-related brain plasticity . Due to their trait-like nature and relationship to plasticity, sleep spindles are frequently studied as candidate indexes of individual variations in cognitive performance. Sleep spindles are remarkably individual features: sleep spindle parameters are characterized by high intra-individual stability and inter-individual variability (De Gennaro et al., 2005), a strong genetic background (De Gennaro et al., 2008), and a correlation with anatomical properties of the brain (Piantoni et al., 2013;Saletin et al., 2013).
Due to their high prevalence and specific signal properties automatic detection methods have proven to be viable and preferable alternatives to visual detection. Some of the earliest studies (Broughton et al., 1978;Campbell et al., 1980) used phaselocked loop devices for automatic sleep spindle detection and already reported an adequate agreement with visual detection.
An early combined software-hardware system (Ferri et al., 1989) also reliably replicated visual spindle detection results. Software solutions for automatic spindle detection were introduced somewhat later (Schimicek et al., 1994) and reported relatively high (approx. 70%) specificity for 90% sensitivity, while an improved method (Devuyst et al., 2006) could increase this to almost 76% in a clinical sample. More recently, sophisticated automatic sleep spindle detection methods using artificial neural networks (Acır and Güzeliş, 2004;Ventouras et al., 2005) and decision trees (Duman et al., 2009) reached even higher performance, with correct classification frequently exceeding 90%.
Automatic sleep spindle recognition was further refined by adapting algorithms that take into account the inter-individual differences in sleep spindle activity, which vastly exceed intraindividual variation (De Gennaro et al., 2005) and emergeamong others-as a function of age and sex (Driver et al., 1996;Carrier et al., 2001;Huupponen et al., 2002;Genzel et al., 2012). Sleep spindle detection methods have been developed to operate with individually adjusted amplitude limits (Huupponen et al., 2000(Huupponen et al., , 2007Ray et al., 2010). A novel algorithm (Bódizs et al., 2009;Ujma et al., 2014) based on the electrophysiological fingerprint theory of human sleep (De Gennaro et al., 2005 is the Individual Adjustment Method (IAM), which takes into account inter-individual variations not only in the amplitude, but also in the frequency of sleep spindles. In the IAM, sleep spindles are therefore not only detected based on individual amplitude thresholds, but also within the exact frequency bands where they are present in a given individual. A similarly adaptive detection method (based on a probabilistic model) is reported in Nonclercq et al. (2013).
A comparison of four different spindle detection methods (Huupponen et al., 2007) reported acceptable, but not overwhelming concordance. A recent study (Warby et al., 2014) investigated the agreement in spindle detection between expert human raters, non-experts recruited in an internet crowdsourcing effort, and automatic detection algorithms. Concordance was strongest among human experts, followed by non-experts operation in a crowdsourcing scheme, and weakest among automatic algorithms.
While the progress in automatic sleep spindle detection methods is impressive, there are numerous concerns which must be addressed in this field. A practical criticism may arise from the fact that automatic sleep spindle detections are frequently validated against visual detections: however, agreement in the visual scoring of spindles is not perfect (Campbell et al., 1980;Warby et al., 2014), the visual detection of spindles is often considered as a consensus from several raters which may bias results (Ray et al., 2010), and-despite stronger agreement among human raters than algorithms (Warby et al., 2014)-the use of human expert opinion as an absolute gold standard is philosophically questionable in itself (Bódizs et al., 2009).
Further criticism must be given to the fact of the use of standard signal detection terminology (such as sensitivity and specificity) in case of sleep spindle detection algorithms. Sleep spindles are frequent phenomena, but even so the vast majority of a sleep EEG recording does not consist of sleep spindles. Therefore, correct negative classifications are by far the most common result produced by any sleep spindle detector, which might drastically inflate specificity. The ratio of correct hits and false detections-including misses and false positives-would be a much more conservative, but also more informative measure of detection performance.
Sleep spindles are not only biological signals, but important markers of individual traits (De Gennaro et al., 2005 as well as powerful correlates of human cognition (among others: Bódizs et al., 2005;Schabus et al., 2006;Fogel et al., 2007;Ujma et al., 2014). Therefore, an alternative option in order to assess detection algorithms would be to investigate how much they can reproduce trait-like individual averages (instead of comparing individual spindle detections).
To our knowledge, it has never been investigated how strongly spindle measures of different detection methods are correlated if not individual spindle detections, but subject averages are considered. This can evidently not predicted from the signal detection characteristics of the comparison of individual spindle detections of various methods-albeit the literature usually reports moderate agreement between the individual spindle detections of different algorithms, it is unknown whether the different spindle samples obtained by different methods approximate the same individual averages. Therefore, the aim of our study was to reveal the correlation between individual sleep spindle parameters calculated with two different detection methods.

SUBJECTS
We examined polysomnographic data of 161 healthy volunteers (88 males, 73 females, age between 17 years and 69 years, mean age 29.4 years, StD 10.7 years) recorded on the second night spent in a sleep laboratory. All procedures were approved by the responsible institution's ethical board and subjects gave informed consent. A semi-structured interview excluded any history of neurologic or psychiatric disease, but six subjects suffered from frequent nightmares. Subjects were free of drugs and prescription medication (except for contraceptives, all data self-reported). Alcohol and excessive caffeine consumption (over two cups of coffee before noon) was not allowed. Eight subjects were smokers, while the rest were non-smokers (self-reported). This dataset used for analysis was the same as in Ujma et al. (2014), except for the inclusion of one female subject who was excluded from the previous study due to her unavailable IQ score.

SLEEP RECORDINGS
All subjects spent two nights in a sleep laboratory and polysomnographic data from the 2nd night was used for analysis. Since the study was performed in cooperation between multiple sleep laboratories, recordings were performed in four slightly different designs.
For 31 subjects, recordings were performed with 18 EEG electrodes using a Flat Style SLEEP La Mont Headbox device with a HBX32-SLP preamplifier (La Mont Medical Inc. USA), with a sampling rate of 249 Hz, hardware prefiltering 0.5-70 Hz and a precision of 12 bit.
For 16 subjects signals were collected, prefiltered (0.33-1500 Hz, 40 dB/decade anti-aliasing hardware input filter), amplified and digitized with 4096 Hz/channel sampling rate (synchronous) and 12 bit resolution by using the 32 channel EEG/polysystem (Brain-Quick BQ 132S, Micromed, Italy). A further 40 dB/decade anti-aliasing digital filter was applied by digital signal processing which low-pass filtered the data at 450 Hz. Finally, the digitized and filtered EEG was undersampled at 1024 Hz.
For 114 subjects, recordings were performed with a Comlab 32 Digital Sleep Lab device (Schwarzer, Germany) with a sampling rate of 250 Hz, hardware prefiltering 0.53-70 Hz and a precision of 8 bit. In 94 of these subjects, 22 EEG electrode sites were used, while in the others 20 subjects 10 EEG electrodes were used. Common recording sites in all subjects which were used in the analysis were Fp1, Fp2, F3, F4, Fz, F7, F8, C3, C4, Cz, P3, P4, T3, T4, T5, T6, O1, and O2, all referred to the mathematically linked mastoids. For the 20 subjects with only 10 electrodes, data from Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1 and O2 was available and for the other electrodes these subjects were treated as missing data.
In order to correct for potentially different baseline amplitudes depending on the recording device (Vasko et al., 1997),

Frontiers in Human Neuroscience
www.frontiersin.org February 2015 | Volume 9 | Article 52 | 2 the analog-digital conversion and filtering characteristics of all recording devices were measured and sleep spindle amplitudes were corrected for the measured differences as follows (Ujma et al., 2014). We determined the amplitude reduction rate of each recording system by calculating the proportion between digital (measured) and analog (generated) amplitudes of sinusoid signals at typical sleep spindle frequencies (10, 11, 12, 13, 14, and 15 Hz) for both inducing (40 and 355 μV amplitude) signals. Machine-specific amplitude reduction rates were given as the mean amplitude rate between digital and analog values at the two amplitudes and six measured frequencies. Sleep spindle amplitudes were corrected by dividing their calculated values by the amplitude reduction rate of the recording system. Given the individual-and derivation-specific adjustment inherent to both the Fixed frequency method (FixF) and the IAM, sleep spindle densities and durations are amplitude-insensitive measures. Thus, there is no need for the compensation of the different recording systems in these values. Sleep recordings of the second nights were scored according to standard criteria (Iber et al., 2007) on a 20 s basis and artifacts were removed by visual inspection on a 4 s basis. Sleep spindle analysis was performed on artifact-free segments of NREM sleep.

Fixed frequency method of sleep spindle analysis
For the FixF method we determined the 11-13 Hz range as a slow spindle frequency band and the 13-15 Hz window as a fast spindle frequency band. These frequencies were selected to ensure consistency with previous studies (Schabus et al., 2006(Schabus et al., , 2007(Schabus et al., , 2008Chatburn et al., 2013), which used a similar approach for the separation of slow and fast spindles.
Sleep spindles were automatically detected within artifact-free NREM sleep periods on every EEG derivation. For slow spindle detection, data were bandpass-filtered between 11 Hz and 13 Hz. The root mean squares of the filtered signals were determined for 0.25 s length time windows. Next a threshold was calculated at the 95th percentile of the root mean square values for every EEG derivation. A spindle was identified when at least two consecutive root mean square time points exceeded the threshold, and the duration criterion (≥0.5 s) was met. Four spindle characteristics were calculated; these were density (number of spindles/min); amplitude (peak-to-peak difference in voltage, expressed in μV); duration (s), and frequency (number of cycles/s, in Hz). The same procedure was followed for detecting fast spindles, using a band pass filter of 13-15 Hz (Schabus et al., 2007;Gruber et al., 2013).

Sleep spindle analysis according to the IAM
The second sleep spindle detection algorithm was the IAM (Bódizs et al., 2009). This sleep spindle detection method takes into account both inter-individual variations and intra-individual consistency in sleep spindle frequency (De Gennaro et al., 2005, analyzing sleep spindles at the individual peak frequency for all subjects. The IAM procedure (Bódizs et al., 2009) consisted of several steps as described below (illustrated on Figure 1).
i. Average amplitude spectra. Non-overlapping 4 s artifact-free NREM sleep EEG segments are Hanning-tapered (50%), then zero-padded to 16 s. Average amplitude spectra of all-night NREM sleep EEG derivations is computed between 9 Hz and 16 Hz by using an FFT routine (frequency resolution: 0.0625 Hz). ii. Individually adjusted frequency limits of slow and fast sleep spindles. Determination of the individual slow and fast sleep spindle frequencies is based on second order derivatives of the 9-16 Hz amplitude spectra. In order to avoid small fluctuations in convex and concave segments average amplitude spectra of 0.0625 Hz resolution (i) is downsampled (decimated) by a factor of 4 (resulting in a resolution of 0.25 Hz) before calculating the derivation-specific second-order derivatives in this frequency range. Derivation-specific second order derivatives of the amplitude spectra are then averaged over all EEG derivations resulting in a whole-scalp second order derivative for each subject. Individual-specific frequency limits of sleep spindles are defined as pairs of zero crossing points encompassing a negative peak in the whole-scalp second order derivatives. These zero-crossing points are rounded to the closest bins within the high-resolution (0.0625 Hz) amplitude spectra obtained in step i. Two pairs of individual-specific frequency limits and corresponding ranges are defined (one for slow and one for fast spindles). In cases of uncertainty (lack of zero crossing points indicating slow spindles or partial overlap between slow and fast sleep spindles in some cases), frequencies with predominance of power in averaged frontal (Fp1, Fp2, F3, F4, Fz, F7, F8) over averaged centro-parietal (C3, C4, Cz, P3, P4) amplitude spectra were considered as slow spindle frequencies (N = 18). There was no case of uncertainty related to the individual-specific frequency boundaries of fast sleep spindles. iii. Individual-specific spindle middle frequencies. Slow spindle middle frequency of a given subject was quantified as the arithmetic mean of the individual-specific lower and upper limits for slow spindling as obtained above (ii). In case of fast sleep spindling the arithmetic mean of the lower and the upper frequency limits of fast sleep spindles were considered. iv. Individual-and derivation-specific amplitude criteria for sleep spindles. Spindles are defined as those EEG segments contributing to the peak region of the average amplitude spectrum. Hence we intended to obtain an amplitude criterion corresponding to the line determined by the y-values (μV) pertaining to the individually adjusted pairs of frequency limits (ii) in the average amplitude spectra (i).
a. The number of high resolution (0.0625 Hz) frequency bins (i) falling in the individual-specific slow-and fast sleep spindle frequency ranges (ii) is determined. b. The amplitude spectral values (i) at the individually adjusted frequency limits for slow and fast sleep spindles (ii) are determined. This is performed in a derivationspecific manner. c. Number of bins for slow and fast sleep spindling (iv/a) are multiplied by the arithmetic mean of the pairs of derivation-specific amplitude spectral values for slow and fast sleep spindle frequency limits (iv/b), respectively. Outcomes are individual-and derivation specific amplitude criteria for slow and fast sleep spindle detections.
v. Envelopes of sleep spindling. EEG data is band-pass filtered for the slow and fast spindle frequency ranges by using an FFT-based Gaussian filter with 16 s windows: f (x) = eˆ(−(x− x m )/(w/2)), where x varies between zero and the Nyquist frequency according to the spectral resolution, x m is the middle frequency of the spindle range (iii), and w is the width of the spindle range (ii) (ii and iii). Filtered signal is rectified and smoothed by a moving average weighted with a Hanning window of 0.1 s length and multiplied with π/2 (the latter is the inverse of the mean of a rectified sine wave). vi. Detection and characterization of sleep spindles. If envelopes of this band-pass filtered and rectified data (v) exceed the individual and derivation-specific threshold as defined above (iv) for at least 0.5 s, a sleep spindle is detected. Sleep spindles detected this way are analyzed and average sleep spindle density (number of spindles per minute), sleep spindle duration (s), as well as median and maximum amplitude (expressed as all-night means of intra-spindle envelopes in μV at the middle of the detected spindles and at the maxima of the spindles, respectively) is calculated for the subject.

STATISTICS
FixF and IAM spindle parameters were compared using pairedsample t-tests (α = 0.05). The Benjamini-Hochberg method of false detection rate correction was performed in order to correct for multiple comparisons.
We computed Pearson's point-moment correlation coefficients between comparable sleep spindle measures (that is, sleep spindle parameters computed from the same electrode) produced by IAM, and the FixF method.

IAM FREQUENCY BANDS
For the IAM method, individual slow spindle lower frequency limits ranged from 8.98 Hz to 12.95 Hz (mean: 10.96 Hz), while higher frequency limits ranged from 10.14 Hz to 13.7 Hz (mean: 11.9 Hz). Slow spindle middle frequencies ranged from 9.59 to 13.28 Hz (mean: 11.43 Hz). Fast spindle lower frequency limits ranged from 11.82 Hz to 14.77 Hz (mean: 13.06 Hz), while higher frequency limits ranged from 13.04 Hz to 16.03 Hz (mean: 14.36 Hz). Fast spindle middle frequencies ranged from 12.49 Hz to 15.38 Hz (mean: 13.71 Hz).
Individual slow spindle frequency bands were on average 0.94 Hz wide (range: 0.34-2.2 Hz). Individual fast spindle frequency bands were on average 1.3 Hz wide (range: 0.84-1.89 Hz). Figure 2 shows the distribution of individual sleep spindle frequencies.

FixF vs. IAM SPINDLE PARAMETERS
IAM provides an approximately twice higher sleep spindle density than the FixF method in case of both slow and fast spindles as well as 1.5-2 times longer sleep spindle durations. Standard deviations of the individual averages of the FixF parameters are much smaller than in case of IAM parameters, even proportionally to the lower mean values.
Sleep spindle parameters are shown in Table 1. It must be noted that while FixF and IAM amplitude measures are displayed and compared, they are not expected to be on the same scale due to the narrower frequency band of IAM and the fact that in the FixF method amplitude was expressed as the mean maximum peak-to-peak voltage difference within a spindle, while in IAM amplitude was defined as the mean maximum of intra-spindle envelopes of the individually band-passed EEG.
The difference between comparable FixF and IAM spindle parameters is significant in all cases at p < 0.0001, and all comparisons remain significant after correction for multiple comparisons.

CORRELATIONS BETWEEN FixF AND IAM SPINDLE PARAMETERS
Despite the differences in the results, individual spindle parameters obtained with the FixF and IAM methods are strongly correlated in case of the amplitude and duration of fast spindles. These correlations are always over 0.5 for amplitude and over 0.4 for duration and they are highest (>0.8 for amplitude, >0.7 for Frontiers in Human Neuroscience www.frontiersin.org February 2015 | Volume 9 | Article 52 | 5 duration) in derivations where fast spindles are most prominent (central and parietal electrodes) as well as in occipital derivations. There is, surprisingly, a negative correlation between fast spindle density calculated by the IAM and the FixF method. There is only a week concordance between FixF and IAM slow spindle parameters. There is no significant FixF-IAM correlation in case of slow spindle density and duration, and only a modest correlation in case of slow spindle amplitude (r < 0.5 except for F3). Table 2 presents the Pearson correlation coefficients depicting the linear relationship between corresponding IAM and FixF spindle parameters on all electrodes.
Given that 1) our sample consisted of several datasets recorded on various EEG devices and 2) the FixF ranges we analyzed-while based on previous literature-did not correspond well to the frequency ranges computed by IAM, we reanalyzed our sample divided in subsamples as well as with different FixF ranges set with slow spindles between 10 Hz and 12.5 Hz and fast spindles between 12.5 Hz and 15 Hz. In both re-analyses, we attempted to replicate our most prominent results, and investigated fast spindle parameters on P4 and slow spindle parameters on F3. F3 was selected over Fz because of the higher availability of this electrode in the sample.

DISCUSSION
While previous studies compared sleep spindle detections between various manual and automatic methods (Huupponen et al., 2007;Warby et al., 2014), to our knowledge no previous study compared individual averages of sleep spindle parameters Frontiers in Human Neuroscience www.frontiersin.org February 2015 | Volume 9 | Article 52 | 7 IAM is tuned to individual spindle frequencies as well as individual and derivation-specific amplitude limits, making it inherently more sensitive as evidenced by higher spindle density and longer duration. FixF, on the other hands, focuses on the upper 5% of the amplitude distribution of filtered EEG signals. While FixF appears to detect "the tips of the icebergs" with this approach, the fast spindles detected by FixF are able to realistically approximate the same fast spindle durations and amplitudes as the IAM. Concordance is much weaker, however, in case of slow spindle amplitude, completely absent in case of slow spindle density and duration, while a very surprising negative correlation between fast spindle densities were found. To explain these findings, some empirical tendencies must be considered.
First, while the 13-15 Hz FixF window for fast spindles was similar to the empirically determined individual frequencies of IAM fast spindles, this was not the case for the 11-13 Hz slow spindle window. Fast spindle middle frequencies were below 13 Hz in only 11.24% of all cases and over 15 Hz in 1.87% of cases, while slow spindle middle frequencies were below 11 Hz in 27.5% of all cases and over 13 Hz in 1.25% of all cases. This poor demarcation of slow spindles in the FixF method might explain why FixF slow spindle parameters correlate more strongly with IAM fast spindle parameters than IAM slow spindle parameters (FixF slow vs. IAM fast correlations on Cz: r density = −0.092 p = 0.275; r duration = 0.547 p < 0.001; r amplitude = 0.603 p < 0.001, with similar tendencies on all electrodes, see Table 2 for correlations with IAM slow spindle parameters). This finding, together with poor agreement on density measures suggests that some FixF slow spindles may actually be classified as fast spindles by the IAM procedure and vice versa, explaining the confusion in both density measures and slow spindle parameters in general. This phenomenon is exemplified by some dissimilar findings in the field. That is, both slow and fast sleep spindle measures correlate with cognitive abilities in cases when the FixF method is used (Schabus et al., 2006(Schabus et al., , 2008, while in case of IAM fast spindles are much more stable correlates of cognitive performance (Bódizs et al., 2005(Bódizs et al., , 2008Ujma et al., 2014). It must be noted that sleep spindles are not stationary sinusoidal processes: they are known to shift frequencies (chirp). Negative spindle chirps (decreasing frequencies) have been reported in humans (Andrillon et al., 2011;Schonwald et al., 2011), while increasing spindle frequencies were reported in rats (Sitnikova et al., 2014). These frequency shifts are not large enough to eclipse the difference between slow and fast spindles (Andrillon et al., 2011) but spectral chirps arising in spindles close to the 13 Hz boundary might be large enough to make them "jump" it and be detected in the opposite category.
Second, the average width of the individual fast spindle frequency band was 1.3 Hz, while in case of slow spindles it was only 0.94 Hz. That is, our results show that individual fast spindle frequency bands rarely fell outside the 13-15 Hz range and they were generally closer to the 2 Hz window of the FixF method than slow spindle frequency bands. The fact the re-analysis with FixF bands resembling the empirically determined individual frequency bands of IAM (10-12.5/12.5-15 Hz, compare with IAM frequency bands on Figure 2) did not significantly improve concordance between the two methods suggests that the differences in individual spindle bandwidth may be even more important for the lack of concordance between the two methods than the mere whereabouts of the frequency limits. This is in line with previous results from an adaptive, probabilistic model (Nonclercq et al., 2013) which reported a similar robustness to the input frequency range.
Based on the above findings we hypothesize that the lack of consistency between FixF and IAM slow spindle parameters is caused by the above factors: IAM slow spindles are determined at a lower and narrower frequency, with a larger distance from fast spindle frequencies in the same subject. The same phenomenon might be speculated to explain the negative correlation between IAM and FixF fast spindle density: in subjects with higher numbers of fast spindles (by IAM definitions) around the 13 Hz cutoff point cross-contamination with slow spindles may have been elevated in the FixF method.
There is little consistency in the sleep spindle detection methods used in previous research literature concerning the relationship between spindles and human cognition. Not all studies about the relationship between sleep spindle parameters and individual differences in psychometric variables separated slow and fast spindles: many analyzed sleep spindles in general or spectral power from a broader frequency band Fogel and Smith, 2006;Fogel et al., 2007;Tucker and Fishbein, 2009;Lustenberger et al., 2012;Gruber et al., 2013). Most studies which specifically analyzed slow and fast spindles and their correlation with psychometric variables used a post-hoc classification of spindles based on their central frequency, usually with 13 Hz as the split point (Schabus et al., 2006(Schabus et al., , 2008Chatburn et al., 2013). Other studies used a slightly different ad-hoc division of sigma power into slow (11.5-12.5 Hz) and fast (13.5-14.5 Hz) sigma bands (Bang et al., 2014). Only a handful of studies relied on individually determined spindle frequencies, either by using the IAM method (Bódizs et al., 2005(Bódizs et al., , 2008Ujma et al., 2014) by computing individual relative sigma power defined as power ± 2 Hz around a single maximal spectral peak relative to the background EEG (Gottselig et al., 2002;Geiger et al., 2011) or by using an adaptive, probabilistic method (Nonclercq et al., 2013).
In sum, our results show that in case of fast spindles, duration and amplitude can be estimated reliably with both fixed and individual frequency methods. Much less consistency can be reached in case of slow spindles, and fixed cutoff frequencies may also lead to a poor separation of slow and fast spindles. Our results suggest that the cutoff frequencies and bandwidths for slow and fast spindles must be selected carefully and individually determined frequency bands should be considered.
It is notable that the concordance between the two methods is generally highest on typical spindle locations (frontal electrodes for slow spindles and centro-parietal electrodes for fast spindles). Concordance is usually lowest on temporal leads, but remains relatively high in occipital leads, in line with the relatively high spindle amplitude on these electrodes reported in the same dataset (Ujma et al., 2014). Lead-specific findings suggest that the lack of concordance between different spindle detection algorithms is especially problematic when non-prominent (e.g., temporal) leads are investigated.
There are limitations of our study that must be mentioned. Medicine (2007) are not met in several subsamples of our study. That is, the analog-to-digital conversion rate is low (8 bits) in the largest subsample (N = 114), while the sampling rates are close to the minimally required values (249 and 250 Hz) in two subsamples. Second, while the study compared methodically well-established methods with previous practical applications in science, it must be acknowledged that IAM and the FixF algorithm operate with different philosophical underpinnings, they are designed to detect different features: the FixF method considers the background-relative amplitude of the filtered signal as the key feature of a spindle event, while IAM looks for an amplitude threshold based on the inflection points of the individual EEG spectrum (IAM and FixF detections, together with visual detections are illustrated on Figure 3). Therefore, a perfect agreement between their results cannot be expected, and in the absence of a "gold standard" the inherent superiority of any method cannot be ascertained.