Abstract
The connectomic analyses of large-scale volumetric electron microscope (EM) images enable the discovery of hidden neural connectivity. While the technologies for neuronal reconstruction of EM images are under rapid progress, the technologies for synapse detection are lagging behind. Here, we propose a method that automatically detects the synapses in the 3D EM images, specifically for the mouse cerebellar molecular layer (CML). The method aims to accurately detect the synapses between the reconstructed neuronal fragments whose types can be identified. It extracts the contacts between the reconstructed neuronal fragments and classifies them as synaptic or non-synaptic with the help of type information and two deep learning artificial intelligences (AIs). The method can also assign the pre- and postsynaptic sides of a synapse and determine excitatory and inhibitory synapse types. The accuracy of this method is estimated to be 0.955 in F1-score for a test volume of CML containing 508 synapses. To demonstrate the usability, we measured the size and number of the synapses in the volume and investigated the subcellular connectivity between the CML neuronal fragments. The basic idea of the method to exploit tissue-specific properties can be extended to other brain regions.
Introduction
Cajal’s neuron doctrine was proven correct by the experiments in the late 1950s to 1960s, which directly observed the synapses with EM (Gray, 1959; Colonnier, 1968). Thanks to the advancement in molecular biology and optics, various methods to observe the synapses such as genetic labeling or immunochemical staining in combination with high-resolution light microscopes (LMs) are widely used (Ippolito and Eroglu, 2010; del Valle Rodríguez et al., 2011). However, the resolution limit of LM and the type specificity of the molecular markers often restrict the use of these technologies. Especially for the connectomics, whose ambition is to map the complete wiring diagram of nervous systems, EM is, presently, the only available solution since all the neurons and synapses in a nerve tissue are homogeneously imaged in EM (Denk and Horstmann, 2004).
A connectome is hypothesized to be the physical substrate of any mental processes of a life (White et al., 1986; Abbott et al., 2020). The connectome of a nematode, C. elegans, still remains the only complete connectome (White et al., 1986). Recently, a fruit fly connectome has become within reach, since a complete fruit fly brain was imaged by EM and semi-automated volumetric reconstruction is being performed (Zheng et al., 2018; Dorkenwald et al., 2020). It is foreseen that a mouse connectome will be the next goal and will become available within next 10 years (Abbott et al., 2020).
For connectomics, image analysis technologies are crucial to reconstruct the neurons and to detect the synapses from the EM image data. Recent advancement in the neuron reconstruction technologies, which are based on the deep learning AI, has rendered automatic reconstruction with small human intervention (Januszewski et al., 2018; Lee et al., 2019). Similar computational technologies have been developed for synapse detection as well (see Table 1 for the references). The advancement of synapse detection technology is lagging behind compared to that of the neuron reconstruction technology chiefly because the study for synapse detection began later.
TABLE 1
| Publication | Resolution (nm) | Animal | Region | Test set size (μm3) | F1-score |
| Kreshuk et al., 2011 | 5 × 5 × 9 | Rat | Somatosensory cortex | 241 | 0.905 |
| Becker et al., 2013 | 5 × 5 × 5 | Rat | Cerebellum | 66 | 0.941 |
| Becker et al., 2013 | 5 × 5 × 5 | Rat | Hippocampus | 58 | 1 |
| Becker et al., 2013 | 6.8 × 6.8 × 6.8 | Rat | Somatosensory cortex | 22 | 1 |
| Kreshuk et al., 2014 | 4.5 × 4.5 × 45 | Mouse | Visual cortex | 970 | 0.904 |
| Plaza et al., 2014 | 10 × 10 × 10 | Drosophila | Optic lobe | 27,000 | 0.785* |
| Roncal et al., 2015 | 6 × 6 × 30 | Mouse | Somatosensory cortex | 114 | 0.815* |
| Dorkenwald et al., 2017 | 10 × 10 × 30 | Mouse | Striatum | 19,200 | 0.854 |
| Dorkenwald et al., 2017 | 9 × 9 × 20 | Zebra finch | Area X | 909,000 | 0.905 |
| Dorkenwald et al., 2017 | 9 × 9 × 21 | Zebrafish larval | Spinal cord | 422,000 | 0.858 |
| Staffler et al., 2017 | 11.2 × 11.2 × 28 | Mouse | Somatosensory cortex | 237 | 0.883 |
| Heinrich et al., 2018 | 4 × 4 × 40 | Drosophila | calyx | 375 | 0.877 |
| Huang et al., 2018 | 10 × 10 × 10 | Drosophila | Optic lobe | 27,000 | 0.83* |
| Xiao et al., 2018 | 2 × 2 × 50 | Mouse | Cortex | 2,325 | 0.892 |
| Parag et al., 2018 | 6 × 6 × 29 | Mouse | Somatosensory cortex | 164 | 0.93* |
| Buhmann et al., 2021 | 4 × 4 × 40 | Drosophila | calyx | Whole region | 0.74 |
| Buhmann et al., 2021 | 4 × 4 × 40 | Drosophila | Lateral horn | Whole region | 0.68 |
| Buhmann et al., 2021 | 16 × 16 × 40 | Mouse | Cerebellum | 320 | 0.94 |
| Park et al., 2022 (this work) | 12 × 12 × 50 | Mouse | Cerebellum | 1,757 | 0.955 |
Accuracy of various synapse detection methods (selected animals).
The accuracies of recent synapse detection methods are shown together with the types of the data and animal species. The * symbol indicates that the value is read from a graph.
We have a practical motivation to develop an automatic synapse detection method since we plan to connectomically study the cerebellum. Although the connectomics concerns the entire brain, the studies on the neural circuits do not necessarily require a complete connectome. The investigations into the microcircuits in “partial connectomes” from diverse brain regions and species yield crucial knowledge’s of the fundamental principles of neural connectivity and function (Takemura et al., 2013; Kim et al., 2014; Ohyama et al., 2015). For the cerebellum, the development of the climbing fiber in the infant mouse cerebellum, a new type of Purkinje cell layer interneuron, and the connectivity between granule cells and Purkinje cells have been studied by 3D EM image analyses (Wilson et al., 2019; Nguyen et al., 2021; Osorno et al., 2021). Nonetheless, the functional circuit mechanisms of the motor control and learning of the cerebellum are still largely in veil. In search of the clues, we plan to investigate a small EM image volume of the CML of a mouse taken by serial block-face scanning EM (SBEM).
For semi-automated neuronal reconstruction, we employed one of the competitive technologies and a proofreading pipeline with paid workers (Kim et al., 2014; Lee et al., 2017). For synapse detection, we require a method that shows a practically applicable level of accuracy (>95%). Various synapse detection methods have been proposed, and their accuracy has increased during the last decade. However, only a few have been developed and tested specifically for the cerebellum, and none of them exceed the accuracy bound (Table 1). Therefore, we decided to develop a novel, fully automated method, which is specialized for the cerebellum and can handle our EM image data whose resolution (12 nm × 12 nm × 50 nm voxel size) is relatively low (Table 1 and Figure 1).
FIGURE 1
Such requirement is difficult to accomplish as seen from the preceding studies. In general, there are several challenges in the connectomic EM image analyses, for both reconstruction of neurons and synapse detection. First, the quality of EM images is not always ideal. The defects in the sample, which occur during tissue preparation or staining, hinder precise image analysis. The image resolution is often compromised over the expenses in time and money that large-scale imaging requires. Second, the neuronal structures have intrinsic biological ambiguity such as thin processes and small synaptic structures. Third, the image analysis technologies often generalize poorly, and they show good performance only for the data on which they are developed and tested. A new technology or extensive fine tuning is needed for new data.
To overcome such challenges, we developed a method that is specialized to the cerebellum because the goal seems to be unreachable with a general solution. It utilizes two deep learning AIs, prior knowledge of the cerebellum, and fine tuning of parameters. The method works for a 3D EM image volume, provided together with the reconstruction of the neuronal fragments and their type information. The contacts between a pair of neuronal fragments are extracted from the reconstruction. The contacts are classified into synaptic or non-synaptic through multiple steps; each of which selects a subset of the contacts from the previous step (Staffler et al., 2017). The first selection is conducted based on the types of the neuronal segments, and only the contacts between the types that can have synapses are chosen. The second and third selections are conducted with the aid of the AIs, which 3-dimensionally evaluate the visual cues of the synapses (Cicek et al., 2016; Lee et al., 2017). The method can also assign the synaptic partners into pre- and postsynaptic sides (Buhmann et al., 2021) and determine the excitatory and inhibitory types.
The method is shown suitable for the cerebellar connectomics research. It is applied to a small test volume to evaluate the accuracy and to showcase the usability. The accuracy is 0.955 in F1-score for the test dataset containing 508 synapses. The size and the density of the cerebellar synapses are inspected. The parallel fibers are shown to innervate the consecutive Purkinje cells along the transverse axis in a random manner. Although this method was designed for the cerebellum, the basic idea of specialization exploiting tissue-specific properties can be extended to other brain regions.
Materials and Methods
Electron Microscope Image and Reconstruction
An adult wild-type mouse was used, and a slice of cerebellar tissue was processed for SBEM following standard protocols (Briggman et al., 2011; Xu et al., 2020). The tissue was conventionally stained with the osmium compound and then infiltrated with epoxy resin. The specimen was cut and imaged roughly along the sagittal axis by a Merlin VP field emission scanning electron microscope (Carl Zeiss) equipped with 3View2 in-chamber ultramicrotome and a backscattered electron detector (Gatan). XY resolution was 12 nm, and nominal thickness was 50 nm. One block-face image was acquired in 2-by-3 tiles with 10∼20% of an overlap between the tiles, each of which is 5,000 by 5,000 pixels. Consecutive 1,000 block faces were imaged.
Each of the 6 stacks of 1,000 images was aligned separately and then merged using Image J and TrakEM2 plugin software (Cardona et al., 2012) together with in-house MATLAB codes. After the registration, the size of image volume is 14,600 × 10,200 × 1,000 voxels, approximately corresponding to a 175 μm × 122 μm × 50 μm physical dimension.
To automate the reconstruction of this large volume, an AI implemented by a modified 3D U-Net was employed (Cicek et al., 2016; Lee et al., 2017) to segment the images into different neurons using the cellular membrane as the boundary. To train the network, eight subvolumes from the entire volume were taken at various locations and sizes as training data. They were manually reconstructed by human experts (advanced paid workers) with specialized software, VAST (Berger et al., 2018). Then, the trained network segments the entire volume to reconstruct the putative neuronal fragments.
Since the AI-aided segmentation contains errors, it was proofread by paid workers to yield the final reconstruction using in-house software with an interactive graphical user interface and a few kinds of background software for work process management (Kim et al., 2014). The proofreading was conducted progressively for one neuronal fragment after another, and each neuronal fragment was represented by one segment after proofreading was done.
The segments of proofread neuronal fragments were saved in a separate volume. The separate volume gradually turns from sparse to dense as the proofreading progresses. We used a snapshot of such volume from a fixed date where 57.6% of the volume is filled with the segments of proofread neuronal fragments. This volume will be called as “completed segmentation” hereafter.
Further details of the animal, sample preparation, SBEM acquisition, alignment, image segmentation, and proofreading for 3D reconstruction will be reported elsewhere.
Synaptic Structures
The mammalian synapses in EM images are characterized by the cloud of presynaptic neurotransmitter vesicles and the postsynaptic density (PSD), which are protein complexes specialized for synaptic transmission (Ziff, 1997). These structures are electron dense and visually prominent as they appear dark in EM images (Figures 1A,B and Supplementary Figure 1). Narrow synaptic clefts are also visible in high-resolution EM micrographs, but they are hardly observable when the image resolution is worse than roughly 10 nm per voxel, which is within the resolution range commonly used for connectomics (Table 1). In the 3D representation of reconstructed neurons, the presynaptic boutons (“b”), which appear as swelling on the axon and the postsynaptic spines (“s”), which appear as short protrusion from the dendritic shaft, are also characteristic (Figure 1C). Most of the synapse detection methods for EM, including human visual inspection and our method, take advantage of these visual cues.
Overview of the Method
The method assumes the EM image volume, the corresponding segmentation of reconstruction, and the type information of each neuronal fragment as the input (Figure 2). The contacts between pairs of neuronal fragments are extracted from the reconstruction. They are classified into synaptic or non-synaptic through 3 steps (Figure 2, bottom row).
FIGURE 2
The first step selects the synaptically “relevant contacts” utilizing the type information. The cerebellar cortex has only a few anatomically distinct types of neuronal fragments, and the connectivity between the types is assumed to be regular. A type of neuronal fragments can have connections only to limited types of partners. A contact is regarded as relevant when it is made between such types (Figure 3). Since the neuronal fragments that make irrelevant contacts are known not to have any connection to each other, the irrelevant contacts can be excluded from the candidates for synaptic contacts. This efficiently reduces search space and helps decrease false positive errors, which is otherwise difficult (see section “Accuracy Evaluation”).
FIGURE 3
For the second and third steps, two 3D U-Nets are separately trained to evaluate the likelihood of each voxel being a synaptic contact (SC) voxel and a vesicle-cloud (VC) voxel based on the visual cues for synapses. They are called as SC-Net and VC-Net, respectively (Supplementary Figure 2). The networks learn from the examples in the training data where the SC voxels and VC voxels are annotated by human experts. The networks mimic what humans do and label the SC and VC voxels with a confident level, or the likelihood (Supplementary Figure 3).
In the second step, “synapse candidate” contacts are selected from the relevant contacts. The synapse candidates are those contacts whose voxels have an SC-likelihood distribution that has a peak at a high value (Figure 4). The final step determines the candidates as the synaptic contacts if a candidate contact has a VC in a close distance. The VC is a segment of connected voxels with high VC likelihood (Figure 5).
FIGURE 4
FIGURE 5
The assignment of the pre- and postsynaptic neuronal fragments is straightforward from the final step, because those containing the VC can be assigned as presynaptic and the other as postsynaptic. One presynaptic bouton can innervate multiple postsynaptic spines (Toni et al., 1999; Federmeier et al., 2002). The excitatory and inhibitory synapse types are determined based on the type information that is associated with the neuronal fragments that form each contact. The synapse type is determined following the excitatory vs. inhibitory nature of the presynaptic side.
The Datasets
Seven subvolumes that were taken at various locations from the entire imaged volume were used for this work (Table 2). Each subvolume is the combination of an EM image volume and the corresponding segmentation volume (Figures 3B,C). Six out of the eight subvolumes, which were used for the training of segmentation AI, were used again for the training of the synapse detection AIs. The segmentations of these subvolumes were completely reconstructed by manual tracing as mentioned above (see section “Electron Microscope Image and Reconstruction”). For the SC-Net, five were used as training sets and one as a validation set. For the VC-Net, four were used as training sets and one as a validation set. The dataset 3 was used for SC-Net validation, and the dataset 4 was used for VC-Net validation. One additional subvolume, which is much larger than the training and validation sets, was prepared as a test set. The segmentation volume of the test set was taken from the completed segmentation volume, in which 57.6% of the volume is reconstructed (Figure 3A) by the semi-automated reconstruction (see section “Electron Microscope Image and Reconstruction”). All of the datasets mostly consist of neuropil, and the soma is only minimally included.
TABLE 2
| Dataset | ID | Size (voxels) | Reconstructed neuronal fragments | Contacts | Relevant contacts | Vesicle clouds | Synapses |
| Train and Validation | 1 | 256 × 256 × 64 | 80 | 340 | N/A | 14 | 13 |
| 2 | 256 × 256 × 64 | 92 | 346 | N/A | 17 | 13 | |
| 3 | 384 × 384 × 96 | 217 | 996 | N/A | 49 | 43 | |
| 4 | 384 × 384 × 96 | 184 | 802 | N/A | 44 | 46 | |
| 7 | 384 × 384 × 96 | 217 | 1,078 | N/A | 39 | 32 | |
| 8 | 512 × 512 × 128 | 184 | 846 | N/A | N/A | 29 | |
| Test | 992 × 992 × 248 | 598 | 5,857 | 1,973 | N/A | 508 |
The basic statistics of the datasets.
The size, numbers of the segments of neuronal fragments, contacts, relevant contacts, vesicle clouds, and ground truth synapses are shown for the datasets.
The Datasets 1, 2, 4, 7, and 8 were used as training sets, and Dataset 3 was used as a validation set for the SC-Net training.
The Datasets 1, 2, 3, and 7 were used as training sets, and Dataset 4 was used as a validation set for the VC-Net.
The fields for irrelevant data are marked as “not applicable (N/A)”.
Type Classification of Neuronal Fragments
The cerebellar molecular layer contains four major types of neurons or neural processes (Figure 3A): Purkinje cell (PC); climbing fiber (CF), which is the axonal projection from the inferior olivary nucleus neurons; parallel fiber (PF), which is a part of the axon of the cerebellar granule cell; and molecular layer inhibitory interneuron (IN). Since only a part of the neurons or neural processes is in the data, we referred to all these as neuronal fragments for simplicity. The 3D mesh rendering of each neuronal fragment was visually inspected by human experts upon the completion of the proofreading. Human experts can tell the types from the completed segmentation volume (14,600 × 10,200 × 1,000 voxels, 175 μm × 122 μm × 50 μm size) where the large-scale context of the neuronal structure is preserved. PCs have spiny dendrites, PFs are long and straight along the transverse axis, CFs arborize parallelly to PCs, and INs have dendrites that are confined within the CML. The type information determined from the completed segmentation volume was transferred to the neural segments in the subvolumes when applicable (Figures 3A,C).
Contact Extraction
The computations in the “Materials and Methods” section below were performed by custom written MATLAB codes unless otherwise noted. The contacts between a pair of neuronal fragments are extracted from the segmentation volumes of all the datasets (Figure 3D). When there were background voxels due to extracellular space or annotation gaps between neuronal segments in the volume, the segments were dilated until they saturated the volume to ensure that neighboring segments touched each other.
The segmentation volumes are a 3D-labeled image stack, where each voxel is labeled by a numerical ID of a segment (Figure 3C). To extract the contacts from these data, we searched the voxel locations that the segment ID changes values into 6-neighborhood. As the calculation is symmetric, contact voxels are found on both sides of a pair of neuronal fragments, yielding two-voxel thickness. All the contacting voxels between a pair of neuronal segments were grouped by connected component analysis, and each connected component is considered as one contact (Figure 3D). The contacts with 200 or less voxels (roughly 0.03 μm2 or less) were considered as noise and were excluded.
Cell Type Restriction for Relevant Contacts
Only five pairs among the four types of neuronal fragments of the CML are known to have connections, from CF to PC, PF to PC, PF to IN, IN to PC, and IN to IN (Eccles et al., 1967; Kim and Augustine, 2021). The contacts between these five pairs were accepted as the relevant contacts (Figure 3E). The relevant contacts are required during the prediction of the synapses but not during the training of the AIs. The SC-Net and VC-Net are trained only by the EM images with the ground truth labels (see sections “Ground Truth and Data Labeling,” “SC-Net Architecture and Training,” and “VC-Net Architecture and Training”). Therefore, the relevant contacts are not computed for the training and validation sets (Table 2).
Ground Truth and Data Labeling
The synapse structures in the datasets were labeled by human experts. First, the synapses are searched for and identified in the datasets. To find the synapses, two experts visually inspected all the extracted contacts in the training and validation sets and voted for synaptic, non-synaptic, or ambiguous using the visual cues discussed above as the criteria (see section “Synaptic Structures”). For the disagreements, the two experts had a debate on them and then voted again. The tenacious disagreements in the second voting were labeled as ambiguous. For the test set, only the relevant contacts were inspected and then labeled with the same method. The ambiguous cases were not used as the ground truth for AI training and were not included in the accuracy evaluation.
Next, the synaptic structures are actually marked on the data. The extracted contacts, which were consented as synaptic, directly became the SC in most cases (Figure 4A, left panel and Supplementary Figure 3A, left panel). Occasionally, however, the area of the PSD, which is biologically relevant area for synapse, is smaller than the contact. When it is the case, human experts erased the part on the contact that lies outside the PSD in the training and validation sets. The sizes before and after erasion of each contact can be used to assess the overestimation of approximating the synapse size by the contact size. The fraction of the size change, (sizebefore−sizeafter)/sizeafter, is measured for each contact (section “Synapse Size and Density”).
The human experts also searched for and labeled a VC for each SC. The label for a VC is a 3D area inside the perimeter formed by the outermost neurotransmitter vesicles (Supplementary Figure 3B, left panel). We used VAST for erasing of the SC and labeling of the VC. The VCs were not labeled for Dataset 8, since it was used only for SC-Net training (Table 2).
Measurement of Accuracy
The accuracy was measured in a contact-wise manner by comparing the results of the method and the labels by human experts. The contacts in the validation sets and the test set were labeled as synaptic, non-synaptic, or ambiguous from the voting. The method classifies each contact into synaptic and non-synaptic, and it is compared to the ground truth labeling. The experiments were performed on the test set, and we calculated the precision and recall by counting the true-positive (TP), false-positive (FP), and false-negative (FN) cases.
The ambiguous contacts were not included in the calculation of accuracy. Regardless of the prediction on the ambiguous contacts, they were not counted as any of the true positive, false positive, nor false negative.
The accuracy was measured in Fβ -score. The Fβ -score, defined as below, is the generalization of the F1-score. F1-score (Fβ -score for β = 1) is the harmonic mean of precision and recall. Fβ -score gives more weight on recall when β > 1 (penalize false-negative errors more) and gives more weight on precision when β < 1 (penalize false-positive errors more).
The precision-recall (PR) curves are used to decide the optimal parameters and calculate the maximum possible Fβ -scores. In a PR curve, the precision and recall values for a varying, controllable parameter are drawn in a connected scatter-plot.
SC-Net Architecture and Training
Both the SC-Net and VC-Net were implemented and trained using Caffe (Jia et al., 2014) with its Python interface (Lee et al., 2017).
The SC-Net takes an image volume as the input and produces the likelihood map of the voxels being SC as the output. The network architecture was adopted from the 3D U-Net (Ronneberger et al., 2015), and a few details were modified (Supplementary Figure 2). The sizes of the first two kernels were decided in such a way that the anisotropic data (different resolutions in XY and Z) to the input layer become isotropic to the inputs of the next layers. The dropout layers were introduced to avoid overfitting caused by the lack of training data (Srivastava et al., 2014). The number of parameters in each layer was increased from those of the original 3D U-Net to increase the efficiency of the dropout.
The data augmentation was employed to enhance the training data. It was applied on-the-fly as follows. In each training iteration, 3D patches (12 × 44 × 44 voxels) were sampled, and one of the four augmentation operations (flipping, misaligning, gray scaling, and warping) was randomly conducted. Masking was used to handle the class imbalance between SC- and non-SC voxels. The class imbalance hinders the training because any prediction biased toward the majority class would result in high accuracy (Provost and Weiss, 2003). The labeled datasets are extremely imbalanced. For example, one of the training datasets contains 14,155,776 voxels, and only 73,190 voxels are SC (0.5%). Therefore, the SC-Net was trained only with the boundary voxels of reconstructed neurons, masking out all the rest voxels.
The SC-voxels and non-SC voxels on the neuronal boundary are the positive and negative training data, respectively. The loss function is the sigmoid cross-entropy error. The learning rate was initially set to 1 × 10–5 and then multiplied by 0.98 every 6,000 iterations. The training was terminated after 1 million iterations when the error reached an asymptote (Supplementary Figure 3A).
Finding Synapse Candidates Using SC Likelihood
The SC-Net output is the map of voxel-wise likelihood of a voxel being on an SC. The contact-wise likelihood for being synaptic was estimated as follows. The SC-likelihood values (Figure 4A, middle panel) were masked by the relevant contact voxels (Figure 4A, left panel) to collect only the SC-likelihood values for each relevant contact (Figure 4A, right panel). Note that the non-contact voxels can have high SC likelihood because the SC-Net was trained only with the boundary voxels using the mask (Figure 4A, middle panel). The likelihood values for non-contact voxels, no matter they are high or low, are irrelevant.
The distribution of the SC likelihood of the contact voxels reveals the likelihood of the contact being synaptic (Figure 4B). The synaptic contacts would have the distribution that is narrowly peaked at a high likelihood value. To capture such a distribution pattern, we utilized two percentile features. The 95th percentile represents the bias of the distribution toward high values. The range between the 85th and 99th percentiles indicates the width of the peak. Indeed, the synaptic contacts in the training sets tend to exhibit a high 95th percentile and a small 85th- to 99th percentile range (Figure 4C). However, there is a gray zone in the plot, and the boundary between the synapses and non-synapses is ambiguous. A support vector machine (SVM) was recruited to decide the boundary. The three parameters for percentile (95; 85 and 99) were chosen from many experiments to yield the best accuracy (data not shown).
Conventionally, the decision threshold of an SVM is the zero SVM score. The result for the test set shows numbers of false-positive and false-negative errors when the zero threshold is used (Figure 4D). We wanted to prioritize making less false-negative errors to making less false-positive errors, because false-positive errors could be eliminated at later stages, but false-negative errors are lost once they are excluded from the candidates. To this end, the SVM threshold was tuned using the PR curve for the test data (Figure 4E). The SVM threshold −0.2, which gave the maximum F2-score, 0.947, was chosen.
The inspection on the errors revealed a special error mode, which is that the fraction of high SC-likelihood voxels is small because the synaptically relevant part of the contact is much smaller than the entire contact (see section “Ground Truth and Data Labeling”; Supplementary Figure 4). To rescue these errors, we added a voxel-count based rule (number of high-SC-likelihood voxels; NHSCLV), in addition to the voxel fraction-based SVM decision that a contact is synaptic if at least 400 contact voxels have 0.9 or higher SC likelihood. The square boxes in Figure 4C indicate the contacts that transit from negative to positive, before and after applying this rule. Although more false-positive errors are newly introduced than the false negative errors are eliminated, they are intended as the same strategy discussed above, which makes less false-negative errors. Overall, the PR curve for varying SVM threshold is shifted to the right when this rule is applied (Figure 4E).
VC-Net Architecture and Training
In a naive approach, the synapse candidates found by the procedure so far could be considered as the final result of synapse prediction. The SC-Net would implicitly exploit the same visual cues, including the VC, as human experts do, because SC-Net utilizes the context information in the large field of a view. However, to further improve the accuracy and to assign the pre- and postsynaptic neurons, we introduced the VC-Net to explicitly utilize the visual cues of the VCs.
The VC-Net takes a volume of image as its input and produces the likelihood map of the voxels belonging to a VC as the output. The architecture of the VC-Net is almost identical to the SC-Net except for a few parameters and the fact that the VC-Net does not have dropout layers (Supplementary Figure 2B). The same data augmentation strategy was used as the case of the SC-Net. The loss function is the sigmoid cross-entropy error. The learning rate was kept at 1. × 10–3 throughout the training for fast convergence. The training was terminated after 1.62 million updates (Supplementary Figure 3B).
Synapse Prediction and Assignment Using Vesicle-Cloud Likelihood
The output of the VC-Net is the map of voxel-wise likelihood that a voxel is in a VC (Figure 5A). The segments of individual VCs were obtained from the map by the similar method used for the segmentation of neurons, which uses a watershed algorithm to aggregate the similar voxels (Turaga et al., 2010; Zlateski and Seung, 2015). The VC likelihood was considered to represent the affinity between neighbor voxels (Turaga et al., 2010). The voxel-wise likelihood map was converted to a 3D-undirected affinity graph by repeating the likelihood values to three axes. To prevent a VC from hanging across multiple neurons, the likelihood map was masked by the neuronal boundaries. A watershed algorithm was used to turn the affinity map into segmentation (Figure 5B; Zlateski and Seung, 2015). The parameters for watershed were selected from many experiments (data not shown).
The VC segmentation is used to predict the synapses from the synapse candidates. For each synapse candidate, the distance to the closest VC is measured where the VC needs to be inside either of the pair of neuronal segments forming the contact. The distance is defined as the voxel distance between the closest VC voxel and the contact voxel of the synapse candidate (Figure 5C). If the distance is 5 or less, the synapse candidate is considered to have a corresponding VC, and it is finally predicted as a synapse. At this stage, the SVM threshold −0.2 selected above yields the highest F1-score, 0.955 (Figure 5D). We also tried the VC size threshold as the parameter, because too small VCs might be noise and need to be discarded. The result shows that the VC size threshold 1,000 voxels yields the highest F1-score (Figure 5D), and the parameter is accepted. Lastly, the neuron to which the VC belongs is assigned as the presynaptic neuron. The other one naturally becomes the postsynaptic neuron. The synapse type is determined based on the type information and excitatory vs. inhibitory nature of the presynaptic neuronal fragment.
Structure and Connectivity Analysis
All the analyses were performed by custom-written MATLAB codes. The size of the synapse was calculated from the contact size as follows. As the anisotropic volume has a voxel size 12 nm × 12 nm × 50 nm, a face of the contacting voxel has the area 600 nm2 in the YZ or ZX plane and 144 nm2 in the XY plane. The size of the contact can be calculated by counting the contacting faces for each axis direction. The number of contacting faces can be counted during the contact extraction step. It is the same as the number of locations that the segment ID changes along the XYZ axes.
The number of synapses per bouton was found as follows. The VC was considered to be unique to a bouton, and a VC was used as the proxy of a bouton. During the last step of the synapse detection, a VC was matched for each synapse. Here, we counted the number of synapses that were matched to a VC.
Results
Reconstruction and Synapses in the Test Set
The results are discussed and evaluated for the test set. The volume of the test set (11.9 μm × 11.9 μm × 12.4 μm) is 0.15% of the entire data. It contains 598 neuronal fragments in total, which consist of 574 PFs, 4 PCs, 17 INs, and 3 CFs (Figure 3A). Two neuronal fragments (1 glial cell and 1 Golgi cell) were not reconstructed nor considered in this work. All the reconstructed neuronal fragments were neuropils, except one IN soma. About 57.6% of voxels of the volume belong to reconstructed neuronal fragments, and the chief proportion of the remaining voxels belong to glial cells. Other proportion includes small numbers of PFs and INs. Since the brain sample was sectioned and imaged roughly sagittally, the PFs align parallelly and the PCs align perpendicularly with the Z axis (Eccles et al., 1967; Kim and Augustine, 2021). All the PFs pass through both sides of the volume along the Z axis. The PCs are roughly laminated, each occupying one lamina. The CFs and INs appear to irregularly arborize at this scale.
The 598 neuronal segments yield a total of 5,857 contacts, 1,973 of which are the relevant contacts (Table 3). The majority of the relevant contacts involve the PFs. About 59.5% are PF-PC (n = 1,173) and 36.8% are PF-IN (n = 726). The remaining 74 relevant contacts consist of 44 IN-PC, 13 IN-IN, and 17 CF-PC.
TABLE 3
| Pre-Post (cell count) | Relevant Contacts (%) | Total Synapses (%) | Non-synapses (%) | Ambiguous (%) | FP | FN | TP | Precision | Recall | F1-score |
| PF (489) – PC (4) | 1,173 (59.5) | 360 (70.9) | 675 (58.3) | 138 (45) | 9 | 11 | 349 | 0.975 | 0.969 | 0.972 |
| PF (370) – IN (17) | 726 (36.8) | 116 (22.8) | 451 (38.6) | 159 (51.8) | 14 | 9 | 107 | 0.884 | 0.922 | 0.903 |
| CF (3) – PC (4) | 17 (0.9) | 8 (1.6) | 7 (0.6) | 2 (0.7) | 0 | 0 | 8 | 1 | 1 | 1 |
| IN (14) – PC (4) | 44 (2.2) | 21 (4.1) | 20 (1.7) | 3 (1) | 0 | 3 | 18 | 1 | 0.857 | 0.923 |
| IN (6) – IN (4) | 13 (0.7) | 3 (0.6) | 5 (0.4) | 5 (1.6) | 0 | 0 | 3 | 1 | 1 | 1 |
| Total (598) | 1,973 | 508 | 1,158 | 307 | 23 | 23 | 485 | 0.955 | 0.955 | 0.955 |
Accuracy for the test set by cell type.
The synapse detection accuracies for different cell-type pairs are given together with the basic metrics.
From these, 508 are labeled as synapses in the ground truth (Figure 6A). About 1,158 and 307 are labeled as non-synaptic and ambiguous, respectively. About 70.9% of the total synapses are between PF and PC (n = 360), and 22.8% are between PF and IN (n = 116). Of the remaining 32 synapses, 21 are IN-PC, 3 are IN-IN, and 8 are CF-PC (Figure 6B and Table 3).
FIGURE 6
Accuracy Evaluation
The final accuracy of the proposed method was measured to be 0.955 in F1-score (Figures 5D, 6C,D). We estimated the impact of each contact selection rule on the accuracy. We calculated the F1-score when the contacts selected by the rules are assumed to be the final prediction for synapses. The highest F1-score for varying SVM thresholds, when the voxel-fraction-based SVM rule is applied to the relevant contacts, is 0.923. After the voxel-count-based rule (NHSCLV) is applied, the highest F1-score increases to 0.934. The use of VC further raises the F1-score to the final value, 0.955. The utilization of the VCs in addition to the SC likelihood raised the accuracy beyond the 0.95 barrier (Table 1).
The details of accuracy can be evaluated for different contact types, which are determined by the pairs of neuronal-fragment types (Figure 6C and Table 3). The PFs seem to be the major source of errors as they are involved in the majority of the contacts. The PF-IN is least accurate (90.3%) followed by IN-PC (92.3%) and then by PF-PC (97.2%). The accuracies of other contact types appear high, but the number of instances is too small to conclude. Further investigation shows that the errors are most common when the size of the synapse is small in all contact types. They tend to have thin and vague PSD, a small contact area, and a small VC. The low resolution of the image further aggravates the accurate decision for small synapses (data not shown). The PF-related errors occur mainly because the PF synapses are inherently small. The IN synapses have varied sizes, and most of the IN-involved errors occur for small synapses (section “Synapse Size and Density”). A few error examples are shown in Supplementary Figures 5, 6.
Next, we wondered about the impact of type restriction and using relevant contacts (Figure 6D). When the type restriction was not applied and all the contacts were used instead of the relevant contacts, the PR curves for varying SVM thresholds and for varying VC size thresholds are shifted toward down-left, compared to the cases when the relevant contacts were used. The maximum F1-score when the relevant contacts are not used is 0.926 as opposed to the final F1-score, 0.955. While the 92.6% of accuracy is still very competitive to those in the pieces of literature, it is clear that the type restriction enhanced the accuracy even more.
The contacts that were labeled as ambiguous in the ground truth were excluded from the accuracy calculation. We estimated the lower bounds of the accuracy when they were included. If all the ambiguous contacts were actually synapses, then our prediction yields 0.832 F1-score. If all the ambiguous contacts were, indeed, non-synapses, the F1-score of our prediction became 0.862. These values seem to be too low compared to the highest value of 0.955; however, they are still competitive to a few other methods (Table 1). The exclusion of ambiguity is advantageous for the training of AI and is often adopted for accuracy evaluation as well (Dorkenwald et al., 2017).
Synapse Size and Density
In the following sections, we demonstrate the usability of the proposed method by investigating the properties of synapses and connectivity between CML neurons. First, we measured the size of the synapses. The size of a synapse was calculated from the number of voxels in the synaptic contact (see section “Structure and Connectivity Analysis”).
The synapse size was estimated for each contact type (Figure 7A, left four columns). The median area of PF-PC synapses (0.26 μm2) was much smaller than that of IN-PC synapses (1.2 μm2). The PF-IN (0.46 μm2) and CF-PC synapses (0.61 μm2) have a small median synaptic area, too. This is because the excitatory neurons (PF and CF) tend to innervate the dendritic spines, and inhibitory neurons (IN) tend to innervate dendritic shafts as is well known (Eccles et al., 1967). This is clearer when the five contact types are grouped into excitatory and inhibitory types (Figure 7A, right to columns). The median area of the excitatory synapses (0.29 μm2) was much smaller than that of the inhibitory synapses (1.13 m2).
FIGURE 7
The size of PF synapses has the smallest average, median, and variation at the same time, but there are many outliers whose size is larger than the median or average by several folds. The size of the outliers was overestimated by the size of the entire contact. The contacts of PFs are often formed in an elongated shape (Figure 6A), only a part of which is the synaptically relevant contact corresponding to the PSD. We visually inspected the biggest outliers, and all of them were the case.
Such overestimation can be quantitatively estimated using the ground truth labels (see section “Ground Truth and Data Labeling”). The fraction of the size change before and after the erasion of synaptically irrelevant parts of the contacts exhibits a skewed distribution. The 25% of the contacts did not change in size. The median of the fraction of size change is 20%. Considering the 20% of change as typical, the median of the synaptically relevant area of the contacts is estimated to be 0.24 μm2. This value is larger than the calculation from the images of fluorescently labeled postsynaptic proteins, 0.12 ∼ 0.13 μm2 (Zhu et al., 2018). Since this difference can potentially undermine rigorous analyses, we plan to improve the method to manage this issue.
The spatial density of the synapses for different contact types is 0.005/μm3 (CF-PC), 0.002/μm3 (IN-IN), 0.01/μm3 (IN-PC), 0.07/μm3 (PF-IN), and 0.203/μm3 (PF-PC). The PF-PC synapses outnumber all the rest synapses by far. These densities are underestimated because the reconstruction is not complete. When the size of each synapse and the density of the synapses are considered together, the total sum of the area of PF-PC synapses (158.59 μm2) is much larger than that of IN-PC synapses (21.27 μm2). The excitatory neurons jointly provide a larger total synaptic area (230.86 μm2) than the inhibitory neurons do altogether (24.48 μm2).
Multiple-Synaptic Boutons
One presynaptic bouton can innervate multiple postsynaptic sites, where the multiple sites can be either on one neuron or on multiple neurons. We inspected the number of postsynaptic sites that one presynaptic bouton innervates (Figure 7B). The boutons that make more than one synapse were found to be 9%, 42 out of the total number of boutons, 462. This number is probably underestimated, because the reconstruction is not complete. The multiple-synaptic boutons are known to have various functional roles including those related to synaptic plasticity (Harris, 1995; Kim et al., 2019).
Laminar Organization of the Cerebellar Molecular Layer
The dendrites of a PC form a flat arborization along the sagittal plane and different PC dendrites align parallelly to one another. The PFs are parallel to one another and perpendicular to the PC arborization (Eccles et al., 1967). This laminar organization of the CML was quantified for the test set.
We calculated the volume distribution of the PCs by counting the number of voxels along the Z axis (Figure 7C). The distances between adjacent PCs can be determined from the median of each distribution. The maximum distance 5 μm is measured between the two PCs at the center. The average distance is 3.6 μm; however, it is an underestimation because the PCs on both sides of the Z axis are cut off by the border of the data. Therefore, the typical inter-PC distance is assumed to range between 4.5 and 5.5 μm.
We then measured the distance between adjacent PF-PC synapses on a PF along the Z axis. The location of a synapse is represented by the median voxel of the contact. The PF-PC inter-synapse distances are broadly distributed with a prominent peak near 4.5 ∼ 5 μm (Figure 7D). The typical PF-PC inter-synapse distance is consistent with the inter-PC distance.
Lastly, we tested the correlation of PF-PC connections (Figure 7E). Here, we considered cell-to-cell connectivity rather than the synaptic level connections. Let us call the four PCs as PC 1∼4, in order of increasing Z. We measured the fraction of the PFs that synapse also to PC (m + k) among the PFs that synapse to PC (m). It is a conditional probability that a PF has a connection to PC (m + k), given the condition that it has a connection to PC (m). We computed the mean of the conditional probability over m. It appears that the average conditional probability is roughly constant in the small test set. The result suggests that a PF makes a connection to a PC independently of whether or not it has a connection to another PC in a close distance.
Discussion
We report an accurate and automated synapse detection method for cerebellar EM connectomics. It exhibits over 95% of accuracy with full automation. It provides the location, type, size, and direction of the synapses without human intervention. The over 95% accuracy was accomplished for the first time for any data (Table 1). The result is remarkable, considering that the accuracy for the cerebellar sample is lower than those for other brain regions by the same method in a previous report because the cerebellar synapses are small and dense (Becker et al., 2013).
The high accuracy of this method can be attributed to a few factors. First, it utilizes two deep learning AIs, the SC-Net and the VC-Net, which were trained with large amount of data. The VC-Net complements the SC-Net, while they exploit the same visual cues that human experts refer to. Second, the parameters such as the SVM threshold and the VC size threshold were carefully fine-tuned for the test data, and they may be close to the optimum for the entire data we will analyze. Third, the idea of relevant contact and type restriction greatly increased the accuracy. It is particularly beneficial for the case of PFs. The PFs are tightly packed forming bundles and make many contacts with each other (Figure 3C). They may result in many false positive errors without the exclusion of the irrelevant PF-PF contacts (Supplementary Figure 6).
The method has a few limitations, too. First, the reconstruction is a prerequisite (Staffler et al., 2017; Parag et al., 2018; Buhmann et al., 2021). Other methods are needed if a researcher wants to reconstruct only a few neurons first, find their synapses, and then backward trace the synaptic partners of the first neurons from the synapses. Second, the method regards an entire contact as the synapse even when the PSD is only at a small part of it (Staffler et al., 2017). This needs to be improved in the future studies. Third, the type restriction by relevant contacts may limit the chance for the exploration of unknown connections. However, the method can still be used to test whether there exist unknown connections for given types, only if the contacts of those types are set to be relevant. In such cases, however, the accuracy may be low.
Most of the synapse detection software in the pieces of literature, including this work, consists of many files of source code rather than a readily executable program. It is a hard task to reproduce the reported results even for the researchers with expert-level computational skills. There had been a few executable software packages, which were claimed to be easily usable, to be generally applicable, and to have good accuracy (Morales et al., 2011; Becker et al., 2013). Nevertheless, they have been recently replaced by newer and more accurate technologies equipped with deep learning AIs. The AIs have to be trained and fine-tuned before application. It is unfortunate particularly for the common neurobiologists with basic computational skills. Efforts are being made toward generally usable software with AI (Staffler et al., 2017; Buhmann et al., 2021).
The proposed method can easily scale up as the computation time for processing the test set is less than 20 min on a desktop computer with 8 CPU cores and 32 GB memory. While the method is tweaked for, and may seem to be limited to, the cerebellum and the data, the approaches and ideas may be extended to other brain regions as well. The requirements for the extension can include the change in the network architecture of the AIs, new training of the AIs with the image data from the region, fine tuning of the parameters, and the region’s having well-defined cell types and type connectivity. Nevertheless, the more important lesson of this study may be is the idea that new specialized designs exploiting tissue-specific properties of the different brain regions will enhance the performance of the methods. For example, there can be different policies to replace the type restriction for other brain regions. The high accuracy of the method for individual synapses is advantageous for the inspection of subcellular wiring specificity. The connectomic analyses on the small test set already showed interesting results. We plan to report the larger-scale analyses of the entire data soon. All in all, the method is practically useful for large-scale cerebellar connectomics.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Statements
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The animal study was reviewed and approved by KBRI Committee for Animal Research.
Author contributions
JK designed the study with inputs from CP and JG. CP and JG wrote the codes and analyzed the data with the supervision of JK. KL advised on biology. SL advised on computation. CP and JK interpreted the data and wrote the manuscript with inputs from JG, SL, and KL. All authors contributed to the article and approved the submitted version.
Funding
This research was supported by KBRI Basic Research Program (19-BR-01-07, 19-BR-01-01, 21-BR-03-01) funded by the Korean Ministry of Science and ICT; Brain Research Program (2017M3C7A1048086) and Basic Science Research Program (2018R1A5A1060031) through the National Research Foundation of Korea (NRF) funded by the Korean Ministry of Science and ICT; and Basic Science Research Program (2019R1A6A1A10073079) through NRF funded by the Korean Ministry of Education. CP acknowledges the support by the BK21 FOUR program through NRF funded by the Korean Ministry of Education.
Acknowledgments
We would like to thank J. -E. Son and S. Bahn for computational support and K. Lee for help on AI. J. Shin, J. Kim, J. Yoon, H. Suh, D. Yoo, H. Kim, J. Kang, D. Cho, and C. Ham worked for EM reconstruction and annotation. The EM images were acquired at the Advanced Neural Imaging Center of KBRI.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnana.2022.760279/full#supplementary-material
References
1
AbbottL. F.BockD. D.CallawayE. M.DenkW.DulacC.FairhallA. L.et al (2020). The mind of a mouse.Cell1821372–1376. 10.1016/j.cell.2020.08.010
2
BeckerC.AliK.KnottG.FuaP. (2013). Learning context cues for synapse segmentation.IEEE Trans. Med. Imaging321864–1877. 10.1109/TMI.2013.2267747
3
BergerD. R.SeungH. S.LichtmanJ. W. (2018). VAST (volume annotation and segmentation tool): efficient manual and semi-automatic labeling of large 3D image stacks.Front. Neural Circuits12:88. 10.3389/fncir.2018.00088
4
BriggmanK. L.HelmstaedterM.DenkW. (2011). Wiring specificity in the direction-selectivity circuit of the retina.Nature471183–188. 10.1038/nature09818
5
BuhmannJ.SheridanA.Malin-MayorC.SchlegelP.GerhardS.KazimiersT.et al (2021). Automatic detection of synaptic partners in a whole-brain Drosophila electron microscopy data set.Nat. Methods18771–774. 10.1038/s41592-021-01183-7
6
CardonaA.SaalfeldS.SchindelinJ.Arganda-CarrerasI.PreibischS.LongairM.et al (2012). TrakEM2 software for neural circuit reconstruction.PLoS One7:e38011. 10.1371/journal.pone.0038011
7
CicekO.AbdulkadirA.LienkampS. S.BroxT.RonnebergerO. (2016). 3D U-Net: learning dense volumetric segmentation from sparse annotation.arXiv[preprint]. arXiv:1606.06650,
8
ColonnierM. (1968). Synaptic patterns on different cell types in the different laminae of the cat visual cortex. An electron microscope study.Brain Res.9268–287. 10.1016/0006-8993(68)90234-5
9
del Valle RodríguezA.DidianoD.DesplanC. (2011). Power tools for gene expression and clonal analysis in Drosophila.Nat. Methods947–55. 10.1038/nmeth.1800
10
DenkW.HorstmannH. (2004). Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure.PLoS Biol.2:e329. 10.1371/journal.pbio.0020329
11
DorkenwaldS.MckellarC.MacrinaT.KemnitzN.LeeK.LuR.et al (2020). FlyWire: online community for whole-brain connectomics.bioRxiv[preprint]. 10.1101/2020.08.30.274225
12
DorkenwaldS.SchubertP. J.KillingerM. F.UrbanG.MikulaS.SvaraF.et al (2017). Automated synaptic connectivity inference for volume electron microscopy.Nat. Methods14435–442. 10.1038/nmeth.4206
13
EcclesJ. C.ItoM.SzentágothaiJ. (1967). The Cerebellum as a Neuronal Machine.Berlin: Springer.
14
FedermeierK. D.KleimJ. A.GreenoughW. T. (2002). Learning-induced multiple synapse formation in rat cerebellar cortex.Neurosci. Lett.332180–184. 10.1016/s0304-3940(02)00759-0
15
GrayE. G. (1959). Axo-somatic and axo-dendritic synapses of the cerebral cortex: an electron microscope study.J. Anat.93(Pt. 4)420–433.
16
HarrisK. M. (1995). How multiple-synapse boutons could preserve input specificity during an interneuronal spread of LTP.Trends Neurosci.18365–369. 10.1016/0166-2236(95)93930-v
17
HeinrichL.FunkeJ.PapeC.Nunez-IglesiasJ.SaalfeldS. (2018). “Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete Drosophila brain,” in Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention – MICCAI 2018, edsFrangiA. F.SchnabelJ. A.DavatzikosC.Alberola-LopezC.FichtingerG. (Cham: Springer), 317–325. 10.1007/978-3-030-00934-2_36
18
HuangG. B.SchefferL. K.PlazaS. M. (2018). Fully-automatic synapse prediction and validation on a large data set.Front. Neural Circuits12:87. 10.3389/fncir.2018.00087
19
IppolitoD. M.ErogluC. (2010). Quantifying synapses: an immunocytochemistry-based assay to quantify synapse number.J. Vis. Exp. JoVE45:2270. 10.3791/2270
20
JanuszewskiM.KornfeldJ.LiP. H.PopeA.BlakelyT.LindseyL.et al (2018). High-precision automated reconstruction of neurons with flood-filling networks.Nat. Methods15605–610. 10.1038/s41592-018-0049-4
21
JiaY.ShelhamerE.DonahueJ.KarayevS.LongJ.GirshickR.et al (2014). Caffe: convolutional architecture for fast feature embedding.arXiv[preprint]. arXiv:1408.5093, 10.1016/j.sjbs.2019.12.004
22
KimH. W.OhS.LeeS. H.LeeS.NaJ. E.LeeK. J.et al (2019). Different types of multiple-synapse boutons in the cerebellar cortex between physically enriched and ataxic mutant mice.Microsc. Res. Tech.8225–32. 10.1002/jemt.23054
23
KimJ. S.GreeneM. J.ZlateskiA.LeeK.RichardsonM.TuragaS. C.et al (2014). Space-time wiring specificity supports direction selectivity in the retina.Nature509331–336. 10.1038/nature13240
24
KimJ.AugustineG. J. (2021). Molecular layer interneurons: key elements of cerebellar network computation and behavior.Neuroscience46222–35. 10.1016/j.neuroscience.2020.10.008
25
KreshukA.KoetheU.PaxE.BockD. D.HamprechtF. A. (2014). Automated detection of synapses in serial section transmission electron microscopy image stacks.PLoS One9:e87351. 10.1371/journal.pone.0087351
26
KreshukA.StraehleC. N.SommerC.KoetheU.CantoniM.KnottG.et al (2011). Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images.PLoS One6:e24899. 10.1371/journal.pone.0024899
27
LeeK.TurnerN.MacrinaT.WuJ.LuR.SeungH. S. (2019). Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy.Curr. Opin. Neurobiol.55188–198. 10.1016/j.conb.2019.04.001
28
LeeK.ZungJ.LiP.JainV.SeungH. S. (2017). Superhuman accuracy on the SNEMI3D connectomics challenge.arXiv[preprint]. arXiv:1706.00120,
29
MoralesJ.Alonso-NanclaresL.RodríguezJ. R.DefelipeJ.RodríguezA.Merchán-PérezA. (2011). Espina: a tool for the automated segmentation and counting of synapses in large stacks of electron microscopy images.Front. Neuroanat.5:18. 10.3389/fnana.2011.00018
30
NguyenT. M.ThomasL. A.RhoadesJ. L.RicchiI.YuanX. C.SheridanA.et al (2021). Structured connectivity in the cerebellum enables noise-resilient pattern separation.bioRxiv[preprint]. 10.1101/2021.11.29.470455
31
OhyamaT.Schneider-MizellC. M.FetterR. D.AlemanJ. V.FranconvilleR.Rivera-AlbaM.et al (2015). A multilevel multimodal circuit enhances action selection in Drosophila.Nature520633–639. 10.1038/nature14297
32
OsornoT.RudolphS.NguyenT.KozarevaV.NadafN.MacoskoE. Z.et al (2021). Candelabrum cells are molecularly distinct, ubiquitous interneurons of the cerebellar cortex with specialized circuit properties.bioRxiv[preprint]. 10.1101/2021.04.09.439172
33
ParagT.BergerD.KamentskyL.StafflerB.WeiD.HelmstaedterM.et al (2018). “Detecting synapse location and connectivity by signed proximity estimation and pruning with deep nets,” in Proceedings of the 2018 European Conference on Computer Vision (ECCV) Workshops, edsLeal-TaixéL.RothS. (Cham: Springer), 354–364.
34
ParkC.GimJ.LeeS.LeeK. J.KimJ. S. (2022). Automated synapse detection method for cerebellar connectomics. Front. Neuroanat.16:760279. 10.3389/fnana.2022.760279
35
PlazaS. M.ParagT.HuangG. B.OlbrisD. J.SaundersM. A.RivlinP. K. (2014). Annotating synapses in large EM datasets.arXiv[preprint]. arXiv:1409.1801v2,
36
ProvostF.WeissG. M. (2003). Learning when training data are costly: the effect of class distribution on tree induction.J. Artif. Intell. Res.19315–354. 10.1613/jair.1199
37
RoncalW. G.PekalaM.Kaynig-FittkauV.KleissasD. M.VogelsteinJ. T.PfisterH.et al (2015). “VESICLE: volumetric evaluation of synaptic interfaces using computer vision at large scale,” in Proceedings of the British Machine Vision Conference (BMVC), Swansea, 81.1–81.13. 10.5244/C.29.81
38
RonnebergerO.FischerP.BroxT. (2015). U-Net: convolutional networks for biomedical image segmentation.arXiv[preprint]. arXiv:1505.04597,
39
SrivastavaN.HintonG.KrizhevskyA.SutskeverI.SalakhutdinovR. (2014). Dropout: a simple way to prevent neural networks from overfitting.J. Mach. Learn. Res.151929–1958.
40
StafflerB.BerningM.BoergensK. M.GourA.SmagtP. V.HelmstaedterM. (2017). SynEM, automated synapse detection for connectomics.ELife6:e26414. 10.7554/eLife.26414
41
TakemuraS. Y.BhariokeA.LuZ.NernA.VitaladevuniS.RivlinP. K.et al (2013). A visual motion detection circuit suggested by Drosophila connectomics.Nature500175–181. 10.1038/nature12450
42
ToniN.BuchsP. A.NikonenkoI.BronC. R.MullerD. (1999). LTP promotes formation of multiple spine synapses between a single axon terminal and a dendrite.Nature402421–425. 10.1038/46574
43
TuragaS. C.MurrayJ. F.JainV.RothF.HelmstaedterM.BriggmanK.et al (2010). Convolutional networks can learn to generate affinity graphs for image segmentation.Neural Comput.22511–538. 10.1162/neco.2009.10-08-881
44
WhiteJ. G.SouthgateE.ThomsonJ. N.BrennerS. (1986). The structure of the nervous system of the nematode Caenorhabditis elegans.Philos. Trans. R. Soc. Lond. B Biol. Sci.3141–340. 10.1098/rstb.1986.0056
45
WilsonA. M.SchalekR.Suissa-PelegA.JonesT. R.Knowles-BarleyS.PfisterH.et al (2019). Developmental rewiring between cerebellar climbing fibers and purkinje cells begins with positive feedback synapse addition.Cell Rep.292849–2861.e6. 10.1016/j.celrep.2019.10.081
46
XiaoC.LiW.DengH.ChenX.YangY.XieQ.et al (2018). Effective automated pipeline for 3D reconstruction of synapses based on deep learning.BMC Bioinformatics19:263. 10.1186/s12859-018-2232-0
47
XuZ. X.KimG. H.TanJ. W.RisoA. E.SunY.XuE. Y.et al (2020). Elevated protein synthesis in microglia causes autism-like synaptic and behavioral aberrations.Nat. Commun.11:1797. 10.1038/s41467-020-15530-3
48
ZhengZ.LauritzenJ. S.PerlmanE.RobinsonC. G.NicholsM.MilkieD.et al (2018). A complete electron microscopy volume of the brain of adult Drosophila melanogaster.Cell174730–743.e22. 10.1016/j.cell.2018.06.019
49
ZhuF.CizeronM.QiuZ.Benavides-PiccioneR.KopanitsaM. V.SkeneN. G.et al (2018). Architecture of the mouse brain synaptome.Neuron99781–799.e10. 10.1016/j.neuron.2018.07.007
50
ZiffE. B. (1997). Enlightening the postsynaptic density.Cell191163–1174. 10.1016/S0896-6273(00)80409-2
51
ZlateskiA.SeungH. S. (2015). Image segmentation by size-dependent single linkage clustering of a watershed basin graph.arXiv[preprint]. arXiv:1505.00249,
Summary
Keywords
connectomics, cerebellum, synapse, electron microscopy, image analysis, machine learning, computer algorithm
Citation
Park C, Gim J, Lee S, Lee KJ and Kim JS (2022) Automated Synapse Detection Method for Cerebellar Connectomics. Front. Neuroanat. 16:760279. doi: 10.3389/fnana.2022.760279
Received
17 August 2021
Accepted
14 February 2022
Published
11 March 2022
Volume
16 - 2022
Edited by
Zoltan F. Kisvarday, University of Debrecen, Hungary
Reviewed by
Jon Storm-Mathisen, University of Oslo, Norway; Rosa M. Villalba, Emory University, United States
Updates
Copyright
© 2022 Park, Gim, Lee, Lee and Kim.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jinseop S. Kim, jinseopskim@skku.edu
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.