# ELECTRON-MICROSCOPY-BASED TOOLS FOR IMAGING CELLULAR CIRCUITS AND ORGANISMS

EDITED BY : Yoshiyuki Kubota PUBLISHED IN : Frontiers in Neural Circuits and Frontiers in Neuroanatomy

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-257-2 DOI 10.3389/978-2-88963-257-2

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# ELECTRON-MICROSCOPY-BASED TOOLS FOR IMAGING CELLULAR CIRCUITS AND ORGANISMS

Topic Editor:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

Citation: Kubota, Y., ed. (2019). Electron-Microscopy-Based Tools for Imaging Cellular Circuits and Organisms. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-257-2

# Table of Contents


Anne Jorstad, Jérôme Blanc and Graham Knott

*20* SBEMimage*: Versatile Acquisition Control Software for Serial Block-Face Electron Microscopy*

Benjamin Titze, Christel Genoud and Rainer W. Friedrich

*29 Fast Homogeneous* En Bloc *Staining of Large Tissue Samples for Volume Electron Microscopy*

Christel Genoud, Benjamin Titze, Alexandra Graff-Meyer and Rainer W. Friedrich


Adrian A. Wanner and Ashwin Vishwanathan


Ting Zhao, Donald J. Olbris, Yang Yu and Stephen M. Plaza


*140 Verifying, Challenging, and Discovering New Synapses Among Fully EM-Reconstructed Neurons in the Leech Ganglion* Jason E. Pipkin, Eric Allen Bushong, Mark H. Ellisman and William B. Kristan Jr

*152 A Pipeline for Volume Electron Microscopy of the* Caenorhabditis elegans *Nervous System*

Ben Mulcahy, Daniel Witvliet, Douglas Holmyard, James Mitchell, Andrew D. Chisholm, Yaron Meirovitch, Aravinthan D. T. Samuel and Mei Zhen


Anna Lena Eberle and Dirk Zeidler


# Editorial: Electron-Microscopy-Based Tools for Imaging Cellular Circuits and Organisms

Yoshiyuki Kubota1,2 \*

*<sup>1</sup> Division of Cerebral Circuitry, National Institute for Physiological Sciences, Okazaki, Japan, <sup>2</sup> Department of Physiological Sciences, The Graduate University for Advanced Studies (SOKENDAI), Okazaki, Japan*

Keywords: volume EM, segmentation (image processing), neural network, artificial intelligence, connectome, SSEM, synapse

**Editorial on the Research Topic**

#### **Electron-Microscopy-Based Tools for Imaging Cellular Circuits and Organisms**

Electron microscopy (EM)-based reconstruction of neuronal circuits from serial ultrathin sections was introduced more than three decades ago (White and Keller, 1987). Initially, all the steps were conducted manually, including cutting serial ultrathin sections using the ultramicrotome, image capturing with transmission electron microscope (TEM), and reconstruction using cardboard pieces of selected profiles of neural structures to provide the impression of depth. In the 1990s, computer software assisted reconstruction methods to make it more efficient were introduced (Harris et al., 1992; White et al., 1994). This reconstruction analysis software was used in a limited number of laboratories where good skills for obtaining serial ultrathin sections had been established and, thus, significant and valuable results were obtained. In general, however, this reconstruction technology was not popular because of a high demand on skills to obtain high quality serial ultrathin sections. In the early 2000s, a number of groups, with many represented in this special issue, started to conduct neural network analyses with reconstruction of serial sections by adapting new EM technologies such as focused ion beam-scanning electron microscopy (FIB-SEM; Knott et al., 2008), serial block-face electron microscopy (SBEM; Denk and Horstmann, 2004; Ohno et al., 2014), automated tape-collecting ultramicrotomy (ATUM) with SEM (Terasaki et al., 2013), transmission electron microscope camera array (TEMCA; Bock et al., 2011), and transmission-mode SEM (Kuwajima et al., 2013). These approaches have been modified and improved vigorously (Kubota et al., 2018a,b), and a large amount of noteworthy results were published in the last decade (Tomassy et al., 2014; Kasthuri et al., 2015; Lee et al., 2016; Villa et al., 2016; Schmidt et al., 2017; Takemura et al., 2017; Bae et al., 2018). The size of EM volume data sets has grown year by year, and it could be huge especially when data are obtained with high-throughput EM systems of either TEMCA (Bock et al., 2011; Lee et al., 2016), multi-beam SEM (Eberle and Zeidler; Shibata et al.) or parallel processing with multiple single beam SEM systems (Plaza and Funke; Scheffer.) For instance, a 100 cubic µm EM data set with 5 nm/pixel and 30 nm z-step of mouse cortex block, which amounts to 1.3 TB, was obtained that provides a sufficient resolution to detect synaptic contacts. This data set should contain about 1,000,000 synapses (Merchan-Perez et al., 2009) and be sufficient in volume to include many different kinds of connections among a wide variety of cortical neuron subtypes and afferent axonal fibers from other brain regions. Such large volume EM data sets could not be acquired with the conventional manually operated EM using the ultramicrotomes and TEM (White and Keller, 1987; Kubota and Kawaguchi, 2000; Kubota et al., 2015; Marc et al.).

Edited and reviewed by: *Edward S. Ruthazer, McGill University, Canada*

> \*Correspondence: *Yoshiyuki Kubota yoshiy@nips.ac.jp*

Received: *15 August 2019* Accepted: *24 September 2019* Published: *11 October 2019*

#### Citation:

*Kubota Y (2019) Editorial: Electron-Microscopy-Based Tools for Imaging Cellular Circuits and Organisms. Front. Neural Circuits 13:64. doi: 10.3389/fncir.2019.00064*

The success of large volume EM acquisition using these new EM systems has created an issue, i.e., how to process large image data sets thus obtained. Soon it became obvious that it was difficult to handle large EM volume data sets using conventional 3D reconstruction image processing computer applications that had been developed for the conventional EM data sets obtained with the ultramicrotome and TEM. Therefore, new image processing tools that can handle large volume data sets have been developed. For example, NIH imageJ plugins provide useful tools to stitch tiles for montage and to align serial section images (Cardona et al., 2012). The current bottle neck is the segmentation process. Currently, the majority of researchers working on EM volume data pursue segmentation of their data obtained from brains of a wide variety of animal species: Caenorhabditis elegans (Mulcahy et al.), leech (Pipkin et al.), Drosophila (Takemura et al., 2017), Zebrafish (Wanner and Vishwanathan), mouse (Maclachlan et al.), rat, marmoset, and human, and use manual image processing applications including: VAST, Reconstruct, Knossos, and others (Fiala, 2005; Dorkenwald et al., 2017; Berger et al.). To achieve segmentation easily and efficiently, automated segmentation computer applications have been developed (Januszewski et al., 2018; Lee et al., 2019) and used for many EM volume data sets. Segmentation performance has increasingly been improved and achieved coverage of ∼90% of the volume, but it is not yet perfect (Plaza and Funke). Annotators are used to fix segmentation errors to create correct wiring of brain networks. This can be done manually using image software with a proof-reading function (Zhao et al.; Katz et al.). Hopefully, segmentation performance will improve further in the near future to reach an almost 100% success rate while reducing the time required for the proof-reading process. Toward this goal, the histological process should be improved (Hua et al., 2015; Mikula and Denk, 2015; Mikula, 2016; Genoud et al.; Maclachlan et al.; Nguyen et al.) and image processing tools with better performance (Berger et al.; Jorstad et al.; Titze et al.) should be developed. Finally, despite these technological advances, analyzing fully segmented EM volume data sets can be done only manually by researchers so far, who must have a good knowledge and understanding of brain networks. In addition, automated cell type identification tool (Schubert et al., 2019), automated synapse detection tool (Staffler et al., 2017), and correlated light and electron microscopy methods (Kubota et al., 2015; Wanner and Vishwanathan) are

#### REFERENCES


useful for neural network analyses. This special issue covers most of the cutting-edge 3D-EM methods currently available.

On September 8th, 2017, I contacted Shawn Mikula at the Max Planck Institute in Martinsried, to ask him if he would be willing to work as co-editor with me on a special issue of Frontiers in Neural Circuits Research Topic "Volume electron microscopy for neuroscience." I wanted Shawn to be my partner to edit the special issue because I knew that he had a deep knowledge not only for the EM volume data set analysis but also histology, chemistry, mathematics and other areas. He immediately accepted the invitation and we chose the title of the Research Topic "Electron-Microscopy-Based Tools for Imaging Cellular Circuits and Organisms." We started to invite contributors to the special topic issue at the end of October, 2017. Shawn invited many excellent researchers who have been developing image analysis applications or systems and/or working on large volume EM data sets. His selections indeed led to the success of this special topic issue. Subsequently, Shawn joined my laboratory at the National Institute for Physiological Sciences in Okazaki, Japan, briefly from January 2nd to March 30th, 2018. After visiting his family in the USA in April and May of 2018, he moved to the Keio University School of Medicine in Tokyo. With great sadness and most unfortunately, on July 8th, 2018 we lost Shawn tragically, when we just started reviewing a few manuscripts submitted.

This special issue reflects Shawn Mikula's great interest in the brain network architecture and his commitment to introduce the best technology available to all researchers conducting neuroscience research with EM volume data set analyses. I took over all the editorial work after Shawn was lost and always kept these convictions with me during the editorial work. I hope he would appreciate the results. Finally, I express my sincerest condolences and special thanks to Shawn Mikula's family.

# AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and has approved it for publication.

#### ACKNOWLEDGMENTS

I thank Dr. Shawn Mikula and Mrs. Sarah Mikula for valuable contributions.


**Conflict of Interest:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Kubota. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# NeuroMorph: A Software Toolset for 3D Analysis of Neurite Morphology and Connectivity

Anne Jorstad\*, Jérôme Blanc and Graham Knott\*

Biological Electron Microscopy Facility, Centre of Electron Microscopy, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

The geometries of axons, dendrites and their synaptic connections provide important information about their functional properties. These can be collected directly from measurements made on serial electron microscopy images. However, manual and automated segmentation methods can also yield large and accurate models of neuronal architecture from which morphometric data can be gathered in 3D space. This technical paper presents a series of software tools, operating in the Blender open source software, for the quantitative analysis of axons and their synaptic connections. These allow the user to annotate serial EM images to generate models of different cellular structures, or to make measurements of models generated in other software. The paper explains how the tools can measure the cross-sectional surface area at regular intervals along the length of an axon, and the amount of contact with other cellular elements in the surrounding neuropil, as well as the density of organelles, such as vesicles and mitochondria, that it contains. Nearest distance measurements, in 3D space, can also be made between any features. This provides many capabilities such as the detection of boutons and the evaluation of different vesicle pool sizes, allowing users to comprehensively describe many aspects of axonal morphology and connectivity.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Joachim H. R. Lübke, Forschungszentrum Jülich, Helmholtz-Gemeinschaft Deutscher Forschungszentren (HZ), Germany Daniel Raimund Berger, Harvard University, United States

#### \*Correspondence:

Anne Jorstad anne.jorstad@epfl.ch Graham Knott graham.knott@epfl.ch

Received: 21 April 2018 Accepted: 29 June 2018 Published: 23 July 2018

#### Citation:

Jorstad A, Blanc J and Knott G (2018) NeuroMorph: A Software Toolset for 3D Analysis of Neurite Morphology and Connectivity. Front. Neuroanat. 12:59. doi: 10.3389/fnana.2018.00059 Keywords: neuroimaging software, 3D modeling, data visualization, serial section electron microscopy, cell morphology, neuron, synapse, connectomics

# 1. INTRODUCTION

The development of volume EM imaging methods now provides unprecedented opportunities to understand the detailed morphology and connectivity of neurons (Briggman and Denk, 2006; Kornfeld and Denk, 2018). Geometrical analysis of the imaged structures, however, requires either measuring the required features directly on the serial images, or interacting with 3D models, once they have been extracted after segmentation. Here we present a set of software tools for exploring, annotating and measuring various features of 3D models.

Open source software such as Fiji<sup>1</sup> (Cardona et al., 2012), KNOSSOS<sup>2</sup> (Helmstaedter et al., 2011), Espina<sup>3</sup> (Morales et al., 2011), Reconstruct<sup>4</sup> (Fiala, 2005), and ITK-SNAP<sup>5</sup> (Yushkevich et al., 2006), and proprietary software including Amira<sup>6</sup> , exist for annotating and measuring features

#### <sup>1</sup>fiji.sc

<sup>2</sup>knossostool.org 3 cajalbbp.cesvima.upm.es/espina 4 synapseweb.clm.utexas.edu/software-0 <sup>5</sup>http://www.itksnap.org/pmwiki/pmwiki.php <sup>6</sup>www.fei.com/software/amira-avizo

on 2D serial images, and constructing 3D models that can be examined visually but generally not manipulated. In contrast, NeuroMorph has been developed to analyze and interact with the models reconstructed from any of these tools directly in a 3D environment. The NeuroMorph tools augment these models, and perform specialized analyses directly on them.

To explain the different functionalities of these tools, we present how they can be used to measure a range of different features of axonal boutons. This includes how the 3D models can be visualized together with the original image stacks, and features such as synapses and vesicles added to the model. We explain how to make volume, surface area, and length measurements on any part of the model, and how a centerline of an axon can be used to show the changing densities of elements such as vesicles along its length. In addition we include tools for measuring the degree to which structures such as boutons are in contact with other cellular features such as dendrites. This is useful for understanding more about axonal function.

The software and detailed instructions, along with a stack of EM images and corresponding scale models of biological structures including the ones shown in this paper, are available from our website<sup>7</sup> which links to our GitHub page<sup>8</sup> .

The Blender models created and analyzed with these tools are also compatible with the simulation software MCell<sup>9</sup> that uses the same Blender software via the CellBlender graphical user interface to simulate various aspects of cellular processes (Kerr et al., 2008).

#### 2. MESHES

Blender<sup>10</sup> is a widely used, free, open-source software package developed primarily for 3D computer graphics applications. NeuroMorph is a toolset comprising "addons" that can be integrated into Blender to provide specialized tools for the analysis of 3D models derived from electron microscopy imagery of neurons. However, much of its functionality can be applied to models derived from any source. This paper extends our previous work (Jorstad et al., 2015), which presented an earlier version of the NeuroMorph Measurements tool, described here briefly in section 4.

A 3D model is comprised of a mesh that is defined by points called vertices, edges connecting the vertices, and polygons called faces that are bounded by the edges, to create that 3D surface, see **Figure 1**. The surfaces can be either closed like a ball or open like a piece of cloth.

Meshes can be loaded into Blender from a variety of sources, or constructed directly within the software itself. For example, .obj files of annotations made in Fiji<sup>11</sup> can be imported into Blender using the NeuroMorph Import Objects tool (found in the "Other Tools" section of the toolkit).

FIGURE 1 | Mesh geometry fundamentals. (Left) Part of a sphere, with four orange vertices that are connected by edges, forming a single face. (Right) A closed object and an open object.

In this paper we will analyze the 3D meshes of an axon containing meshes of mitochondria, vesicles, and synaptic contacts (see **Figure 2**). In section 3 we will describe how to visualize the serial images, as well as how to add spherical meshes at the position of each vesicle, and also create surface meshes representing the synapses. In section 5 we will also make use of a centerline, running through the axon, and consisting of vertices strung together by edges, without any faces. This centerline can be used to carry out various analyses that are useful for describing the geometry of the axon.

#### 3. DRAWING IN 3D

A common method of creating 3D models from serial EM images is to annotate each image pixel by pixel, painting the structure of interest. This can be done in software such as Fiji and Microscopy Image Browser12. The models are then exported as 3D objects. This process can be very time-consuming, and the repetitive task of drawing on 2D planes gives little information of the 3D nature of the structure of interest. The NeuroMorph 3D Drawing tool offers a faster alternative, by allowing users to mark and draw onto the serial images directly in the 3D workspace of Blender. The image stack can be efficiently navigated while the 3D structures are being constructed, providing the user with a better sense of the structure as it is being created.

When the user loads the image stack into the software, an "Image Stack Ladder" object is created in Blender that is the height of the image stack, located in the corner of the images, and consists of small triangular faces pointing to the locations of

<sup>7</sup>neuromorph.epfl.ch

<sup>8</sup>https://github.com/NeuroMorph-EPFL/NeuroMorph

<sup>9</sup>mcell.org

<sup>10</sup>www.blender.org <sup>11</sup>https://fiji.sc/

<sup>12</sup>http://mib.helsinki.fi

each image in the stack, see **Figure 3**. This allows the user to select any vertex in the Image Stack Ladder, and view the image at that location in the image stack. Multiple images can be displayed at once. Clicking on a single image, the user can scroll through the image stack, allowing the user to explore the image stack and see how the 3D objects align with the images.

From a single image stack, with many images stacked in the Z dimension, a tool to generate image stacks of the same data in the X and Y dimensions is also provided, called "Generate 3D Image Stacks" (found in "Other Tools"). All three stacks can be loaded into Blender and explored for extended 3D analysis. Special thanks to Tom Boissonnet for the contribution of this feature.

#### 3.1. Plotting Synaptic Vesicles

The "Mark Points on Image" tool allows the user to create spherical meshes of any size with a single click on an image plane, see **Figure 4**. Here we use this tool to place spheres of a chosen size in the axonal bouton at the exact position of each vesicle. After placing each vesicle sphere, the user can scroll back and forth between the nearby images to visualize the sphere's placement in 3D. If an annotation is incorrect, it can be quickly deleted and the sphere placed again. In this manner, many vesicles can be quickly plotted. If a large number of spheres are going to be created, an option is available to construct spheres using fewer vertices, which results in meshes that look less smooth, but are computationally more efficient.

The vesicles created with this tool will be used in the Sphere to Surface Distances tool described in section 6 to analyze the proximity of vesicles to the pre-synaptic membrane.

#### 3.2. Drawing Surfaces

The "Draw Curves and Surfaces on Image" tool allows the user to draw curves on images, connect these curves from several layers of the image stack, and create a 3D surface. Here this is used to construct a pre-synaptic membrane.

The user can draw along the boundary of a neuronal structure in an image, erasing if necessary, to correctly outline an object, see **Figure 5**, top left. When the desired curve has been drawn, it can be converted into a mesh curve consisting of vertices and edges. For faster annotation, the tool can be set to convert curves automatically as soon as the mouse click used for drawing is released. Curves outlining an object should be drawn on several images, either on adjacent images for more precision, or leaving out a few images in between each curve for faster annotation, see **Figure 5**, top right. Once several curves outlining the same object have been created, the "Construct Mesh Surface from Curves" tool will fit a mesh surface through the curves, adding faces to result in a 3D mesh surface of the object that was outlined, see **Figure 5**, bottom row.

It is also possible to construct surfaces with holes, such as perforated synapses. Simply drawing curves on either side of the hole on each image, with no drawing inside the hole, then constructing the mesh surface from the curves as before, will result in a 3D mesh surface with holes as annotated, see **Figure 6**.

#### 3.2.1. Limiting Cases

When two adjacent curves have very different trajectories, the resulting surface might be incorrect. The algorithm works by fitting a linear surface through each adjacent pair of curves, and then combining all surface segments together at the end to form a single continuous surface. The surface construction functions well when adjacent curves are near parallel, with endpoints not too far from each other. However, when adjacent curves are close to perpendicular, the algorithm cannot be sure which endpoints should correspond, and this can result in a self-intersecting surface. In this case, the user should add more intermediate curves, to better define the progression of the surface being reconstructed.

If a curve is drawn too fast, it is possible that the constructed mesh curve might include extraneous vertices outside of the desired chain of vertices. If this happens, the extra vertices can simply be deleted.

If the constructed surface is not sufficiently smooth for a given application, the user is able to add more intermediate curves to provide as much fine detail as necessary.

#### 3.3. Drawing Tubes

It is also possible to connect closed curves into closed tubular objects, see **Figure 7**. Checking "Closed Curve" tells the tool that the curves should be closed, like a circle. Constructing a mesh

surface from these closed curves will create a closed tube, and the ends of this tube can also be closed to form a closed object. This tool is not currently capable of handling branching objects, but can be used for annotating tubular objects such as mitochondria. It is possible to handle U-shaped tubular structures with more than one cross section in a single z-plane, but such objects must be constructed in parts with only one cross section per z-plane, and then joined together, which is a simple operation in Blender.

# 4. MEASUREMENTS

#### 4.1. Lengths

The NeuroMorph Measurements tool provides three different length measurement functionalities.

• **Distance Between 2 Points:** calculates the distance between two selected vertices, ignoring the mesh surface.


#### 4.2. Surface Areas

The surface area of a mesh or any subsection of a mesh can be calculated by highlighting the desired faces and clicking the "Surface Area" button, see **Figure 8**.

#### 4.3. Volumes

The volume of a mesh or any subsection of a mesh can be calculated by highlighting the faces defining the region and clicking the "Volume" button. The tool will first close any holes in the mesh, and then calculate the volume of the closed region, see **Figure 8**.

For more details on measurement calculations, see Jorstad et al. (2015) where the NeuroMorph Measurements tool and its limitations are described in full.

## 5. CENTERLINE AND CROSS SECTIONAL ANALYSES

This suite of tools allows the user to gather structural information about the object (the axon in this case) in terms of the presence of other objects (organelles), or how certain geometric properties change along its length.

The tool first facilitates the creation of a line down the center of the axon, which provides an object along which various properties can be measured. The Vascular Modeling Toolkit (VMTK) (Antiga et al., 2008) is a separate software package that provides a useful tool for centerline creation13, based on the algorithm from Antiga and Steinman (2004). The NeuroMorph Centerline tool exports the axon in a format that can be processed by VMTK, and then reads back in the centerline mesh to be used with the rest of this tool. VMTK must be installed separately in order to use this tool.

The constructed centerline does not run down the exact geometric center of the axon, but is instead of smooth curve that always remains on the inside of the structure, and serves as a representative skeleton of the axon, see **Figure 9**.

Although VMTK is able to construct branching centerlines, the NeuroMorph centerline functionality is based on a single non-branching centerline per axon. To handle branches, a second additional centerline can be constructed, and the calculations performed separately on that branch. Output data can then be combined as determined by the user. U-shaped and S-shaped axons are handed correctly by all the functionality provided by this tool, even in the case when there are multiple cross-section per z-plane.

If the user prefers instead to construct their own axon centerline using standard functionality in Blender, the tools in this section will all function, as long as the centerline is entirely contained inside the axon mesh; extreme precision is not required. The user must simply tell the software about the handmade centerline by clicking the "Update Centerline" button. The only functionality that is lost by working with user-created or user-edited centerlines is that the minimum axonal radius at each centerline vertex will not be exported. This is data provided by VMTK, and is not re-calculated by NeuroMorph.

The number of vertices that define the centerline can be set by the user. This value determines the precision of the rest of the functions in this section. In practice, we generally use a number of vertices on the order of 200, or a vertex spacing of not less than 100 nm. The points are generally not precisely equally spaced along the centerline curve, but there exact spacing is given in the exported data file.

Functionality is also provided to clean the axon mesh as a pre-processing step before further handling. This removes nonstandard geometry such as self-intersections, floating vertices, and other non-manifold geometry that can sometimes result when surfaces are imported from other tools. This mesh cleaning is often necessary in order for other tools, including VMTK, to be able to properly function. Sometimes the input mesh has too many problems, and the tool will delete a large chunk of the mesh (don't worry, every action in Blender can be undone). This is a sign to the user that the mesh should be inspected and modified by hand near the deletion point, possibly by removing some of the problematic regions of the surface and filling in the surface holes with simple faces, an action easily accomplished in Blender. The mesh cleaning function should then be re-run to confirm that the final mesh used for processing is clean. It is recommended to always clean meshes using this tool before they are analyzed, and this functionality has broad utility outside of the context of this suite of tools.

#### 5.1. Cross-Sectional Surface Areas

Once the centerline has been generated, cross-sectional surfaces of the axon can be constructed at every vertex on

<sup>13</sup>http://www.vmtk.org/tutorials/Centerlines.html

FIGURE 8 | (Left) Selecting a region to calculate its surface area and volume. (Right) An open surface, and the result after it is closed by the tool to perform the volume calculation.

the centerline, see **Figure 10**. The cross-sectional surface areas are then calculated along the axon providing a quantitative measure of how the axon's shape changes along its length. This can be used to accurately define the position of axonal boutons, using the bouton detection tool, see section 5.6.

The cross sections are generated perpendicular to the centerline. Therefore, in regions where the centerline is bending

back toward itself with high curvature, the user should note that some cross-sections may intersect. However, the area values for each cross section is correct.

#### 5.1.1. Limiting Cases

If the diameter of the axon is particularly wide, the tool must be told to use a larger plane when performing the planeaxon intersection calculation that produces the cross section. The computation time of this intersection calculation increases with size, so the tool by default uses a moderately sized plane, but the user is able to adjust the diameter of this intersection plane as necessary via a parameter provided in the user interface.

# 5.2. Max Radius of Each Cross Section

This tool calculates the maximum radius of each cross section, as measured from the centroid of the cross section to each of its vertices separately. The centroid is calculated as the average location of all boundary vertices of the cross section, and the intersection point of the centerline with the cross section does not affect this calculation.

Note that for C-shaped cross sections, it is possible that the location of the centroid can technically be outside of the cross section mesh. This does not affect the calculation, and it is up to the user to decide in these cases if the "maximum radius" makes sense.

# 5.3. Project Spheres to Centerline

This function aids in the analysis of the distribution of objects such as vesicles along an axon, see **Figure 11**. The user provides a collection of input mesh objects, such as the vesicles created in section 3.1, which are assumed to be spheres, but are not required to be. Only the centroid of each object is considered. The centroid of each object is defined as the average (x,y,z) location of its boundary surface vertices, and the user should keep in mind that irregularly shaped objects may not be well-represented by the centroid of their vertices.

The distance from each centroid to each distinct vertex on the centerline is computed, and the object is said to be "projected" to the closest vertex. The number of distinct objects projected to each vertex is tallied, and the function returns the final count of projected objects for each vertex. This data will be exported along with all other data for the centerline.

# 5.4. Project Surface Areas to Centerline

This function aids in the analysis of the distribution of surface contact with objects such as synapses along an axon. The user provides a collection of surfaces which do not have to be continuous.

The distance from the centroid of each surface face to each distinct vertex on the centerline is computed. The individual areas of each surface face projecting to each centerline vertex are summed, and the function returns the total projected area

sum for each centerline vertex, see **Figure 12**. This data will be exported along with all other data for the centerline.

## 5.5. Centerline-Based Output

Using the tools in this section, a file can be exported for further analysis that contains some or all of the following data for each centerline vertex:


#### 5.6. Detect Boutons

A final tool is provided that helps the user to define bouton swellings of an axon in a well-defined, reproducible manner, based on certain geometric criteria. Applications of this tool were first reported in Gala et al. (2017).

The user is able to input and experiment with three variables that define the possible beginning and end of a bouton:

> A = Area Change (ratio) D = Distance for Area Change M = Minimum Max Radius

Colored spheres are then placed along the centerline at locations that meet certain geometric constraints based on these values, as follows, see **Figure 13**.

#### 5.6.1. Increasing/Decreasing Cross-Sectional Surface Area

If the cross-sectional surface area is increasing or decreasing by at least a factor of A (A = 2.0 in **Figure 13**) over a distance of D (D = 0.2 **Figure 13**) along the axon, this might indicate the presence of a bouton. (The direction of increase is defined from the lowest to the highest centerline vertex index used in its mesh representation in Blender.)


#### 5.6.2. Large Cross-Sectional Radius

If the maximum radius of the cross section at a vertex is greater than M (M = 0.2 in **Figure 13**), this might indicate the presences of a bouton. The radius is measured from the centroid of the cross section (the average location of all its vertices) to each of its vertices separately, and the maximum radius is defined as the largest of these distances.

• A blue sphere indicates that the maximum radius of the cross section at that vertex is greater than M.

From the possibly many spheres placed along the centerline, the user can select the two that they decide best bound the desired bouton. The tool will then extract the region of the axon between these two points, returning a new bouton object whose volume can be calculated using the NeuroMorph Measurement tools as described in section 4.

# 6. PROXIMITY ANALYSIS

The tools described in this section enable the analysis of regions of two different classes of objects that are close to one another.

# 6.1. Sphere to Surface Distances

This tool computes the shortest distance in 3D from each instance of one class of object, such as the vesicles created in section 3.1, to a given mesh object, such as a synapse surface created in section 3.2, see **Figure 14**. This tool was first developed for use in Barnes et al. (2015).

Each vesicle object is assumed to be a sphere, and only their centroids are used in the calculation. Non-spherical mesh objects will be processed without question, but the user should keep in mind that irregularly shaped objects may not be well-represented by the centroid of their vertices. The distance from the centroid of each vesicle to each vertex on the selected synapse surface is calculated, and the shortest distance found for each vesicle is exported in an output file.

#### 6.2. Interacting Regions

The NeuroMorph toolset also provides functionality to extract the regions of two different objects that lie within a certain distance of each other. Here, we show in **Figure 15** how it can be used to measure the area of apposition between the axonal bouton and an astrocytic process that lies alongside. The functionality could equally be used to analyze the interactions between endoplasmic reticulum and mitochondria.

Given two different objects, or two different classes of objects with pieces joined together into two Blender objects, the Proximity Analysis tool extracts the regions of the objects that are less than a user-defined distance threshold T from each other, see **Figure 15**, and exports the corresponding surface area pairs of each interaction sub-region for analysis.

All interacting sub-regions of the two input objects will be created as child objects of the original input objects. The objects are initially not visible, in order to not clutter the scene, but can be viewed individually using Blender's visibility toggle. All child objects can be made visible together using the NeuroMorph Parent-Child Tools, provided in NeuroMorph's "Other Tools" toolbox.

The algorithm works by determining all vertices on each object that are less than the distance T from a vertex in the other object. A k-d tree (Bentley, 1975), which is an optimized geometric search structure, is used to speed up the processing time of the distance calculations. The vertices are grouped into contiguous units on each object, and paired with their corresponding nearby regions on the other object. The surface areas of each of these distinct sub-regions are calculated.

The output file provides the names of the sub-regions of the first object (e.g., "object1.001"), their surface areas, the corresponding regions of the second object (e.g., "object2.027"), their surface areas, and the centroid of the two regions in order to provide some context for the interaction in space. It also provides the total non-overlapping surface area of each object class, which is generally not the same as the sum of the surface areas of each individual sub-region, see below.

#### 6.2.1. Understanding the Output

#### **6.2.1.1. Regions where surface area = 0**

Edges and vertices that are not part of any faces are cleaned away at the end of the procedure. This means that there may be a region of mesh faces on one object with no corresponding region on the second object, because the corresponding region consisted only of vertices or edges, but no full faces, so had a surface area of 0. If these deleted regions are important to the user, a finer mesh should be provided where entire faces lie within the threshold distance. Remeshing, or simply subdividing faces to result in a finer mesh, is a straightforward operation in Blender<sup>14</sup> .

#### **6.2.1.2. Doubly counted overlapping regions**

The provided results consist of pairs of interacting individual contiguous mesh regions from each object. If a region on one object corresponds to two separate regions on the second object, its surface area will be included in two separate entries in the output file. For this reason, summing the interacting surface areas may result in a greater overall surface area than the true area in space. This may be the desired result, depending on the application. The last line of the output file provides the total non-overlapping surface area for each object.

# 7. DISCUSSION

Serial electron microscopy is now a commonplace technique for exploring cell and tissue structure. Although many methods have appeared in recent years for reconstructing different features from the image stacks (e.g., Morales et al., 2011; Sommer et al., 2011; Cardona et al., 2012; Belevich et al., 2016), few provide any means by which geometric data can be extracted directly from the resulting 3D models. The NeuroMorph tools were primarily constructed, therefore, not as a tool for segmentation or reconstruction, but to allow the user to make detailed measurements of any part of the models. We have integrated these into the Blender software as this open-source platform is arguably the most comprehensive and well-maintained of its type. Its 3D view allows the user to manipulate any part of a mesh while simultaneously viewing the original bitmap images, therefore,

<sup>14</sup>https://docs.blender.org/manual/en/dev/modeling/modifiers/generate/remesh. html

providing the opportunity to further add to the models or make corrections.

Blender's versatility as modeling and visualization software has been exploited for other biological applications, leading to the creation of independent tools developed for this platform. BioBlender (Andrei et al., 2012) was developed as a molecule visualization tool so that molecular models imported from various sources can be viewed, and manipulated. This enables their physical and chemical properties to be included so that their activity can be seen in a realistic way. Similarly, MCell (Kerr et al., 2008) is a Blender-based piece of software into which cellular models can be imported and populated with different molecules that are assigned with their kinetic properties. The software can then carry out particlebased Monte Carlo simulations to understand the molecular diffusion and interactions within biologically relevant cellular geometries.

The NeuroMorph toolset is entirely complementary to these other software packages, giving the user the ability to quantify the geometry. We show here how the different parts of the software can provide details about the morphology from a single glutamatergic axon, but all of this functionality could equally be used to study any other cellular elements represented by mesh models. The computational functionality of NeuroMorph is limited only by the speed and memory of the computer on which it is run. We have successfully tested scenes with many hundreds or thousands of objects on a personal computer. When only the location of an object matters, NeuroMorph also provides a tool to reduce objects to their centroids, so that more objects can be processed.

As the list of algorithms for segmenting different features from serial electron microscopy images grows, tools such as these will become increasingly more in demand as scientists continue to map and quantify cellular environments.

### AUTHOR CONTRIBUTIONS

AJ and GK conceived and planned the development of this software. AJ developed the software which GK and JB tested. JB constructed the models shown and provided with this paper. AJ and GK wrote the paper. GK supervised the project.

# FUNDING

JB is funded by a Swiss National Science Foundation Grant awarded to GK.

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Jorstad, Blanc and Knott. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# SBEMimage: Versatile Acquisition Control Software for Serial Block-Face Electron Microscopy

Benjamin Titze<sup>1</sup> \*, Christel Genoud<sup>1</sup> and Rainer W. Friedrich1,2

<sup>1</sup> Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland, <sup>2</sup> Faculty of Natural Sciences, University of Basel, Basel, Switzerland

We present SBEMimage, an open-source Python-based application to operate serial block-face electron microscopy (SBEM) systems. SBEMimage is designed for complex, challenging acquisition tasks, such as large-scale volume imaging of neuronal tissue or other biological ultrastructure. Advanced monitoring, process control, and error handling capabilities improve reliability, speed, and quality of acquisitions. Debris detection, autofocus, real-time image inspection, and various other quality control features minimize the risk of data loss during long-term acquisitions. Adaptive tile selection allows for efficient imaging of large tissue volumes of arbitrary shape. The software's graphical user interface is optimized for remote operation. In its user-friendly viewport, tile grids covering the region of interest to be acquired are overlaid on previously acquired overview images of the sample surface. Images from other sources, e.g., light microscopes, can be imported and superimposed. SBEMimage complements existing DigitalMicrograph (Gatan Microscopy Suite) installations on 3View systems but permits higher acquisition rates by interacting directly with the microscope's control software. Its modular architecture and the use of Python/PyQt make SBEMimage highly customizable and extensible, which allows for fast prototyping and will permit adaptation to a wide range of SBEM systems and applications.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Kea Joo Lee, Korea Brain Research Institute, South Korea Erik Bloss, Howard Hughes Medical Institute (HHMI), United States

\*Correspondence:

Benjamin Titze benjamin.titze@fmi.ch

Received: 27 April 2018 Accepted: 25 June 2018 Published: 31 July 2018

#### Citation:

Titze B, Genoud C and Friedrich RW (2018) SBEMimage: Versatile Acquisition Control Software for Serial Block-Face Electron Microscopy. Front. Neural Circuits 12:54. doi: 10.3389/fncir.2018.00054 Keywords: SBEM, imaging software, connectomics, serial block-face, SEM, volume EM, microtome, 3View

# INTRODUCTION

The efficient reconstruction of neuronal circuits and other biological ultrastructure by electron microscopy requires fast, reliable, and high-quality acquisition of large volumetric image datasets (Lichtman and Denk, 2011; Denk et al., 2012). Several automated acquisition methods based on scanning or transmission electron microscopy have been developed for this purpose (reviewed in Briggman and Bock, 2012; Peddie and Collinson, 2014; Titze and Genoud, 2016). One approach is to collect series of ultrathin sections on a supporting structure before image acquisition (Hayworth et al., 2006; Schalek et al., 2012), which preserves sections but requires solutions for large-scale section collection and image alignment. Alternatively, a stained and embedded tissue block can be cut with an ultramicrotome inside the vacuum chamber of a scanning electron microscope that is used to image the block-face after each cut (Denk and Horstmann, 2004). This approach, termed serial block-face electron microscopy (SBEM), achieves reliable thin sectioning and requires only minimal alignment of successive images. However, it does not permit repeated imaging of the same tissue since sections are destroyed during the acquisition. It is therefore crucial to

ensure high reliability of data acquisition and to have efficient error detection procedures in place to prevent data loss when running long SBEM acquisitions. Also, to minimize the duration of such acquisitions, it is desirable to efficiently restrict image acquisition to regions of interest. Furthermore, solutions to integrate data acquisition with image post-processing would allow to perform operations such as stitching, alignment, and image analysis while the acquisition is running.

SBEM has been successfully used for ultrastructural imaging of large tissue volumes in the past (Briggman et al., 2011; Wanner et al., 2016a,b; Schmidt et al., 2017, among others), but further software improvements are desired to optimize the acquisition workflow. After the initial development of automated SBEM (Denk and Horstmann, 2004), a commercial system (Gatan 3View) based on the original design was introduced and is now widely used. The 3View microtome and image acquisition process are controlled via the proprietary software DigitalMicrograph (Gatan Microscopy Suite; GMS 2 or 3). This software provides the basic functionality to set up and run acquisitions on 3View systems. However, it does not provide key features such as automatic debris detection that are important to prevent data loss during long acquisitions, and it does not allow for fine-grained control and customization. Furthermore, DigitalMicrograph currently limits the image acquisition rate to 2 MHz. These restrictions prompted the development of SBEMimage as a flexible and powerful open-source acquisition platform for SBEM/3View systems.

SBEMimage provides essential quality control features for long-term acquisitions. When running an acquisition continuously over days, weeks or even months, the following problems can occur: (1) debris falling on the sample surface and obscuring the region of interest; (2) focus and astigmatism drifts or jumps; (3) electron beam instability; (4) malfunctions of the (proprietary) control software (here: DigitalMicrograph and SmartSEM); (5) I/O errors such as disk or network drive unavailability for writing image data; (6) hardware failures (stage motors, SEM cathode, backscattered electron detector, vacuum system, power). With these issues in mind we designed SBEMimage to provide improved error handling capabilities, more extensive monitoring functions, and other features that enhance stability and reduce user interaction time. An automatic debris detection and removal mechanism and a reliable autofocus function combined with slice-by-slice tile monitoring solve problems (1), (2), and (3). Error detection procedures pause the acquisition when errors of type (4), (5), or (6) occur that cannot be resolved by the software. Additional features allow for imaging of volumes with complex geometries and for the integration of image analysis procedures into the acquisition process. A flexible and user-friendly graphical interface minimizes the risk of human error when setting up and running acquisitions. The highest priority during development was to prevent data loss in case of major failures.

SBEMimage is an open-source project, released on GitHub under the MIT License, and intended as a free-touse community-supported resource. The code repository, installation instructions, and further documentation can be found on GitHub<sup>1</sup> . The software was developed with a modular architecture, and the source code is fully commented, which should allow Python programmers to easily customize it and add new functionality.

# IMPLEMENTATION

We implemented SBEMimage in Python (version 3.6), a highlevel programming language that is widely used in the scientific community. The toolkit PyQt 5 was used for the graphical user interface. SBEMimage is currently designed to operate a 3View microtome combined with a ZEISS Merlin microscope. It interacts with two pieces of existing proprietary software: (1) The microscope control software SmartSEM, which must be installed on the EM server PC, and (2) DigitalMicrograph (Gatan Microscopy Suite 2 or 3), which runs on a support PC. Together, these two applications control the SBEM system in the conventional configuration (**Figure 1A**).

Acquiring images in DigitalMicrograph with DigiScan (Gatan's scan generator) limits the acquisition rate to 2 MHz. To overcome this limit, SBEMimage acquires images via SmartSEM, which permits acquisition rates of up to 40 MHz. This approach requires an adapter that connects the amplified BSE detector output to one of the microscope's acquisition boards. Details are provided on the GitHub page.

To achieve maximum flexibility, we first had to find a way to control all relevant low-level functions of the 3View system through Python commands. For the ZEISS Merlin and other ZEISS microscopes, a powerful API already exists: the SmartSEM Remote API, developed by Carl Zeiss Microscopy, implemented as an ActiveX control, with test programs available in C++, C#, and Visual Basic. We built a wrapper module that allows all relevant commands of that API to be used in a Python application. This module may be adapted in the future to allow SEMs from other manufacturers to be controlled with the same SBEMimage Python commands.

DigitalMicrograph offers an internal scripting language that provides a number of commands to control the 3View stage and the microtome's knife. However, there is no publicly accessible and documented API that lets programs running outside of DigitalMicrograph use these functions. Our solution was to create a communication script that runs in DigitalMicrograph and enables the exchange of commands and parameters with external programs. This information exchange is achieved through reading and writing files. In 0.1-s intervals, the DigitalMicrograph script checks for the existence of a trigger file. If this trigger file is detected, the script reads an input file that contains a command and up to two parameters. In this way, SBEMimage can send commands to DigitalMicrograph and read return values. Through the SmartSEM remote API and the DigitalMicrograph communication script, SBEMimage can thus control all relevant low-level functions of the SBEM system (**Figure 1B**).

<sup>1</sup>https://github.com/SBEMimage

FIGURE 1 | (A) Conventional setup of the 3View system. The software SmartSEM (Carl Zeiss Microscopy, Cambridge, United Kingdom), running on the EM Server PC, controls the scanning electron microscope (SEM). The software DigitalMicrograph (Gatan), running on a support PC, controls the 3View hardware (diamond knife and motorized stage). DigitalMicrograph can also indirectly control the SEM via SmartSEM. (B) SBEMimage interacts with SmartSEM through a proprietary remote API, provided by Carl Zeiss Microscopy, and with DigitalMicrograph through a custom-written communication script. SBEMimage can thus control all relevant low-level functions of the SBEM system and exert full control over the acquisition process.

## DESCRIPTION OF THE USER INTERFACE

The graphical user interface consists of two windows, by default positioned next to each other on a wide screen (**Figure 2A**). The interface was designed with remote desktop software such as TeamViewer and VNC in mind: All functions (including basic SEM operations such as turning on the high voltage and focusing the beam) are accessible on a single screen.

The window "Main Controls" on the right displays at a glance all relevant settings, the acquisition status, the current electron dose, and real-time estimates for the duration of the acquisition and the storage size of the dataset. Various dialogs let the user set up all acquisition parameters and, when using the program for the first time, perform a motor speed and stage calibration. Calibrating the stage is necessary because the X and Y motor axes are rotated and scaled with respect to the SEM coordinate system (**Supplementary Figure S1B**). Additional tabs of the "Main Controls" window contain a novel tool for manual focusing (explanation below) and various functions for testing and debugging.

The window "Viewport" on the left lets the user set up and monitor acquisitions visually. The workspace shown in the viewport's main tab covers the entire accessible range of the stage motors. When sufficiently zoomed out, the stage boundaries are shown as solid white lines, and the X and Y stage axes as dashed white lines. To obtain an overview of the entire surface of the sample holder ("stub") mounted on the 3View stage, the user can acquire a "stub overview image," a large low-resolution (372 nm pixel size) mosaic of specified size that is displayed in the workspace as the main background image (**Supplementary Figure S1A**).

The user can use this stub overview image to comfortably locate the region of interest and navigate there to acquire smaller overview images at higher resolution (typically 100–200 nm pixel size). To acquire image tiles at the target resolution for analysis (typically 5–20 nm pixel size), the user can set up a tile grid in the region of interest. Grid size, tile size, overlaps/gaps between tiles, and acquisition parameters (frame size, pixel size, and dwell time) are specified for each grid. Tiles can be individually selected or deselected for imaging and the whole grid can be shifted when necessary. For complex acquisition tasks, multiple overviews can be set up to cover the region(s) of interest, and multiple grids can be created with different imaging parameters. The user can choose for each overview image and for each grid whether it should be acquired on every slice, or in intervals. This permits, for example, to image a region of interest with alternating pixel sizes from one slice to the next, or to acquire an overview stack at low resolution with a high-resolution mosaic on every tenth slice.

The basic elements described above are displayed as different layers inside the viewport. The background layer consists of the stub overview image, which provides the main reference frame for an acquisition. The layer above contains the overview images that cover the regions of interest. They are primarily used for debris detection and to position the tile grids. The tile grids are placed in the next layer above the overview images. Finally, additional imported images (see feature description below) are shown in the foreground, typically with a transparency setting that allows the user to see through these images. Users can freely position all tile grids, overviews, and imported images within the accessible motor range and choose whether to show or hide them.

The visual scene can be panned by left-click dragging, and zoomed in and out with the mouse wheel or a zoom slider. All elements can be selected and edited with mouse clicks and dragged to new positions. The viewport is fully functional even while an acquisition is running.

#### BRIEF DESCRIPTION OF FEATURES

In the following, we have summarized the key features of SBEMimage and a few additional features that may be of interest

FIGURE 2 | SBEMimage user interface. (A) The default window arrangement during a stack acquisition. Recommended screen resolution is 1920 × 1080. Left window: Viewport showing an overview image and two tile grids. The highlighted tiles have been selected for imaging. A low-resolution stub overview mosaic is displayed in the background. The right window contains the main controls: Settings are shown for the SEM and the microtome, the current overview image and one of the tile grids. In the lower part of the window, all activity is logged. In the stack acquisition panel, users can select or deselect features to be used during the acquisition and watch the progress of the acquisition. Electron dose range and duration/storage estimates are shown on the right side of the panel. (B) The slice-by-slice viewer lets the user view recently acquired images at full resolution. With the mouse wheel, the user can scroll through consecutive slices to assess cutting and image quality. (C) In the statistics tab, the user can select an overview image or a tile, for which the current reslice, the histogram, and mean/SD plots are shown. All errors, warnings, and debris events are logged in the lower right part of this tab.

for special applications. Users can select before and during an acquisition which features should be active while the acquisition cycle (**Figure 3**) is running.

### Debris Detection and Removal

When the debris detection option is activated, each newly acquired overview image is compared to the one on the previous

slice to check if debris is present. To initiate this process, the user is asked to confirm that the very first overview image at the beginning of a stack does not contain debris. Three different methods (based on comparisons of mean/SD values, histograms, and pixel differences, respectively; for details, refer to the GitHub repository) are currently available to detect debris on the surface. The default detection procedure divides the region of interest into four quadrants and compares the pixel mean and standard deviation in each quadrant between successive images. If the maximum difference across all four quadrants exceeds a user-defined threshold, the program assumes that debris is present. In this case, SBEMimage performs a "sweep" (first described in Helmstaedter et al., 2013): After the sample is lowered by 70 nm (or a different amount as specified by the user), the knife is moved across the sample surface in an attempt to push the debris away. The process can be repeated until the user-specified maximum number of sweeps has been reached. Depending on the option settings, the program will then either pause the acquisition or continue regardless of the debris. Detection parameters should be finetuned for a given sample and imaging settings to achieve optimal results. In our hands, the automated procedure detected and removed all medium- to large-size chunks of debris (>10 µm). The detection and removal of very small flakes of debris on the order of microns or smaller may, however, be less efficient, depending on sample properties, imaging parameters, and residual charging.

# Focus and Stigmation Control

For remote focusing with TeamViewer or VNC, SBEMimage offers a **focus tool** that acquires a 512-pixel × 384-pixel throughfocus series on a specified tile or overview image (**Figure 4A**). The user can then choose the optimal focus setting from this image series. Optimal X/Y stigmation settings are chosen with the same procedure. For refinement, the cycle can be repeated. This focus tool offers two advantages: (1) It can be used remotely (when no adjustment knobs and no access to the EM server PC are available) and even at slow connection speeds of the remote desktop software; (2) The procedure minimizes the electron dose, at least in case of small focus corrections, since each image series is acquired only once, whereas manual refocusing with the microscope software relies on continuous scanning. The focus tool can also be used to set working distances for individual tiles, which is needed for setting up an **adaptive focus for tilted surfaces**. A slight tilt of the 3View knife is usually unavoidable, which for very large distances across the sample surface can lead to a difference in working distance: One part of the region of interest may be well focused, but a more distant part will be out of focus. This also typically occurs when mounting a sample in the 3View holder that will not be cut. Any initial tilt of the surface will thus not be physically corrected by the blade. For these cases, SBEMimage offers a gradient correction mechanism (**Figure 4B**). To correct the focus during an acquisition, two **autofocus** methods are implemented in SBEMimage: Method (1) uses the built-in SmartSEM autofocus/stigmator, which is called in regular intervals, as specified by the user. The user can choose the reference position(s) where the focus/stigmation procedure should be performed, and decide if both autofocus and autostigmation are to be performed on the same slice or a specified number of slices spaced apart to minimize beam exposure. Method (2) uses a continuous heuristic autofocus procedure based on autocorrelation. The algorithm works on tiles that have already been acquired and applies corrections continuously. This approach was first used in Briggman et al. (2011) and is described in appendix A of Binding et al. (2013).

# Adaptive Tile Selection

Within each grid, tiles can be selected as "active," or deselected, with single mouse clicks. Only active tiles are acquired. Deselecting tiles outside the region of interest therefore makes imaging non-rectangular regions of interest more efficient (**Figure 5A**). When debris detection is used, SBEMimage can automatically adjust the detection region to cover only the region of the overview image that contains active tiles. The active tile pattern can be adjusted by the user while an acquisition is

FIGURE 4 | (A) The focus tool (the second tab in the main controls window) allows the user to remotely focus individual tiles or overview images and adjust the stigmator settings. A brief explanation is provided in the main text. (B) For imaging tilted surfaces, SBEMimage offers a focus gradient correction feature. When the optimal focus for three tiles on a given grid is specified (see drawing in the dialog window), SBEMimage computes the gradient and adjusts the focus (working distance) automatically for every tile in the grid. In the viewport, the different working distances for each tile are displayed below the tile number.

running. Grids can be shifted when the acquisition is paused. When an error occurs or the user pauses the acquisition, SBEMimage remembers which tiles on the current slice have already been acquired and will resume the acquisition at the correct position in the grid.

#### Slice-by-Slice Tile Monitoring

On each slice, the mean and standard deviation of a selected tile is compared to the mean and standard deviation of the same tile in the previous slice. If the differences exceed userdefined thresholds, the acquisition is paused. This approach permits detection of sudden unexpected changes (loss of focus, darkening, and blank images). In the second tab of the viewport window (**slice-by-slice viewer**), the user can load the most recently acquired tiles and overviews into memory and view them at full resolution (**Figure 2B**). Panning and zooming work as in the viewport. The mouse wheel and control buttons let the user go backward and forward through the acquired slices to check if the cutting is regular and to assess image quality. Distances in the images can be measured with a ruler tool (also available in the viewport). The viewport offers **in situ tile previews** during acquisitions. A preview image (512 pixels × 384 pixels) is generated from each tile as soon as it is acquired. This preview image is then immediately displayed in the tile grid, such that the user can inspect it in its relative position to the other tiles. To check whether the specified overlaps are sufficient and the stage calibration is accurate, tile previews can be shown either in "overlap mode," where they are placed at the exact stage positions where they were acquired, or in "gap mode," where artificial gaps are put between the tiles, which is useful to verify the alignment and the width of overlapping areas.

### Image Inspection and Selection

After each image acquisition, the module ImageInspector reads the acquired image file from disk into memory and performs a number of image integrity and quality checks. If a check fails, the user is alerted via e-mail and the acquisition is paused automatically. It is also possible to test incoming images for other user-defined features and either select or discard them on the basis of these tests. This feature can save storage space and also time since it can be exploited to carry out some data processing and pre-selection operations already during the acquisition.

# Data and Metadata Handling

Images are first saved on a primary drive (usually the local hard disk or SSD of the support PC; see **Supplementary Figure S2** for SBEMimage's folder structure and file name conventions). The user can specify a second local or network drive as a **mirror drive**, where the acquired image data and the metadata are mirrored during the acquisition. This feature provides a backup solution and makes the data available for post-processing while the acquisition is running. For seamless integration into an image post-processing pipeline, SBEMimage **metadata** can be exported in the TrakEM2 image list format (Cardona et al., 2012). Metadata can also be transferred to a remote server while the acquisition is running. SBEMimage integrates with the Volume Image Environment (VIME; Gerhard et al., in preparation<sup>2</sup> ), which allows for the visualization and post-processing of images in real-time as they are acquired by SBEMimage. A communication protocol between SBEMimage and VIME lets users implement a flexible remote quality control system. In case of problems with post-processing of the acquired image data – for example when stitching is not possible due to insufficient overlap – the acquisition can be paused remotely by VIME and the user is notified.

# Error Handling

When an error is detected, SBEMimage in most cases makes a second or third attempt to carry out the failed operation and

<sup>2</sup>https://github.com/vime

writes a warning message into the log. If the second/third attempt also fails, the acquisition is paused and the user is notified via e-mail. Error codes have three digits and are grouped according to the first digit: (1) errors related to communication with DigitalMicrograph; (2) errors related to 3View/SBEM hardware; (3) errors related to SmartSEM/SEM; (4) I/O errors; (5) errors related to the acquisition process and image inspection; (6) userdefined errors. Detailed information about the entire acquisition process including all warnings and errors is saved in the main log.

#### Additional Features Configurations for Multi-User Multi-Project Environments

All system settings, calibrations, acquisition parameters, and workspace options are stored in configuration files. This allows each user to maintain his or her own configuration and to work on different projects on the same system.

#### E-Mail Monitoring and Control

In user-specified intervals, a status report is sent via e-mail. The user can customize the content of the report (screenshots, log files, images, and reslices). When a critical error occurs, SBEMimage immediately sends an e-mail to alert the user. The user can also send commands to SBEMimage via e-mail, for example to pause an acquisition remotely when TeamViewer or VNC are unavailable.

#### Importing Overview Images

Existing images can be imported into the workspace at a variable pixel size, stage position, rotation angle, and transparency, which is of special interest for correlative light and electron microscopy (CLEM). For example, light microscopic images can be loaded and superimposed on EM images of the same tissue to align cell bodies or other structures.

#### Real-Time Reslices, Histograms, and Statistics

The z-reslice image for a user-selected overview or tile, its histogram, and its time course of mean and standard deviation measurements are shown in the third tab of the viewport window (**Figure 2C**). By clicking on the plots, the user can select a past slice along the time axis, for which the histogram and mean/SD values will be displayed.

#### Plasma Cleaner Control

SBEMimage includes a module to control the downstream asher GV10x (ibss Group Inc., Burlingame, CA, United States) to clean the inside of the microscope's vacuum chamber and the surface of the BSE detector.

# SUMMARY AND OUTLOOK

With the development of SBEMimage we have addressed key problems encountered frequently in SBEM acquisition projects. The software has been tested extensively and used in multiple projects including long-term acquisitions. One example is a 6 week acquisition of an image stack covering an entire specimen of Platynereis dumerilii at a voxel size of 10 nm × 10 nm × 25 nm (**Figure 5**). This dataset comprises >200,000 image tiles that were unevenly distributed over 11,416 slices (approximately 18 tiles per slice on average). The efficient acquisition of this

dataset depended critically on two SBEMimage features, debris detection and adaptive tiling. Debris was detected and removed on 493 slices. Hence, a substantial fraction of slices would have been compromised at least locally without automated debris detection. Adaptive tile selection allowed us to acquire data only from those tiles that contained sample tissue. The entire Platynereis specimen was contained within a bounding cuboid of approximately 275 µm × 260 µm × 285 µm. However, because the specimen was tilted and had a complex shape, image data had to be acquired from only ∼30% of this cuboid. Adaptive tiling therefore saved a substantial amount of time and resources for data acquisition, post-processing, and storage.

The release of SBEMimage on GitHub<sup>3</sup> allows potential users to test it on their systems and adapt it to their needs. The software currently supports Gatan 3View microtomes and ZEISS SEMs. Operation of SBEMimage with devices from other manufacturers will require the adaptation of SBEMimage modules with devicespecific code for low-level functions. The development of SBEMimage was inspired by a previous open-source microscopy project that has been highly successful: ScanImage, a widely used application for operating laser scanning microscopes (Pologruto et al., 2003).

A promising future application of SBEMimage are "datadriven acquisitions," where image data is used in real-time by the program to algorithmically determine what to acquire next. SBEMimage offers an ideal framework to implement such an approach. For example, machine learning could be used to detect tissue boundaries in overview images. For a given volume to be acquired, the user would specify the starting grid configuration and select all the tiles needed to cover the tissue. The algorithm would then decide from slice to slice whether to shift or expand the tile pattern to follow the tissue through the sample.

Another potential application is sparse and selective imaging of tissue volumes. For many SBEM applications, only a fraction of the acquired data is actually needed for further analysis. As described above, substantial time can be saved by adaptive tile selection to avoid imaging irrelevant parts of the sample. Beyond that, imaging may be restricted to specific objects that are of interest to a user, such as the dendritic tree of a specific neuron. For such applications, SBEMimage provides a framework to detect these features in real-time during acquisitions or to exclude images in which the existence of these features can be

3 https://github.com/SBEMimage

#### REFERENCES


ruled out. Such inspection algorithms can be easily incorporated in SBEMimage's acquisition cycle to select a given tile for registration or to discard it.

### AUTHOR CONTRIBUTIONS

BT developed the software. CG performed beta-testing and provided detailed feedback and suggestions. BT and RF wrote the manuscript.

# FUNDING

This work was supported by the Novartis Research Foundation and by the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation Programme (Grant Agreement No. 742576).

## ACKNOWLEDGMENTS

We thank Adrian Wanner, Alexandra Graff-Meyer, Cameron Cowan, Catherine Maclachlan, and Graham Knott for user feedback and beta-testing of the software; Stephan Gerhard for implementing the remote server infrastructure to receive real-time metadata; Stuart Searle for assistance with Digital Micrograph scripting; Hidde Wallaart for technical support with the SmartSEM API; Adrian Wanner and Cameron Cowan for comments on the manuscript.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fncir. 2018.00054/full#supplementary-material

FIGURE S1 | (A) The viewport's workspace, zoomed out to show a stub overview image that covers the entire motor range of the stage. (B) The stage axes form a coordinate system that is rotated and scaled with respect to the SEM coordinate system. SBEMimage provides a calibration routine. Note that the motors move on slightly curved paths; the straight stage axes shown are an approximation.

FIGURE S2 | Folder structure of a SBEMimage dataset and naming conventions for overview and tile image files. All images are saved as 8-bit greyscale TIFFs.

reconstruction. PLoS One 7:e38011. doi: 10.1371/journal.pone.003 8011



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Titze, Genoud and Friedrich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Fast Homogeneous En Bloc Staining of Large Tissue Samples for Volume Electron Microscopy

Christel Genoud<sup>1</sup> \*, Benjamin Titze<sup>1</sup> , Alexandra Graff-Meyer <sup>1</sup> and Rainer W. Friedrich1,2

*<sup>1</sup> Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland, <sup>2</sup> Faculty of Natural Sciences, University of Basel, Basel, Switzerland*

Fixation and staining of large tissue samples are critical for the acquisition of volumetric electron microscopic image datasets and the subsequent reconstruction of neuronal circuits. Efficient protocols exist for the staining of small samples, but uniform contrast is often difficult to achieve when the sample diameter exceeds a few hundred micrometers. Recently, a protocol (BROPA, brain-wide reduced-osmium staining with pyrogallol-mediated amplification) was developed that achieves homogeneous staining of the entire mouse brain but requires very long sample preparation times. By exploring modifications of this protocol we developed a substantially faster procedure, fBROPA, that allows for reliable high-quality staining of tissue blocks on the millimeter scale. Modifications of the original BROPA protocol include drastically reduced incubation times and a lead aspartate incubation to increase sample conductivity. Using this procedure, whole brains from adult zebrafish were stained within 4 days. Homogenous high-contrast staining was achieved throughout the brain. High-quality image stacks with voxel sizes of 10 × 10 × 25 nm<sup>3</sup> were obtained by serial block-face imaging using an electron dose of ∼15 e−/nm<sup>2</sup> . No obvious reduction in staining quality was observed in comparison to smaller samples stained by other state-of-the-art procedures. Furthermore, high-quality images with minimal charging artifacts were obtained from non-neural tissues with low membrane density. fBROPA is therefore likely to be a versatile and efficient sample preparation protocol for a wide range of applications in volume electron microscopy.

Edited by:

*Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan*

#### Reviewed by:

*Yun-Qing Li, Fourth Military Medical University, China Tom Reese, National Institute of Neurological Disorders and Stroke (NINDS), United States Jon I. Arellano, Yale School of Medicine, Yale University, United States*

#### \*Correspondence:

*Christel Genoud christel.genoud@fmi.ch*

Received: *20 June 2018* Accepted: *29 August 2018* Published: *28 September 2018*

#### Citation:

*Genoud C, Titze B, Graff-Meyer A and Friedrich RW (2018) Fast Homogeneous En Bloc Staining of Large Tissue Samples for Volume Electron Microscopy. Front. Neuroanat. 12:76. doi: 10.3389/fnana.2018.00076* Keywords: EM, protocol, BROPA, connectomics, SBEM, block-face, zebrafish, sample preparation

# INTRODUCTION

Volume electron microscopy (volume EM) is currently the only imaging approach that enables dense reconstructions of neuronal circuits. A current goal for large-scale projects is the acquisition of high-resolution image data from volumes up to 1 mm<sup>3</sup> , which may be achieved by different approaches that rely on automated sectioning and imaging (Briggman and Bock, 2012; Denk et al., 2012; Titze and Genoud, 2016). One strategy is to collect sections on a support for subsequent imaging in a scanning EM (SEM) or in a transmission EM (TEM). Alternatively, stacks of images may be acquired by serial block-face scanning electron microscopy (SBEM), a technique that repeatedly cuts and images the sample block-face in an SEM (Denk and Horstmann, 2004). For both approaches, unsliced tissue blocks must be fixed and impregnated with heavy metals. Efficient methods for en bloc staining of EM samples on this size scale are thus of key importance for connectomics.

En bloc staining methods for connectomics should ideally achieve uniform and strong impregnation of membranes with heavy metals throughout large sample blocks. This has been achieved by the rOTO (reduced osmium—thiocarbohydrazide osmium) protocol (Seligman et al., 1966; Malick and Wilson, 1975) and its recent adaptations and modifications (Deerinck et al., 2010; Tapia et al., 2012a). The rOTO protocol achieves high membrane contrast, but staining intensity decreases with depth (Hua et al., 2015), presumably because the penetration of reagents is inefficient. Acceptable staining can usually be achieved up to 200µm below the tissue surface, but staining of thicker samples remains challenging. Hence, new approaches are required for high-contrast staining of samples containing large neuronal circuits.

Recently, two protocols for efficient staining of larger samples have been introduced. The first protocol is based on modifications of the osmium steps and produced homogeneous and strong staining in 1 × 1 × 1 mm<sup>3</sup> blocks of mouse brain tissue (Hua et al., 2015). However, we obtained variable results when applying this protocol to the brain of adult zebrafish, which is difficult to impregnate because densely packed somata and meninges form strong diffusion barriers. The second protocol, referred to as BROPA, was developed to stain entire mouse brains and uses different reagents such as formamide and pyrogallol (Mikula and Denk, 2015). Uniform staining of an entire mouse brain requires very long incubation times that result in a total protocol duration of 2–3 months. We therefore explored the possibility to modify this protocol to achieve faster staining of smaller samples.

We developed a modified BROPA protocol, referred to as "fast BROPA" (fBROPA), that achieves strong and uniform staining of samples on a millimeter scale. The procedure takes advantage of the reagents used in the BROPA protocol but uses drastically shorter incubation times and contains additional modifications. A lead aspartate (Walton, 1979) incubation step substantially increased the conductivity of the sample, which greatly facilitated SBEM imaging of sample blocks. The protocol does not include uranyl acetate, thus resolving concerns about radiation safety (Odriozola et al., 2017). The time required for the complete fBROPA staining procedure (four days) is similar to the duration of other protocols such as rOTO. We tested fBROPA on different samples including tissue from the adult zebrafish brain and mammalian intestinal organoids using SBEM. In all samples, fBROPA produced uniform staining with high contrast and conductivity. We therefore conclude that fBROPA is a promising staining method for volumetric EM applications in connectomics and other fields.

#### RESULTS

The goal of this study was to develop an en bloc staining protocol for reliable staining of samples on the millimeter scale. Our starting point was the BROPA protocol (Mikula and Denk, 2015), which has been developed for larger samples. In order to adapt it to smaller samples we first used the same reagents and procedures but reduced incubation times by a constant factor. Protocols were then used to stain entire brains of adult zebrafish, which have a maximal diameter of >1 mm. Brains were imaged in an SEM (Zeiss Merlin or FEI Quanta 200 FEG) in low or high vacuum. Under these conditions, intense staining with heavy metals is required to obtain high-contrast images. Moreover, highly conductive samples are required to prevent charging in high vacuum.

We first reduced all incubation times of the original BROPA protocol by a scaling factor given by the approximate length ratio of the zebrafish brain and the mouse brain, which resulted in a total duration of 2 weeks for the complete protocol. However, this approach was not successful. Although we used a fixation procedure that is known to preserve ultrastructure very well (Briggman and Denk, 2006; Deerinck et al., 2010; Tapia et al., 2012b; Hua et al., 2015; Mikula and Denk, 2015) brains showed obvious signs of damage. Staining was poor, membrane integrity was not preserved, and broken nuclear envelopes were observed (**Figure 1A**). Moreover, the tissue contained large empty spaces and was not sufficiently conductive to obtain high-quality SBEM images in high vacuum (**Figure 1B**).

We then varied incubation times to optimize conditions. Surprisingly, we found that shorter incubation times resulted in more intense and more homogeneous staining. Moreover, shorter incubation times eliminated obvious signs of damage and dramatically improved the preservation of tissue ultrastructure (**Figure 2**). To further optimize the protocol we varied the following steps and systematically analyzed staining in the telencephalon of the adult zebrafish brain:


sample conductivity in the rOTO protocol. Consistent with this observation, we found that lead aspartate substantially increased conductivity of samples prepared by fBROPA. This step was necessary to acquire stacks of SBEM images in high vacuum.


The final protocol for fBROPA is described in detail in Materials and Methods and consists of the following main steps:


We used the zebrafish telencephalon to optimize the protocol because pilot experiments had shown that other sample preparation procedures often failed to produce strong and uniform staining in this brain area. One possible explanation for this observation is that the ventricle stretches as a thin sheet over the dorsal telencephalon and hinders diffusion of reagents into the tissue. Nevertheless, fBROPA resulted in uniform high-contrast staining throughout the zebrafish forebrain (**Figure 2**). Contrast and signal-to-noise ratio of images taken deep below the surface appeared indistinguishable from superficial images (**Figures 2B,C**). In some cases, contrast was even higher in deep regions as compared to superficial regions. Hence, fBROPA allows for efficient staining of large samples.

To corroborate this conclusion we analyzed sample blocks that were cross-sectioned through the optic tectum of adult zebrafish where the diameter of the brain is maximal. The diameters of these cross-sections were ∼1.1 and 0.8 mm along the long and short axes, respectively. Homogeneous staining was observed throughout (**Figure 3A**). High-resolution images of sub-regions in different locations demonstrated that contrast was uniformly high (**Figures 3B–D**). As observed with related protocols for en bloc staining of large volumes, synaptic vesicles could be clearly resolved while staining of postsynaptic densities was not prominent (**Figure 3E**). Images with high signal-to-noise ratio could be obtained at all locations using image acquisition

FIGURE 2 | Application of fBROPA to the adult zebrafish brain. (A) Coronal section through the telencephalon of adult zebrafish at the level of Dp (posterior zone of the dorsal telencephalon). Note homogeneous staining. Black particles outside the tissue are silver particles in the surrounding resin to optimize conductivity. (B) Neuropil close to the surface. (C) Neuropil 300µm below the surface.

parameters that are typically used in high-throughput SBEM (1.5 keV landing energy, 15 e−/nm<sup>2</sup> , high vacuum).

We next applied fBROPA to intestinal organoids that are small in volume compared to the zebrafish brain. However, these samples have a lower membrane density than brain tissue and contain relatively large sub-volumes devoid of cells. Organoid samples therefore accumulate less osmium and tend to be less conductive than brain samples. As a consequence, organoid samples are prone to charging effects and present a challenge for high-quality imaging using SBEM. We found that organoid samples prepared using fBROPA allowed for acquisition of high-resolution image stacks (3 keV landing energy, 3 nm pixel size, 50 nm section thickness; **Figure 4**). Charging artifacts were minimal and image quality was similar to that obtained in other samples. The acquisition of high-quality stacks at low section thickness (≤30 nm) was not possible when the incubation in lead aspartate was omitted because charging effects became too strong. These observations show that fBROPA allows for the preparation of volumetric EM samples with high contrast and conductivity from different biological sources.

## DISCUSSION

We developed fBROPA, an en bloc staining method for volume EM. fBROPA is based on similar reagents and procedures as BROPA but uses substantially shorter incubation times, resulting in a total duration of four days that is similar to that of other en bloc staining protocols. Moreover, fBROPA includes an additional lead aspartate incubation step that substantially increases sample conductivity.

Compared to the well-established rOTO protocol, fBROPA substantially extends the volume of homogeneous staining to the millimeter range without an obvious loss in staining intensity, contrast or conductivity. The appearance of synapses is similar as in rOTO-stained tissue, with distinct synaptic vesicles and lightly stained postsynaptic densities. Homogeneous highcontrast staining of cubic millimeter samples has also been achieved by a modification of the original rOTO protocol (Hua et al., 2015). This protocol has been applied successfully to the rodent neocortex. fBROPA now provides a complementary option for en bloc staining of large samples that has been applied successfully to different parts of the zebrafish brain and to organoid samples. These samples present different challenges for staining procedures including diffusion barriers and low membrane density. fBROPA resulted in high contrast and conductivity in all of these samples, indicating that it may be applicable to a wide range of biological specimens. Hence, fBROPA extends the range of available options for en bloc staining of large EM samples. Future studies may thus further explore applications of fBROPA and compare it to alternative protocols.

Recent developments in 3D EM technology substantially increased the speed of data acquisition without loss in image quality. As a consequence, the acquisition of high-resolution image stacks covering volumes as large as a cubic millimeter appears realistic in the near future. Ultrastructural imaging

dissection and is unrelated to fixation or staining.

of such volumes can enable the reconstruction of important neuronal circuits such as entire neocortical columns. We therefore assume that fBROPA will become a valuable method for large-scale connectomics and neuronal circuit reconstruction. Moreover, fBROPA has the potential to facilitate various applications of volumetric EM in other scientific domains.

#### MATERIALS AND METHODS

#### Reagents

Fixative: 2.5% wt/vol glutaraldehyde (16400, Electron Microscopy Sciences [EMS]) in 0.1 M cacodylate buffer (Sigma CO250) with 4% sucrose (Sigma S9378), pH 7.4. Use fresh fixative (within less than 4 hours after preparation). Add 3 ml (2.5% wt/vol) of freshly opened glutaraldehyde (25% vol/vol aqueous solution) to 15 ml 0.2 M cacodylate buffer with 1.2 g sucrose. Correct pH to 7.4, and then fill up to 30 ml with double-distilled water (ddH2O).

Cacodylate buffer (0.2 M stock solution): Prepare 100 ml of 0.4 M cacodylate buffer (8.56 g to 100 ml of ddH2O) and add ∼10.8 ml of 0.2 M HCl. Adjust the pH to 7.4 and fill up to 200 ml with ddH2O. Can be stored at room temperature.

OsO4/K4Fe(CN)<sup>6</sup> solution: 40 mM OsO<sup>4</sup> (EMS 19110) with 35 mM K4Fe(CN)<sup>6</sup> (Sigma Aldrich 60280) in 0.1 M cacodylate buffer and 2.5 M formamide (Sigma 47670). Add 2.5 ml of 4% aqueous osmium tetroxide (stock solution prepared at least 24 h in advance by dissolving 2 g of osmium tetroxide crystals in 50 ml of ddH2O) to 5 ml of 0.2 M cacodylate buffer. Add 0.15 g of K4Fe(CN)<sup>6</sup> (0.147812 g for 35 mM) and 1.125 ml of formamide. Fill up to 10 ml with ddH2O.

OsO<sup>4</sup> solution A: 40 mM OsO<sup>4</sup> (EMS) in 0.1 M cacodylate buffer. Add 2.5 ml of 4% aqueous osmium tetroxide (stock solution prepared at least 24 h in advance by dissolving 2 g of osmium tetroxide crystals in 50 ml of ddH2O) to 5 ml of 0.2 M cacodylate buffer. Fill up to final volume of 10 ml with ddH2O.

FIGURE 4 | Application of fBROPA to early differentiated organoids (two cells). Insert shows details of the membrane ultrastructure between the two cells.

OsO<sup>4</sup> solution B: 40 mM OsO<sup>4</sup> (EMS) in ddH2O. Prepare 2.5 ml of 4% aqueous osmium tetroxide (stock solution prepared at least 24 h in advance by dissolving 2 g of osmium tetroxide crystals in 50 ml of ddH2O) and fill up to final volume of 10 ml with ddH2O.

Pyrogallol (Sigma 16040), 320 mM, pH 4.1, unbuffered: Solution can be used for up to 6 weeks after preparation. Add 2.5 ml of 1.28 M stock pyrogallol solution (stock solution is obtained by adding 40.35 g in 250 ml ddH2O and stored in the dark at 20◦C) in 7.5 ml ddH2O.

Walton's lead aspartate: Ensure that all solutions are kept at 60◦C in a water bath and that pH is measured at this temperature. Mix 0.040 g aspartic acid (Sigma A9256) in 10 ml ddH2O and bring the solution to 60◦C. Then add 0.066 g lead nitrate (EMS 17900) and let the solution stabilize at 60◦C. Adjust pH to 5.5 (at 60◦C) with 1M NaOH (∼350 µl). Keep solution at 60◦C throughout.

Uranyl acetate solution (1% wt/vol in ddH2O): Add 0.2 g of uranyl acetate powder to 20 ml of ddH2O, and agitate gently until solution is transparent.

Durcupan resin (Knott et al., 2009): Mix 33.3 g of resin A/M (Sigma 44611), 33.3 g of hardener B (Sigma 44612) and 1 g of hardener D (Sigma 44614) into a plastic pouring flask. Stir continuously with magnetic stirrer for at least 30 min. Add 16 drops of DMP-30 (tris-(dimethylaminomethyl)phenol, EMS 13600), and stir for a further 10 min.

Durcupan resin (Deerinck et al., 2017): Mix 11.4 g part A/M (Sigma 44611), 10 g part B (Sigma 44612) and 0.3 g part C (Sigma 44613) into a plastic pouring flask. Stir continuously with magnetic stirrer for at least 30 min. Add 0.05-0.1 g part D (Sigma 44614). For both recipes of Durcupan, the EMS products can also be used.

Epoxy "SERVA" resin (Denk and Horstmann, 2004): Mix 11.1 g of glycid ether (SERVA Electrophoresis GmbH 21045), 6.19 g of dodecenylsuccinic anhydride (DDSA, SERVA Electrophoresis GmbH 20755), and 6.25 g of methyl nadic anhydride (MNA, SERVA Electrophoresis GmbH 29451) into a plastic pouring flask. Stir continuously with magnetic stirrer for at least 30 min. Add 0.325 ml of benzyldimethylamine (BDMA, SERVA Electrophoresis GmbH 14835), and stir for a further 10 min.

Epoxy "EMbed812" resin: Mix 20 ml of "EMbed812" (EMS kit 14120), 16 ml of DDSA (EMS kit 14120), and 8 ml of MNA (EMS kit 14120) into a plastic pouring flask. Stir continuously with magnetic stirrer for at least 30 min. Add 0.75 ml of DMP-30 (tris-(dimethylaminomethyl)phenol, EMS 13600), and stir for a further 10 min.

## fBROPA Protocol for Adult Zebrafish Brain


As recommended in Mikula and Denk (2015), tubes should be changed for each staining step.

All experiments were approved by the Veterinary Department of the Canton Basel-Stadt (Switzerland).

#### Imaging

Images were acquired on a Zeiss Merlin SEM (Zeiss, Oberkochen, Germany) and on a Quanta 200 VP-FEG (FEI, Eindhoven, Netherlands; now ThermoFisher Scientific). Both microscopes were equipped with an automated ultramicrotome inside the vacuum chamber for SBEM (3View; Gatan, Pleasanton, CA, USA; now ThermoFisher Scientific). On the Zeiss Merlin, image acquisition was controlled by SBEMimage, an opensource software for image acquisition in SBEM (Titze et al., 2018). Images were acquired with a landing energy of 1.5 keV in analytical mode. Other imaging parameters were chosen to maintain an electron dose of 15 e−/nm<sup>2</sup> (beam current: 300 pA, pixel dwell time: 0.8µs, pixel size: 10 × 10 nm<sup>2</sup> ). These conditions were used to acquire all high-resolution images from the zebrafish brain. Images from intestinal organoids were acquired on the Quanta 200 with a landing energy of 3 keV, a pixel size of 3 × 3 nm<sup>2</sup> , and a pixel dwell time of 2µs.

#### REFERENCES


## AUTHOR CONTRIBUTIONS

All authors contributed to conception and design of the method. AG-M, BT, and CG performed the experiments and imaging. CG wrote the first draft of the manuscript. RF wrote the final manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

# FUNDING

This work was supported by the Novartis Research Foundation and by the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation Programme (Grant Agreement No. 742576).

#### ACKNOWLEDGMENTS

We thank Adrian Wanner for valuable input on the protocol and comments on the manuscript and Stephan Gerhard for the preparation of the movie. We are grateful for helpful discussions with Shawn Mikula, who tragically passed away during the review process of this manuscript.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnana. 2018.00076/full#supplementary-material

Supplementary Data S1 | Movie of an image stack from the adult zebrafish brain prepared using the fBROPA protocol. Field of view is 8 × 8 µm<sup>2</sup> . Original voxel size is 10 × 10 × 30 nm<sup>3</sup> .

in slice preparations for light and electron microscopic analysis. Nat. Protocols 4, 1145–1156. doi: 10.1038/nprot.2009.114


in zebrafish using a digital micromirror device. Nat. Protocols 7, 1410–1425. doi: 10.1038/nprot.2012.072

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Genoud, Titze, Graff-Meyer and Friedrich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Analysis Tools for Large Connectomes

#### Louis K. Scheffer\*

HHMI, Janelia Campus, Ashburn, VA, United States

New reconstruction techniques are generating connectomes of unprecedented size. These must be analyzed to generate human comprehensible results. The analyses being used fall into three general categories. The first is interactive tools used during reconstruction, to help guide the effort, look for possible errors, identify potential cell classes, and answer other preliminary questions. The second type of analysis is support for formal documents such as papers and theses. Scientific norms here require that the data be archived and accessible, and the analysis reproducible. In contrast to some other "omic" fields such as genomics, where a few specific analyses dominate usage, connectomics is rapidly evolving and the analyses used are often specific to the connectome being analyzed. These analyses are typically performed in a variety of conventional programming language, such as Matlab, R, Python, or C++, and read the connectomic data either from a file or through database queries, neither of which are standardized. In the short term we see no alternative to the use of specific analyses, so the best that can be done is to publish the analysis code, and the interface by which it reads connectomic data. A similar situation exists for archiving connectome data. Each group independently makes their data available, but there is no standardized format and long-term accessibility is neither enforced nor funded. In the long term, as connectomics becomes more common, a natural evolution would be a central facility for storing and querying connectomic data, playing a role similar to the National Center for Biotechnology Information for genomes. The final form of analysis is the import of connectome data into downstream tools such as neural simulation or machine learning. In this process, there are two main problems that need to be addressed. First, the reconstructed circuits contain huge amounts of detail, which must be intelligently reduced to a form the downstream tools can use. Second, much of the data needed for these downstream operations must be obtained by other methods (such as genetic or optical) and must be merged with the extracted connectome.

Keywords: analysis of connectomes, EM reconstruction, neural circuits, neural simulation, reproducibility

# 1. INTRODUCTION

A connectome is a detailed description of a neural circuit, including the neurons and the synaptic connections between them. New and improved reconstruction techniques, using electron microscopy(EM) (Chklovskii et al., 2010), optical labeling (Lichtman et al., 2008), or sequencing (Zador et al., 2012), are generating connectomes of unprecedented size. These must be analyzed to generate human comprehensible results and provide input to downstream tools. There are at least

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Ignacio Arganda-Carreras, Universidad del País Vasco, Spain Jeff Lichtman, Harvard University, United States Yunfeng Hua, School of Medicine, Shanghai Jiao Tong University, China

\*Correspondence:

Louis K. Scheffer schefferl@janelia.hhmi.org

Received: 19 June 2018 Accepted: 20 September 2018 Published: 15 October 2018

#### Citation:

Scheffer LK (2018) Analysis Tools for Large Connectomes. Front. Neural Circuits 12:85. doi: 10.3389/fncir.2018.00085

**37**

three very different use cases. The first is interactive analysis, used during the reconstruction itself. Next, there is formal analysis, for reports, papers, and proceedings. Finally, connectomes are used as input to further stages of analysis, such as simulation and machine learning algorithms.

Each of these use cases is rapidly evolving. The increased scale of reconstructions requires new interactive analysis methods for efficiency and quality control. The more formal analyses used so far are often specific to the connectome being analyzed. For example, the analyses used for extremely stereotyped circuits, such as the fly's optic lobe, are very different than the analyses used for the apparently random wiring of portions of olfactory systems. Finally, the usage of connectomes as input to further tools, such as simulation, is just beginning. It is not yet clear what the requirements are.

Analysis of connectomes is likely to follow the path of analysis of genomes. Initially, genomes were difficult to acquire, and the same group that did the acquisition did the analysis. But as the technology for sequencing improved, analysis became the limiting step. Groups that acquired genomes could no longer analyze all the data they collected, and conversely, many of the scientists who analyze genomes had no hand in the data collection. This same transition will likely happen in connectomics. One difference, however, is that connectomics has a much larger variety in the form, function, and usage of its analyses. This differs from genomics, where a few specific forms of analysis dominate the usage, as exemplified by the Basic Local Sequence Alignment Tool, or BLAST (Altschul et al., 1990).

#### 2. PREVIOUS WORK

There is another usage of "Connectome," that refers to the connections between regions of the brain, and not detailed connections between neurons. These apply to much larger animals where detailed neural reconstruction is not yet possible. This paper does not cover analysis of such connectomes, which has its own literature (Sporns, 2003; He et al., 2011; Kaiser, 2011; Leergaard et al., 2012; Xia et al., 2013).

At its heart, a connectome is a directed graph. Since graphs are useful representations in many science and engineering tasks, there has been considerable research into specific tasks on graphs, such as partitioning (Kernighan and Lin, 1970; Pothen et al., 1990; Karypis and Kumar, 1998), clustering (Hartuv and Shamir, 2000; Brandes et al., 2003; White and Smyth, 2005), finding cliques (Everett and Borgatti, 1998), finding patterns (Kuramochi and Karypis, 2005), finding small motifs (Itzkovitz and Alon, 2005) and so on. Only some of these techniques have been applied to connectomes, and it is not clear which, if any, can provide useful answers to practical biology problems.

One challenge with connectomes is that the connectomes are "fuzzy," meaning every instance of a common sub-graph is slightly different. This means that some well-known graph and subgraph matching algorithms (such as Ullmann, 1976), particularly those based on graph invariants (Corneil and Kirkpatrick, 1980), may not work well when applied to connectomes. Conversely, algorithms designed to cope with errors, such as (Messmer and Bunke, 1998), are more likely to be applicable.

"ConnectomeExplorer" (Beyer et al., 2013) is an integrated tool, intended to solve many of the problems indicated in this article. It includes its own visualization tools and analysis language. However, it does not appear to have been used in any of the major connectome analysis efforts, likely because familiarity with conventional tools such as Matlab has outweighed the advantages of a new tool with its corresponding learning curve.

## 3. DISCOVERY

Currently, there are three main use cases for connectomes, here called "discovery," "formal," and "input."

"Discovery" involves inspecting the connectome for interesting features. These tools are typically fast and graphical in nature, and must work with the approximate connectomes that exist as reconstruction progresses. They are often built into the reconstruction tools, and are used to look at reconstruction concerns and ordering, as well as generate science results as early in the reconstruction process as practical. Examples include connectivity tables of various kinds, plot of connections as a function of graph connectivity or distance from the root of the neuron, and comparisons of seemingly similar neurons. In this paper, we look at tools used during past reconstructions, those being used currently in the still larger reconstructions in process, and those we think will be needed in the future.

Tables of connections are one of the most obvious outputs. Typically, these show the upstream and downstream neurons, sorted by strength, as shown in **Figure 1**. Color coding makes connection patterns more obvious. Comparing rows shows the differences between neurons with similar names or types.

Dendrograms are another natural representation. Nervous systems often contain many similar cells, often referred to a "cell type." Cell types are traditionally defined by morphology (Fischbach and Dittrich, 1989) but with connectomes it makes sense to define them by connectivity as well. One natural way to group cells is to represent their connections by a vector of connection strengths to various other types. These vectors can be grouped by distance to create a dendrogram, grouping together cells with similar connectivities and separating cells with very different connection patterns. An example is shown in **Figure 2**.

Another natural representation of a connectome is as an instance of a directed graph. Circuits are easier to visualize connections as a graph rather than a collection of tables, even if the information is the same. In the circuits reconstructed so far, nervous systems are seemingly constructed of several motifs small enough to be easily visualized, including reciprocal connections and small loops. These graphs may be annotated with connections weights (expressed in number of synapses).

A connectome expressed as a graph also facilitates queries defined by connectivity, such as "Find all cells of type A that connect to any instances of type B by a path of 2 hops or less." A connectome can be loaded into a graph database such as Neo4j (Miller, 2013), and then a variety of graph query languages


FIGURE 1 | Example of connection table. Each row shows the connections to a single neuron, sorted by synapse count. Each box shows the identity of the connected neuron, the synapse count in both directions (separated by a ":"), and the internal identifier of the connected neuron. Colors are arbitrary, but all cells of the same type share the same color. Data from Takemura et al. (2015).

(Wood, 2012), such as cipher or Gremlin, can be used query the data.

One often requested graphical form is the connections to just one cell type, as shown in **Figure 3**. In general only the stronger connections are of interest. Also, for some purposes the connections to one instance of a cell are wanted, but in other cases it might be the average connectivity to all cells of the same type.

One of the main reasons to draw a graph, rather than a table or list, is to enable human understanding of circuit operation. It is therefore important that the display diagram be designed not only to be technically correct, but to show the information flow in a way that is easy for humans to understand. Programs that do this for arbitrary electronic circuits (Jehng et al., 1991) and directed graphs (Gansner et al., 1993) have long existed. These could perhaps be mined for ideas helpful for drawing biological networks.

An example of what is desired is shown in **Figure 4**. This diagram was created (manually) to highlight the role of two cell types, Mi4 and Mi9 from the medulla, in the pathways to the motion detecting cell T4. Mi4 and Mi9 have strong crossconnections, and between them receive inputs from many cells from the lamina. In particular, Mi1 is a strong contributor to both paths. The diagram is organized with inputs at the top and the T4 cell at the bottom. Only strong connections are shown, and other inputs to the T4 are ignored in this diagram.

# 4. FORMAL

We define "Formal" analysis as the analysis used in formal scientific documents such as papers, theses, and proceedings. Such analysis should at least be archival and reproducible, and ideally easily extendable. A scientist who seeks to reproduce the results might wish to do so at several levels:


These options require physical access to the data, an understanding by programs and humans of how the data are structured, and ability to run the analysis. We consider each of these issues in turn.

Due to the recent introduction and rapidly evolving nature of connectome analysis, no standards are yet available, and publication of data sets and analysis code is largely handled on an ad-hoc basis. Another problem is that the data sets are large (often many terabytes). Thus the data are too big to publish as supplemental data to a paper, and must be archived elsewhere.

Counts and percentages shown for the five most strongly connected types. Data from Takemura et al. (2015).

# 4.1. Formal Analysis

Analyses of connectomes are varied and often complex (up to tens of thousands of lines of code). For such procedures, as data scientists are well aware, the "Methods" section of a paper is just a summary of the actual analysis performed. Details such as the resolution of ties in sorting procedures, the numerical precision of intermediate results, differences in library routines, and so on, make it almost impossible to precisely reproduce results from the methods section alone. In general (one hopes) this does not affect the main points made in the paper, nor affect the conclusions when comparing substantially different organisms. However, when connectomics advances to comparing closely related species then it will be critical to use the exact same software for both, to ensure that any differences found are the result of biology and not an artifact of slightly different computation.

There are two main approaches to this problem. One is to centralize the analysis, so all researchers are using the same program. The other is to publish the code and the access methods. Then each researcher should be able to run the analysis at their own facility, and ideally get the exact same result.

The field of genomics had similar problems. The adopted solution (at least in the USA) was a funded center, the National Center for Biotechnology Information, that both stored the data and hosted the primary analysis tools. The initial version (Wheeler et al., 2000) stored mostly genetic data but it has since expanded to hold other related items (NCBI Resource Coordinators, 2018). This helps in several ways. Two different papers, using (for example) BLAST, can be compared directly since they use the same analysis tools. Next, since the data sets and analysis tools are hosted on the same site, the network bandwidth requirements are much reduced.

Could such a centralized analysis work for connectomics? Probably not yet, since tools have not yet converged on a commonly used set. To show this, we look at a (small) subset of analyses that have been attempted, and what tools were used, based on published analysis of large connectomes, both our own and others. This is shown in **Table 1**. Even this subset shows that analysis tools span a wide range of methods and techniques, and most analyses so far have typically been computed in an external tool such as Matlab, R, or Python.

One common analysis matches receptive fields to the circuits that compute them, such as in Briggman et al. (2011), Bock et al. (2011), Takemura et al. (2013), and Takemura et al. (2017b). These analyses can't be done with connectomes alone—they need the physical location of the input, such as the location of

FIGURE 3 | Connectivity to and from each T4 cell in Drosophila, as shown by a reconstruction of 7 columns of the optic lobe (Takemura et al., 2015). The incoming strengths are indicated as A/B C%, where A is the total number of synapses to all cells of that type, B is the number of cells connected, and C is the percentage of total input (output). The outgoing strengths are the number of synapses. The area of each circle is roughly proportional to the connection strength.

manually, but automated and semi-automated tools to create such diagrams would be helpful. Data from Takemura et al. (2015).

photoreceptors in the retina or the hexagonal grid of the flys eye. They then require a weighted sum through the network, and perhaps network signs and delays as well. Receptive fields for other modalities such as olfaction, gustation, or auditory, will be very different and require specialized analyses of their own.

Another sample analysis is that of stereotypy. To examine the limits of neural wiring accuracy, Lu et al. (2009) compared the wiring of the same neurons on the left and right sides of a mouse. Similarly, Takemura et al. (2015) examined a particularly stereotyped system, the medulla of Drosophila, comparing each of 7 nearly identical columns against each other, using a detailed statistical model to try to separate the different potential causes of differences—differences in biology of pre- and post-synaptic counts, and reconstruction errors. In both studies, one of the main goals was to measure the rate of biological differences and errors, by manually re-examining all differences between the sides and/or columns. This is unlikely to be a common operation while reconstruction is limited to a single specimen, since such a crystal-like repetition of circuits is not expected in most parts of the brain. It will become more common, however, as comparisons of connectomes across multiple animals are tried, once increased throughput makes this practical.

Another very specific analysis is that of randomness of a specific set of connections. This was examined in the visual cortex of the mouse (Bock et al., 2011), and the olfactory system of Drosophila larva (Eichler et al., 2017) and adult (Takemura et al., 2017a), respectively. In each case, preliminary analysis showed no obvious pattern of connectivity between certain classes of input cells and the output neurons. However, to back up this apparent randomness, a detailed statistical model was required, and then the circuit compared to this model, generating p values, statistical powers, and so on. While the basic problem of modeling seemingly random connections is likely to re-occur, the details of each computation make it unlikely that the exact computations can be re-used.

These examples of the various and sundry analyses used show that it is unlikely that any reconstruction tool could perform all, or even most, of the analyses needed after reconstruction. Therefore, we find no practical alternative to the use of external tools, so the challenge is to make the use of such tools convenient, transparent, and reproducible. Transparency is the easiest to address, with the analysis code posted on a publically available site such as GitHub (Blischak et al., 2016) or included as supplementary data.

#### 4.2. Formal Data Storage

More difficult, perhaps, is storing the connectome data itself in a reproducible and archivable way.

Formal analysis is based on, and analyzes, many different products of the reconstruction process. In all reconstruction techniques to date, EM, optical, or genetic, the raw data is large, and requires significant processing to generate a connectome. While here the discussion concentrates on EM, the same principles will apply if other modes of analysis are used.

In order of decreasing size, the data used in EM connectivity analysis is:


TABLE 1 | Analysis tools as used in a selection of connectome analyses.


The are only examples from a much larger field of studies, and intended only to show the wide variety of tools and languages employed.

neuron. These are typically formatted as SWC<sup>1</sup> files (Carnevale and Hines, 2006) with an additional list of synapse locations.

• A graph, with neurons as nodes and synapse counts as weights.

Reproducing or extending an analysis will require using one or more of these representations. The raw source EM images are probably not of general interest, and "Contact the authors" probably suffices. The aligned, stitched, and normalized images form the source for machine segmentation and human proofreading. These could be made available as a stack of images, with the main problem not the technical storage but instead who will maintain (and pay for) such storage over archival lifetimes. As of mid-2018, the cheapest cloud storage costs about \$4 (US) per terabyte per month. Thus a 100 TB data set costs about \$400/month to store. For an active project this is reasonable, but for a 50 year archive, the cost would be \$250,000 US, or the cost of several researcher-years. Most universities and research institutions would not feel such archiving is their responsibility. Even if they did, research institutions, and their focus areas, come and go over decade-long time scales. Universities and scientific journals have longer histories, but not the budgets to pay for archival storage.

Technically, reading a stack of stored images, no matter how large, should not be problem. Smaller examples can be read by publically available software such as ImageJ (Schneider et al., 2012), or its distribution FIJI (Schindelin et al., 2012), already commonly used in neuroscience. Larger examples can be read by existing software such as BigDataViewer (Pietzsch et al., 2015), a public extension of Fiji. There are higher performance and cloud compatible solutions, such as the internal format "n5" (Saalfeld, 2017) of BigDataViewer, but the longevity of these formats has not been established, whereas a stack of images should be readable for the forseeable future.

Segmentation can be stored similarly, with more bits per pixel but much better compression, due to long runs of the same value.

Skeleton data is smaller and is commonly stored as text files. There is an existing public and funded database for this, "Neuromorpho.org" (Ascoli et al., 2007). However this does not include the synapse locations or any volumetric description, and so can only store part of the results of connectomics.

#### 4.3. Making Sense of the Data

Acquiring the physical bits that describe a connectome is only part of the problem—the next problem is making sense of it. There are two main technical methods by which external tools can get connectome data for analysis. In the oldest method, the reconstruction software writes out the relevant data as files, normally in text formats such as JavaScript Object Notation (JSON) (ECMA International, 2017) or Extensible Markup language (XML) (Bray et al., 1997) for the connectome, and SWC for the skeletons. Then an external program can read and parse these files, then do the requested analysis. In a more modern approach, a program wishing to do analysis requests the data it needs from a reconstruction server. This has been done for CATMAID (Saalfeld et al., 2009), VAST (Berger, 2015), and DVID(Katz, this issue), three recent reconstruction tools. This method has several advantages - no export step is necessary, only the needed data is transferred, and the external analysis gets the

<sup>1</sup> SWC is not an abbreviation, but the initials of the developers (Stockley et al., 1993).

most recent version of the data (or the requested version, if the reconstruction tool supports versioning).

Multi-decade archiving requirements are most easily met by using simple text files, combined with programs in standard languages such as C++ or Python. Such files and programs will likely be readable and usable decades from now—for example, many FORTRAN programs from 50 years ago, such as LINPACK (Dongarra et al., 1979) are still used today. Furthermore, files from multiple sources can be easily combined, and files have a much lower barrier to entry. Students anywhere in the world, by themselves, could easily download the relevant files and start analysis. A one-time file format conversion is typically a one day job for an undergraduate, whereas modifying a server to support a different set of queries can take months of an expert's time. Furthermore, if running the analysis requires connection to a server, the process has considerably more human overhead, either requiring someone to run and maintain a local instance of a server, or considerable cooperation from already busy researchers.

The approach of querying a server for connectome data, while undoubtedly convenient, has downsides for reproducibility, with exactly the same risks as references to web sites. In a decade or two, the servers may be unavailable, the queries that are supported may have changed, or the owners may have moved on and no longer remember (or care) how to run the needed servers. A number of technical fixes to this problem have been proposed, such as scientific workflows (SWFs) (Altintas et al., 2004), virtualization (Dudley and Butte, 2010), and automated build systems such as Docker (Boettiger, 2015). However, each of these has their own disadvantages and overheads, particularly when combining two or more analyses that were archived using different methods. Furthermore, the author is skeptical that these methods will remain effective over the multi-decade timescales desired for scientific reproducibility.

However, despite these drawbacks, the use of servers with queries instead of text files is technically inevitable. Text forms are not efficient enough for the bigger data sets, and with a large data set a way to get desired subsets will be needed in any event. Larger and more powerful computers will not solve this problem, as their capacity will surely be used to attack correspondingly larger problems. Therefore it is incumbent on the researchers in the field of connectomics, in the interest of scientific reproducibility, to make sure their interfaces are efficient, stable, and well-documented.

#### 4.4. Formal Analysis Conclusions

Public and archival storage of connectomic data and algorithms remains an area for development. For now, the field is dependent on the good will of practitioners to preserve and provide access to the data they collect, and the algorithms that operate on that data. We urge that they continue to use best practices, and perhaps a concensus solution will emerge. A funded center, with storage and the most common analysis tools, seems like the long term answer. The National Center for Biotechnology Information already stores and analyzes many forms of biological data, in addition to its original charter of genetic information. It would make sense for this center, or its equivalent in other countries, to pick up the task of storing and providing access to connectomic data.

### 5. INPUT

The final use case is "Input," where the connectome is used as input to another process. In general the goal of connectome reconstruction is not the connectome itself, but a mechanistic understanding of the operation of the nervous system. This involves integrating other data, obtained from other sources by other methods. This is because the EM images typically used for circuit reconstruction show the detailed shapes of cells, and the existance, location, and partners of synapses, but many details critical to the circuit and synapse operation are not visible in these images. Gap junctions and synapse models including transmitter and receptor types are the most obvious examples, but locations of ion channels, receptors and sources for neuromodulators and hormones, biochemical cascades affecting synapses, and sensor/actuator links to the sensory and motor system are needed as well.

This additional data must be generated by methods other than electron microscopy. The neurotransmitter(s) of each cell can often be determined by techniques such as RNA sequencing (Croset et al., 2018), or Fluorescent in-situ Hybridization (FISH) (Spencer et al., 2000). Receptors expressed by a cell can also be found by RNA sequencing, but this does not tell where each receptor is expressed. This is a particular problem in insects, where many of the main transmitters, such as acetylcholine and glutamate, have multiple different receptors, sometimes of differing sign (Osborne, 1996), and all expressed in the same cell. In the case of a single receptor and a single cell type, this problem has been approached via FISH, but techniques with higher throughput are clearly needed. A combination of multicolor labeling (Bayani and Squire, 2004), genetically identified cell lines, and expansion microscopy seems the most likely approach to resolving this. An entirely different approach (Jonas and Turaga, 2016; Tschopp and Turaga, 2018) is to reverse fit the known operation to try to find the sign, strengths, and time constants of the synapses.

Integrating this additional data with connectomes is both an opportunity and a requirement in the quest to understand the operation of the nervous system.

#### 5.1. Input for Simulation

One typical use for connectomes includes neural simulators such as Neuron (Carnevale and Hines, 2006), Genesis (Bower and Beeman, 2012), or Nest (Gewaltig and Diesmann, 2007), or a theoretical model of circuit operation. This seems straightforward in principal, but there are several concerns. First, there can be problems with the accuracy of extracted values. Second, the data (particularly from EM) can be too detailed, and overwhelm downstream tools. Conversely, some of the required data will still be missing, and must be supplied from other sources.

One problem is the accuracy of extracted parameters, such as the cytoplasmic resistance and the membrane capacitance. correctly.

neuron CT1, where reduction to a single node leads to incorrect results. The large size (scale bar is 10µm) and small connecting neurites create a many-millisecond delay between the clusters defined by the dotted ellipses. If the neuron model is compressed to a single node, as is optimal for (A), this delay will not be simulated

Some techniques for obtaining connectomes, such as barcode sequencing, do not generate this information, even approximately, so it must be supplied from other sources. Even techniques that do reveal morphology of cells, such as optical or EM, are subject to errors introduced in the staining and fixing process. None of the reconstruction techniques reveal the resistivity of the cytoplasm. Membrane capacitance is welldefined, per unit area, but influences such as myelinization can change the effective value.

Another problem arises if simulation of extracted connectome, or a theoretical model of operation, is the goal. In these cases the models from EM reconstruction are typically much more detailed than needed, requiring intelligent reduction to get a useable representation (Gornet and Scheffer, 2017). A typical neuron reconstructed by EM has hundreds if not thousands of segments, typically represented as an SWC file. This is much more detail than required, at least when considering electrical effects, and results in impractical runtimes. Reducing the level of detail leads to orders of magnitude better execution times, and for many neurons the resulting error is acceptable. There are some neurons, however, where full reduction leads to inaccurate simulations. This problem is illustrated in **Figure 5**, where the first panel shows the impracticality of including all detail, but the second panel shows a case where the detail cannot be entirely ignored.

This analysis can be quantified using a simple approximation of simulation accuracy, which shows that EM produces much more detail than is likely required, but that larger neurons cannot be reduced to a single compartment. Neurons operate on roughly millisecond time scales. Compartments with much smaller time constants make solving the equations of simulation difficult (due to both the large number of compartments and the wide span of time constants) while adding little accuracy. Compartments with time constants much larger than a millisecond are easy to simulate but may be silently inaccurate. So what is in general desired is a model with time constants somewhat less than a millisecond, but not too much less. The exact tradeoff of course depends on the accuracy needed and the circuit under analysis.

Using a resistor-capacitor(RC) model to estimate time constants, the Elmore delay (Elmore, 1948) d of a cylinder of diameter D, length L, cytoplasmic resistivity ρ, and membrane capacitance Cm, is

$$d = \frac{R \cdot C}{2} = \frac{1}{2} \cdot \rho \frac{4L}{\pi D^2} \cdot C\_m \pi \, dL = \rho C\_m \frac{2L^2}{D} \tag{1}$$

Typical values are <sup>ρ</sup> <sup>=</sup> 1 ohm·m, and <sup>C</sup><sup>m</sup> <sup>=</sup> <sup>10</sup>−<sup>2</sup> F/m<sup>2</sup> . A thin branch might have a diameter D of 100 nm or 10−7m, while a very thick neurite might have a a diameter of 10µm. The resulting delays are shown in **Figure 6**. For example, a length L of 50µm yields a delay of 0.5 millisec for a thin neurite.

Since the delay scales as L 2 , but only inversely as diameter D, this means that even a very thin branch will allow compartments 10µ in length, within which the differential delay will be less than 100µs. Conversely, long neurons (such as those 1 mm or longer) will need to be divided into compartments, even if they are very thick, to keep the differential delay under a millisecond.

This drives the requirement that the EM skeletons must be reduced (otherwise they will overwhelm simulation resources and create time-constant problems), but cannot be reduced indiscriminately (or they will not be accurate). Very similar problem have been addressed in

the field of electronics when simulating circuits derived from IC layouts, and previously developed solutions (for example, Sheehan, 1999; Ionutiu et al., 2011; Reis and Stykel, 2011) can provide good starting points here.

Consider what would be needed to do a simulation of a functional unit comprising a subset of a much larger connectome. Even in a small animal such as an insect, just defining the subset of neurons to simulate is already a big task if performed manually—a typical functional circuit (say the mushroom body, the seat of olfactory learning in insects) already contains a few thousand neurons. In a vertebrate or mammal, the circuits are likely even larger. Here are some of the steps required:


To make this easier, the software that writes the simulator input should do a number of these (nontrivial) operations automatically, then format the file for the simulator concerned (perhaps Neuron, Genesis, or Nest).

An interesting problem that has not yet been seriously addressed is matching simulation results with in-vivo recordings made before the reconstruction. Several data sets have acquired in-vivo 2-photon calcium imaging of nervous system activity before ex-vivo reconstruction, usually with the goal of identifying some subset of cells in later images (Bock et al., 2011; Briggman et al., 2011; Lee et al., 2016). Matching simulated with measured results holds the potential of demonstrating that all relevant factors have been considered. We are quite far from this ideal currently, due to both lack of detailed knowledge of much of the cellular machinery, and limitations of current reconstructions. In particular, all existing reconstructions include many neurons that extend outside the reconstructed volume. Accurate simulation of these neurons is impossible, nor can the activity of all such neurons be adequately measured by existing techniques. Better recording techniques, increased knowledge of cellular detail, and larger reconstructions will all bring this goal closer, but it remains many years away.

Finally, large, and particularly full-animal, connectomes will drive the requirement to co-simulate with mechanical and other simulators. These will be animal and environment specific, such as acoustic simulation for animals that echolocate, hydrodynamic simulation for animals that swim, and aerodynamic simulation for animals that fly. This co-simulation will require cross-domain conversion, for example converting neural activity to muscular forces, to drive mechanical models, and converting joint angles, forces, and other sensory inputs back into neural codes. Steps in these directions have been taken by programs like AnimatLab (Cofer et al., 2010) and FlySim (Huang et al., 2014), but much more remains to be done.

# 6. CONCLUSIONS

Until recently, connectomes have been difficult and time consuming to acquire. Analysis took a comparatively small effort and was performed by the same team doing the reconstruction. Reconstruction technology, however, is rapidly improving and we are about to enter a new era. In this era, analysis rather than data collection will dominate, and the researchers doing analysis will often be distinct from those doing reconstruction. This change happened quickly in the field of genomics, and we need to plan for a similar transition in connectomics.

Along these lines, we note that the many unique analyses required to date are likely a result of our lack of understanding of the principles behind neural circuit organization. It seems likely that as more and more connectomes are analyzed, patterns of circuit organization will emerge. In the future, it is therefore possible that a standard set of analyses may suffice for most users, as is currently the case for genomics.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and has approved it for publication.

#### FUNDING

This research was funded by the Howard Hughes Medical Institute.

#### ACKNOWLEDGMENTS

The author is grateful to Steven Plaza and Bill Katz, for their helpful comments.


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Scheffer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# VAST (Volume Annotation and Segmentation Tool): Efficient Manual and Semi-Automatic Labeling of Large 3D Image Stacks

Daniel R. Berger<sup>1</sup> \*, H. Sebastian Seung<sup>2</sup> and Jeff W. Lichtman<sup>1</sup> \*

<sup>1</sup> Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, United States, <sup>2</sup> Computer Science Department, Princeton Neuroscience Institute, Princeton University, Princeton, NJ, United States

Recent developments in serial-section electron microscopy allow the efficient generation of very large image data sets but analyzing such data poses challenges for software tools. Here we introduce Volume Annotation and Segmentation Tool (VAST), a freely available utility program for generating and editing annotations and segmentations of large volumetric image (voxel) data sets. It provides a simple yet powerful user interface for real-time exploration and analysis of large data sets even in the Petabyte range.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Sean L. Hill, Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Canada Marta Costa, University of Cambridge, United Kingdom Sean Parsons, McMaster University, Canada

#### \*Correspondence:

Daniel R. Berger danielberger@fas.harvard.edu Jeff Lichtman jeff@mcb.harvard.edu

Received: 07 June 2018 Accepted: 24 September 2018 Published: 16 October 2018

#### Citation:

Berger DR, Seung HS and Lichtman JW (2018) VAST (Volume Annotation and Segmentation Tool): Efficient Manual and Semi-Automatic Labeling of Large 3D Image Stacks. Front. Neural Circuits 12:88. doi: 10.3389/fncir.2018.00088 Keywords: connectomics, segmentation, visualization, serial section electron microscopy, CLEM, proofreading, TrakEM2, voxel

# INTRODUCTION

The acquisition of microscopic data is becoming ever faster and more and more automated, leading to the generation of enormous image datasets. At the same time progress in processing speeds and storage capacity of computer hardware enables imaging scientists to work with big data. In neuroscience acquisition of high-resolution volumetric data sets of the nervous system has become routine, with the goal of addressing a number of long-standing questions (Briggman and Bock, 2012; Helmstaedter, 2013; Morgan and Lichtman, 2013, 2017; Plaza et al., 2014; Titze and Genoud, 2016). Projects include descriptions of the entire nervous systems of a variety of animals, for example Caenorhabditis elegans (White et al., 1986; Varshney et al., 2011), Drosophila melanogaster (Zheng et al., 2017), Zebrafish (Hildebrand et al., 2017); wiring diagrams of specific parts of larger nervous systems, for example mouse retina (Helmstaedter et al., 2011, 2013; Kim et al., 2014; Bae et al., 2018), thalamic nuclei (Morgan et al., 2016), and cortex (Bock et al., 2011; Kasthuri et al., 2015; Lee et al., 2016); function-structure relationships, for example directional selectivity in the retina (Briggman et al., 2011; Kim et al., 2014), detection of visual motion in drosophila (Takemura et al., 2013), learning and plasticity in hippocampus (Mishchenko et al., 2010; Bartol et al., 2015), synapse elimination in the neuromuscular junction (Tapia et al., 2012); and many others.

Several experimental techniques have been introduced to enable processing and imaging such large volumes of tissue with electron microscopy. Among these are the development of advanced techniques for preparing and staining very large pieces of tissue for electron microscopy (McIntyre and Fahy, 2015; Mikula and Denk, 2015), block-face cutting and imaging methods like SBEM/SBF-SEM (Denk and Horstmann, 2004) and FIB-SEM (Knott et al., 2011; Hayworth et al., 2015), automated collection of sections on tape, for example ATUM (Hayworth et al., 2006; Kasthuri et al., 2015), and high-speed imaging techniques like TEMCA (Bock et al., 2011), and the Zeiss mSEM (Eberle et al., 2015).

Because of the wider availability of these methods, analyzing big data sets poses a challenge for a growing number of researchers. Many existing software tools are not well suited for big data nor the wide variety of research questions these data sets allow. Many, if not all, of the software applications for analyzing microscopic image data allow labeling of cellular or subcellular constituents of the volumetric tissue. This fundamental process is called segmentation and can be done in several ways including: (1) Bread crumbs/Seeds; Markers are placed inside objects to identify their location from one section to the next, (2) Skeletonization; If objects have a tree- or graph-like structure, they can be described by a set of nodes that are connected by straight lines called edges, (3) Outlining; Objects of arbitrary shape can be delineated by their outlines in each section to create the surface of a 3-dimensional object, (4) Voxel painting; Objects of arbitrary shape can be labeled by filling in their area in each section to create a volume.

As the number of biological laboratories analyzing large image data sets is increasing, it is the purview of computer scientists to create the (segmentation) tools for analyzing such data sets. Taking into consideration the diverse range of potential applications from 3D microscopy, an ideal segmentation tool would have the following features:


**Table 1** shows a comparison of features of several popular programs for EM stack segmentation. Many tools were originally made for smaller data sets and require the complete data to be loaded in memory, which is not feasible for large image stacks, though for some programs workaround exist; these include ITK-SNAP (Yushkevich et al., 2006), trakEM2 (Cardona et al., 2012), Reconstruct (Fiala, 2005), which loads two complete sections at a time, ilastik (Sommer et al., 2011), and IMOD (Kremer et al., 1996). Other tools are limited to be used only by their respective developers, like Eyewire (Marx, 2013; Kim et al., 2014; Bae et al., 2018), and/or are specialized for skeleton tracing, like Catmaid (Saalfeld et al., 2009), Knossos (Helmstaedter et al., 2011), and WebKnossos (Boergens et al., 2017), or for splitting and merging for proof-reading of automatic segmentations like Raveler (Chklovskii et al., 2010; Takemura et al., 2013) and its successor NeuTu<sup>1</sup> . The professional tools Imaris (Bitplane Inc.) and Amira-Avizo (Thermo Fisher Scientific) have workarounds to use large data sets that cannot be fully loaded into memory. However these latter programs appear to have only rudimentary tools for manual segmentation and are not specialized to do voxel painting or proof-reading of automatic segmentations. Neurolucida (MBF Bioscience) is a specific tool for analyzing light microscopy data and appears to be RAMsize limited. Neurolucida 360 does support large datasets beyond the machine's RAM limit, however it is still focused on light microscopy applications.

To address the requirements of an ideal segmentation tool and to supersede the functionality of available segmentation tools, we built VAST, a lightweight, freely available utility program for manual annotation and segmentation of large volumetric image (voxel) data sets even in the Petabyte range. VAST is written in C++ with Direct3D graphics for optimal performance (see **Figure 7** for information about the internal program structure). It provides an intuitive yet powerful user interface for exploring image stacks at interactive speeds, and for labeling structures of interest by voxel painting at multiple resolutions. As **Table 1** shows, VAST solves many of the problems that beset other tools. For ground truth annotation by voxel painting VAST is an excellent choice. In addition, because of the availability of automatically segmented data, we found VAST can reconstruct whole volumes faster than fully manual segmentation (see below). Although this tool was developed for neural circuit analysis of EM datasets, it can load and process any three-dimensional 8- or 24-bit image stack and be used for other applications like multicolor light microscopy, CLEM, video analysis and object tracking. VASTs extensive import and export functions make it easy to integrate it with other applications.

Volume Annotation and Segmentation Tool can open grayscale and RGB image stacks which have been either imported into VAST's own 3D data file format or are stored locally in image tiles, and it can stream image data from several online sources. Multiple image and segmentation layers of the same dataset can be loaded and displayed together. Segmented objects can be named, grouped and organized in a tree structure, and segmentations and their metadata can be imported and exported. Automatic segmentations can be proof-read with merge and split operations and novel trans-layer masking techniques. Custom client programs can exchange data with VAST via a documented API. VAST includes the API client program "VastTools" which runs in Matlab (The Mathworks, Inc.) and provides additional

<sup>1</sup>https://github.com/janelia-flyem/NeuTu


TABLE 1 | Feature comparison

 of popular 3D EM analysis tools.

and new features are added continuously.

shows spiny dendrites in the area in individual colors. The tool windows A and C–F are explained further in section 2 of the main text.

functions for exporting, measuring, and navigating data loaded into VAST.

Volume Annotation and Segmentation Tool has been key for the data analysis for a number of scientific papers and continues to be a versatile tool with growing functionality and an expanding user base (Tomassy et al., 2014; Kasthuri et al., 2015; Ai-Awami et al., 2016; Joesch et al., 2016; Ke et al., 2016; Morgan et al., 2016; Quadrato et al., 2017; Sheu et al., 2017; Zung et al., 2017). Because VAST is designed as a general labeling tool, it is not limited to tracing neurites, but can be used for a large variety of 3D data sets and tasks (see **Figures 5**, **6**). This includes working with electron-microscopic, multi-channel light-microscopic, and Micro-CT data sets as well as videos, and annotating arbitrary structures, regions and locations, depending on the user's needs.

The version of VAST discussed in this paper is VAST Lite 1.2. An earlier version of VAST, which lacked most of the key features reported here, was briefly discussed in the methods section of (Kasthuri et al., 2015). New features implemented since then include: VSVI files (section "Reading Image Files Directly, .VSVI"), VSVR files for Google Brainmaps and Butterfly servers (section "Reading From Online Databases, .VSVR"), collect tool (section "Organization of Segments in Hierarchies"), working with multiple image stack and segmentation stack layers (section "Working With Multiple Image and Segmentation Stack Layers"), image layer coloring and blending (**Figure 3**), filling tool (section "Manual Segmentation by Drawing and Filling"), trans-layer masking (section "Working With Automatic Segmentations" and **Figure 4**), the 3D viewer (section "The Integrated 3D Viewer" and **Figure 5**), and the API and VastTools (section "The VAST API and VastTools"). At the time of writing of this manuscript, the current version of VAST can be downloaded at: https://software. rc.fas.harvard.edu/lichtman/vast/.

#### THE VAST USER INTERFACE

Volume Annotation and Segmentation Tool's user interface is based on familiar Windows controls and is optimized for efficient use of pen tablets for fast and accurate user interaction. The main window of VAST (**Figure 1**) shows a 2D section of the loaded dataset(s) and has several floating tool windows which can be moved and resized. The tool bar of the main window provides quick access to the different editing tools, as well as a switch to hide all image layers (EM), and sliders to control the opacity of the selected segmentation layer (Alpha) and, if enabled, separately for the segment or group of segments selected in the 'Segment Colors' tool window (SelAlpha). To increase the number of distinguishable segment colors beyond RGB, VAST can combine two 24-bit RGB colors with one of 16 patterns to reach a color space of almost 52 bits. The strength of the patterns can be controlled with the 'Pattern' slider. **Figure 2** shows examples of different settings of Alpha, SelAlpha, and Pattern. Further

patterns. The strength of the patterning can be controlled with the "Pattern" slider.

options for color correction and blending of individual layers are provided in the 'Layers' tool window (**Figure 1A**).

The 'Coordinates' tool window (**Figure 1C**) shows the current volume coordinates (center point of the main window) which can be copied from and pasted into the text field to store and go to particular coordinates in the stack. Its drop-down menu lists coordinates that were recently visited.

The 'Drawing Properties' tool window (**Figure 1D**) contains the parameters relevant for the pen tool, including settings for the optional 'masking' mode. This mode can be used to constrain manual painting by an automatic segmentation result, which can lead to an increase of accuracy and speed of manual tracing, even if the automatic segmentation has errors.

Equivalently, the 'Filling Properties' tool window (**Figure 1E**) contains the parameters relevant for the filling tool. Filling can also use masks derived from colored regions in a separate source layer, which allows for efficient proof-reading of automatic segmentations (split and merge operations).

The 'Segment Colors' tool window (**Figure 1F**) lists all segments used in the selected segmentation layer, with their color and label, in a tree folder structure which represents grouping and parts/subparts relationships of segments. Collapsing and expanding these folders determines the display colors in the main window – segments in collapsed folders will be shown in the folder color (**Figure 2**). The search field at the top of the tool window can be used to find segments by (part of) their label text or their internal ID. The 'Menu' button leads to a context window with many more functions to edit segment properties and the tree hierarchy.

All tool windows are listed under 'Window' in the main menu; this also includes a window with control buttons to be used with touch screens, a window showing all keyboard shortcuts, and the options window for the 'Remote Control API server' to link to external programs via TCP/IP.

#### IMPORTING IMAGE STACKS

Volume Annotation and Segmentation Tool can access image data from three types of sources: (1) image stacks which have been imported into VASTs own data file format ("VAST Volume", .VSV/.VSVOL), (2) image data sets stored as image files, accessed via a .VSVI descriptor file ("VAST Volume of Images"), and (3) data sets stored online, accessed via a .VSVR descriptor file ("VAST Volume of Remote data"). Segmentation data can be imported from image files representing segment IDs, and

optionally a metadata description file, into VASTs segmentation file format (.VSS/.VSSEG). The image file formats from which VAST can import are listed in **Table 2**.

## VAST Image Data Files, .VSV/.VSVOL

These files store a complete image stack together with a resolution pyramid of lower-resolution versions of the same image data ["mipmaps", (Williams, 1983)]. During importing, the image data is reordered as 16 pixel × 16 pixel × 16 pixel cubes in optional Z-order and packed with lossless or lossy compression. Mipmaps are precomputed. A file-internal tree of pointer blocks is generated which allows VAST to access arbitrary regions with minimal overhead.

Keeping the image data of one data set in a single file has the advantage that it can only be copied as a whole, making storage and distribution of data sets simpler. Also, the file system does not have to handle thousands or millions of image files. However, this comes at the disadvantage that images in the dataset cannot be modified or the stack extended without regenerating the data file. Also, the importing procedure is impractical for large datasets, not only because the target file can get unwieldy, but also because importing can take a very long time since processing cannot be easily parallelized. For example, importing the ∼6.8 teravoxel data set of (Morgan et al., 2016) took around 30 days on a single computer, with disk and network I/O being the largest bottleneck. Therefore, for large datasets (exceeding one terabyte), we typically keep the image data in individual image tile files which VAST can load directly.

## Reading Image Files Directly, .VSVI

Image stacks can be kept as a collection of image files (.PNG, .TIFF, or .JPG) with a descriptor file for VAST (.VSVI, "VAST Volume of Images"). VAST can then load and cache specific regions directly from the images. The image files have to be stored in a regular directory structure, and reduced-resolution images (mipmaps) have to be precomputed and stored as separate files. Then, a .VSVI file for VAST is prepared. This is a text file following the JSON syntax which specifies the naming scheme and storage location of the image tiles, as well as other metadata for the data set. The .VSVI file can be opened in VAST, which then loads regions of the dataset from the image files as requested.

# Reading From Online Databases, .VSVR

Volume Annotation and Segmentation Tool can also stream in remote data from online databases. Some data sets are too large to be stored locally, and/or they reside on a server which is accessible via HTTP. VAST can load data from such sources dynamically by requesting parts of the data set from the server. Currently, protocols for openconnecto.me, neurodata.io, Harvard Butterfly servers, and Google Brainmaps are supported. The specification of the source address, the data request protocol, and additional metadata is stored in a .VSVR file (also a JSON text file). Once such a file is opened, VAST will request and stream in image data from the server dynamically. In the case of Google Brainmaps, VAST will first negotiate access rights with the server through the OAuth2 protocol (including user login in a browser window). VAST always caches the image data locally to optimize speed and minimize the network load. Several .VSVR files linking to existing online datasets are included in the VAST supplementary package (see section "The VAST API and VastTools").

#### Importing Segmentations, .VSS/.VSSEG

Similar to.VSV files described above, VAST stores segmentations in single files with extension .VSS or .VSSEG. These files contain the voxel data in 16 pixel × 16 pixel × 16 pixel cubes, including mipmaps, as well as the metadata for the segments. To allow

TABLE 2 | Available data and file formats for importing and exporting in Volume Annotation and Segmentation Tool (VAST) and VastTools.


segmentation files to store arbitrary subregions of a large data set and to make arbitrary extension of those regions possible as users continue to trace, a tree of file-internal pointer blocks (16 × 16 × 16 pointers per block/tree node) is maintained which references the storage location of different segmentation image blocks within the segmentation file. Pointer blocks are also cached in memory when the segmentation file is opened in VAST for optimal file access speed. Selective storage of subregions of the dataset keeps file sizes small if sparse segmentations are generated on very large image stacks.

#### ORGANIZING DATA IN VAST

In VAST, image and segmentation data can have attributes in three separate categories; multiple resolutions, semantic hierarchies of segments, and multiple layers showing different data from the same location. The following sections discuss these possibilities.

# Multiresolution Voxel Representation of Image and Segmentation Stacks

Different from many other segmentation tools for large image stacks, VAST uses voxels not only to represent image stacks, but also segmentations. A voxel is a three-dimensional image element, equivalent to the two-dimensional pixel. Analogous to the brightness or color value of a voxel in a microscopic image stack, in a segmentation layer in VAST each voxel stores one ID (segment identifier number). When VAST displays segmentation data, these IDs are translated to colors based on a metadata table.

Voxel representations of segmentations are typically more memory-intensive than vectorized outlines (Fiala, 2005) or 'area lists' (Cardona et al., 2012), because an ID value has to be stored for each voxel rather than a set of coordinate points and edges around the perimeter of the segment region. However, a segmentation stored in voxels has a more direct relationship to the image stack it is based on, can directly represent segmentation data produced by machine learning algorithms which is typically also voxel-based, and allows for voxel-by-voxel masking and filling.

While scaling of vector data is straight-forward in computer graphics, voxel data is more difficult to handle. In VAST, image and segmentation stacks use mipmaps to limit the amount of voxel data which must be handled at one time. Mipmaps are lower-resolution versions of the original images. VAST uses powers-of-two XY mipmaps, which means that for example for original images of 4096 pixel × 4096 pixel, there will also be versions with 2048 pixel × 2048 pixel, 1024 pixel × 1024 pixel, 512 pixel × 512 pixel, and 256 pixel × 256 pixel available. Because the number of pixels on a computer screen is limited, the data which has to be loaded and displayed to fill the screen is always limited by the extent of the display area and the on-screen resolution. Like Google Maps, by loading just the part of the data set at the resolution necessary for the current zoom level and view region, VAST loads and caches data for display from the center of the view area outwards as needed.

Volume Annotation and Segmentation Tool implements LRU (least-recently-used) memory caches for image and segmentation data. LRU caches remove the least recently used image blocks first when memory runs low. Individual cache blocks can be locked and marked as modified. For segmentation data, VAST preferentially removes unmodified cache blocks. Modified cache blocks cannot be deleted, so disk buffering is used when the memory cache overflows with modified cache blocks.

## Organization of Segments in Hierarchies

Grouping objects into different classes and representing each object as a hierarchy of parts can be an important intermediate step for further analysis of labeled objects in a data set. For example, counting spines or measuring their volume and other morphological properties becomes much easier once each spine is represented as a separate sub-object of its dendrite. In VAST, segments can be arranged in a configurable hierarchy tree which is visualized in the 'Segment Colors' tool window. VAST uses the segment hierarchy tree to selectively color and display objects in different sub-branches. Objects in collapsed folders are shown in the folder color, and transparency can be separately controlled for the selected segment and its children versus all other segments, so that users can hide or highlight semantic groups of objects (see **Figure 2**). The same hierarchy is also used for selective exporting, and for operations on the segmentation layer like drag and drop, deleting and welding of branches. The tree structure can be exported or accessed via the API together with the rest of the segmentation metadata (segment IDs, labels, anchor points, bounding boxes) and analyzed externally.

Volume Annotation and Segmentation Tool provides a "Collect" tool which can be used to collect (translocate) segments which are clicked in the 2D view into the selected folder in the segment hierarchy. This can be used to quickly classify segments into different classes represented by different folders in the hierarchy. Since each segment stores an anchor point and a label text, segments can also be used as bookmarks to annotate and store locations of interest. The paint color can be used as a visual marker.

#### Working With Multiple Image and Segmentation Stack Layers

Volume Annotation and Segmentation Tool can load multiple image and segmentation layers of the same dataset at the same time, visualize them together in 2D and 3D, and use the data in one layer to guide labeling in another (trans-layer masking). Both 8-bit graylevel and 24-bit RGB images are supported for image layers. Segmentation layers are currently limited to 16-bit values per pixel (65535 segments maximally). Since VAST can load multiple graylevel and RGB image stack layers, it can also be used to visualize and annotate multi-channel light-microscopic image stacks. VAST can assign an arbitrary color to each channel, and filter and blend the layers in several ways (see **Figure 3**). Using masking, 3D models can be traced semi-automatically from optical image data. For combined light and electron microscopy (CLEM) data sets, once aligned, EM and LM image stacks can be loaded together and superimposed, for example to allow for easy

identification of structures in the electron-microscopic images which are fluorescently labeled in the light-microscopic image stack.

Volume Annotation and Segmentation Tool can also load more than one segmentation layer at once. This can be useful if single pixels should be given more than one ID. For example, one segmentation layer can be used to trace out axons and dendrites, and a second one is used to trace organelles (mitochondria, vesicles, synaptic contacts; see **Figures 6D–F**). The overlap of the labels in different layers can then be used by an external script to analyze which organelles are in which axons and dendrites, and which axons and dendrites are connected with synapses and where. This method was used to compute the synaptic connectivity in (Kasthuri et al., 2015).

Another application is to load different parts of a segmentation and display them together, if those parts are stored in separate files, for example if several people work on segmenting different parts of the same data set.

#### GENERATING AND EDITING SEGMENTATIONS

#### Manual Segmentation by Drawing and Filling

Volume Annotation and Segmentation Tool provides a pen tool with adjustable tooltip size and a 3D fill tool for manual editing. They can be accessed via the pencil and the paint bucket button in the toolbar. A pipette tool is also provided which selects the segment clicked in the 2D view for painting.

In VAST, users always edit the segmentation at the currently viewed mipmap resolution. A segmentation layer can thus contain segmented objects at different resolutions, and VAST automatically combines the information from different mip levels as the user moves and zooms through the data set. This allows for voxel painting on very large image stacks. Voxel painting speed in VAST is independent of the zoom level, allowing users to paint very large regions (gross morphology, cell bodies) at low resolution as instantly as fine axonal processes at high resolution. To our knowledge VAST is the first and only application in existence which provides this functionality.

Editing in VAST is non-destructive in the sense that the source segmentation files are not changed unless the user saves changes back to the opened file explicitly. All changes are kept in VASTs cache system until the user saves them or discards them by exiting the program.

The parameters of the pen tool are accessible in the "Drawing Properties" tool window (see **Figure 1D**). The diameter of the tooltip can be set to specific values and locked. If "Fill" is enabled in the "Drawing Properties" tool window, VAST will automatically fill empty closed contours as they are drawn. Users can choose to paint on all voxels or to restrict painting to only empty voxels or to voxels of the direct parent segment of the selected segment in the hierarchy. Painting and filling can be restricted to a specified mip level, so that a required resolution can be guaranteed. If the "Max Paint Depth" option is set to a nonzero value, VAST will look in the Z-stack for voxels with the paint color (selected segment ID) at the same XY coordinates in neighboring sections and fill the gap in Z with the paint color, up to the specified distance. This can be used to speed up rough manual tracing, for example by setting "Max Paint Depth" to +-8 and painting outlines of the object only in every eighth section. VAST will fill in the vertical overlap between painted outlines in the seven sections which were skipped. Users can then check and correct the segmentation where needed to refine the object shape.

When users hold down the "Delete" key, which can be mapped to buttons on the pen of a tablet, the pen will erase instead of painting. Holding down the "Shift" key will allow users to pick colors from the 2D view. As long as the "Control" key is held down, users can pan the view with the pen or mouse instead of painting. Holding down the "Tab" key allows for quick changes of the pen tooltip size. These modifiers allow for rapid and intuitive access to the different functions needed during manual painting.

The parameters of the "Fill" tool can be set in the "Filling Properties" tool window (**Figure 1E**). Some of the options are (linked) duplicates of the same options in the "Drawing Properties" tool window described above. The fill tool can be used to recolor complete connected components of segments with a single click, but it shows its real power when used in combination with trans-layer masking, as described below.

#### Working With Automatic Segmentations

While VAST does not generate automatic segmentations itself, it can be used to proof-read automatic segmentations generated elsewhere. Automatic segmentations can be loaded into VAST in two fundamentally different ways. First, they can be loaded as an image layer and used to guide manual segmentation in a separate segmentation layer. This allows users to generate a separate proof-read copy of the segmentation data. Second, if the automatic segmentation does not exceed the 16-bit limit for number of segments (65,535), it can be loaded as a segmentation layer itself and then edited by the user with split and merge operations as well as fully arbitrary manual corrections. The VAST download page contains a link to Youtube tutorial videos illustrating the proofreading process (specifically<sup>2</sup>,<sup>3</sup> ).

#### Using Automatic Segmentations to Guide Manual Painting

Volume Annotation and Segmentation Tool can use two types of automatic segmentation results to guide manual painting: boundary (probability) maps, in which each voxel is assigned its probability to be located on a boundary between objects, and candidate segmentations, in which each voxel stores an object ID. In both cases the automatic segmentation is loaded as an image layer (source layer) and the "Masking" feature is used to constrain the painting area in a separate segmentation layer (target layer) based on information from the source layer (see **Figure 4**). In this mode, parts of the automatic segmentation can be transferred from the source to the target layer by the user, keeping the "raw" source and the "proof-read" target segmentations strictly separate.

To use a boundary probability map for this process, it is selected as the source layer, and VAST's "Masking" mode is set up to restrict painting to contiguous regions of "interior" voxels only (excluding the boundaries) by setting an appropriate source layer pixel value range. Either the pen or the fill tool can then be used to create segments with boundaries defined by the boundary probability map.

If the automatic segmentation in the source layer provides candidate objects or supervoxels instead, masking can be set up so that for each pen stroke or fill operation, the color in the source layer is picked and painting of the new segment in the target layer is constrained to the region in which the source layer has the picked color (as shown in **Figure 4**). In this mode, single segments can be copied from the source to the target layer with the pen and fill tools, which allows for correction of split and merge errors. Another strategy is described in

section "Correcting Automatic Segmentations in VAST". VAST supports 8-bit and 24-bit image layers, which restricts the number of representable distinct objects to a maximum of 16,777,215. However, data sets with more segments can be loaded as well, though only 24 bits of their object IDs will be available in VAST. This can lead to mergers of neighboring objects in rare cases, if the originally different IDs are represented by the same 24 bit number. Those mergers can however be corrected easily by the user. In this manner segmentations with virtually unlimited numbers of segments can be used to guide the generation of a proof-read segmentation. Care should be taken during importing such that segments with IDs of multiples of 2<sup>24</sup> are not mapped to 0 and disappear.

#### Correcting Automatic Segmentations in VAST

When an automatic segmentation is loaded into VAST as a segmentation layer, the user has full freedom to manipulate the segments by painting or erasing. Split errors (cases in which a single object consists of several parts in the segmentation) can be corrected non-destructively by collecting all parts of the object into a folder using the 'Collect' tool. If necessary, all segments of a folder can be "welded" to a single segment. Merge errors (several actual objects appear as a single object in the segmentation) are more difficult to correct because the user must define where within the merged object the boundaries should be. To correct mergers, the user can paint over part of a segment with a different segment. For this, VAST's "Parent" mode can be used, which restricts painting to the immediate parent of the current paint color in the segment hierarchy. One side of the split boundary is relabeled in a different segment color by painting, and the remaining part of the branch which should be split off is recolored by filling. Alternatively, the user can split an object into separate connected components by erasing the segment at the point of connection to disconnect the parts, and then use the fill tool to change the segment ID of one of the connected components.

<sup>2</sup>http://www.youtube.com/watch?v=BZ\_0TVMSdjA

<sup>3</sup>http://www.youtube.com/watch?v=4XCNRgDzSjc

Afterwards the boundary where the split was performed can be cleaned up by manual painting.

On-demand automatic segmentation of very large data sets is in principle possible if the segmentation image stack is loaded from a web server and the segmentation is done on the server side. VAST would request parts of the automatic segmentation from the server, which would do the on-demand computation and send the result to VAST. This approach could for example be used to trace axons quickly over very large distances without requiring a complete automatic segmentation of the whole data set.

# Multi-User Segmentation, Splitting and Merging of Segmentations

Even though VAST is not a client-server solution where multiple users can edit a single segmentation at the same time, by requesting data from and committing changes to a central data server, it is possible to have several scientists work on the same dataset and combine the results. VAST provides a merge function by which several .VSS segmentation files can be merged into one, with options to define voxel overwrite and segment ID renumbering rules. When merging a 'source' segmentation onto a 'target' segmentation, conflicts are resolved on a voxelby-voxel basis. The user can decide whether nonzero target voxels can be overwritten by merged-in source data (source precedence) or are write protected (allowing only empty target voxels to be written; target precedence). VAST can also export and import segmentations as image stacks with metadata, allowing for more complicated merge procedures done externally. Also when importing segmentation images onto an existing segmentation, either source or target precedence can be applied.

Parts of segmentation files can be recombined by using branch deleting or branch exporting and merging. A selected segment or branch of segments (selected segment and child tree in the segmentation hierarchy) can be saved to a separate .VSS file using "File/Save Segmentation As Special . . ." and subsequently merged with a target segmentation file. Alternatively, the "Delete Segment + Subtree" function from the context menu of the "Segment Colors" tool window can be used to delete all unwanted segments,

∼100 µm × 100 µm × 200 µm tissue volume. Tracing this cell took ∼15 h for a single expert. (D) Organelles in a neuron cell body (same cell as Figure 5C) shown from two directions. Nucleus white, with pores visible; endoplasmic reticulum green, mitochondria yellow, Golgi apparatus blue. (E) Two spiny apical dendrites with side branches in rat cortex. Synapses shown in yellow. (F) Spiny dendrite in red, with transparent axons making synapses on it. Neurotransmitter vesicles (white) were exported using particle clouds to generate spherical vesicles. All images are based on EM data from the lab of Jeff Lichtman, Harvard. All segmentations except C were done fully manually. A,B,D,F used data published with (Kasthuri et al., 2015).

and the result can be saved to a separate .VSS file and then processed further.

#### DATA VISUALIZATION, EXPORTING, AND ANALYSIS

#### The Integrated 3D Viewer

The 3D viewer in VAST can be used to inspect and visualize image and segmentation stacks. It makes use of a volumetric texture with transparency rather than surface meshes to display voxel data in its native format. Because the 3D textures are retrieved from the 2D view, the same coloring and blending options are available, and image and segmentation stacks can be visualized together. The transparency of the 3D textures can either be derived from the pixel brightness or set to opaque to show a full cube. The view can be rotated and zoomed and screenshot images can be exported. The user can also click on objects in the 3D viewer to set the 2D window to the same location. **Figure 5** shows a selection of examples rendered with the 3D viewer.

#### Image Stack Exporting

For further processing and visualization of voxel data in other applications, VAST includes functions to write image and segmentation data back to stacks of image files. The image stack export dialog can be found under "File/Export . . ." in the main menu. VAST can export image stacks of data in single image layers, segmentation layer data (images encoding IDs in blue and green color channels), and the composited 2D view ("screenshots"), of a definable sub-region of the loaded data set, at

All image layers share the image cache and all segmentation layers share the segmentation cache. (F) Shows the threading structure and the control flow of the main thread. Even though each layer has its own loader thread, cache updating is done in the main thread only to prevent multithreading problems.

a definable mipmap resolution. Large sections can be exported as a mosaic of image tiles. Metadata for segmentation layers can be exported to text files using the "Save Segmentation Metadata . . ." function in the menu of the "Segment Colors" tool window. VAST provides a Matlab script to parse these text files. **Table 2** lists the image formats which are available for image stack exporting.

## The VAST API and VastTools

Volume Annotation and Segmentation Tool includes an API which can be accessed from client programs through the TCP/IP network protocol, either locally on the same computer or remotely through a network connection. Once the TCP/IP port is enabled in VAST, client programs can connect and send API commands to VAST to exchange data. The protocol for the API and all API functions are documented in the user manual, which is part of the supplementary package which can be saved to disk from VAST (under "Info/Save Documentation .ZIP To Disk . . .") or downloaded from the VAST webpage<sup>4</sup> .

"VastTools.m" is a Matlab (The Mathworks, Inc.) script which can communicate with VAST via its API. It implements a number of supplementary functions, in particular for data exporting. VastTools is also included in VAST's supplementary package. Since it is a Matlab script, its source code is fully readable and it can serve as a reference for implementing other client programs. Also, once VastTools is running and connected to VAST, other Matlab scripts can simply call API functions through the hidden global variable "vdata.vast".

The export functions of VastTools are summarized in **Table 2**. Most importantly, VastTools can export surface meshes of segmented objects to generic Wavefront OBJ files, which can then be imported into 3D rendering applications like 3ds Max (Autodesk, Inc.; see example images in **Figure 6**), Blender (Blender Foundation) or Unity (Unity Technologies). It can also export isosurface meshes (based on voxel brightness) which can for example be used to visualize fluorescence signals in light microscopic image stacks, and particle clouds, where a prototype 3D object (for example a small sphere representing a vesicle) is placed at the centers of all separately painted regions and the compound object is exported (3D object instancing). Furthermore, 3D boxes at specific locations in the data set can be exported as wireframe models or with single-color or textured sides, with a texture derived automatically from the loaded image

<sup>4</sup>http://software.rc.fas.harvard.edu/lichtman/vast/

stack. Finally, VastTools can export correctly sized scale bar models. **Figure 6** shows a selection of renderings made from 3D models labeled in and exported from VAST.

VastTools can also render and export projection images along cardinal axes from the loaded data with multiple options, including illumination simulation. These projection images can not only be exported but also be used for single-image-click 3D navigation in VastTools' "Simple Navigator Image" functionality.

"Target Lists" in VastTools can be used to create and organize annotated lists of particular points of interest in the data set loaded in VAST. Target coordinates are stored in the list together with a zoom level and the ID of the selected segment, and text descriptions can be added to each target. The corresponding 2D view of any listed target can then be restored in VAST with a single click in the target list.

Even though VAST is not a specialized analysis tool, several measurement functions are included in VastTools. First, Euclidian distances between any two points in the image stack can be measured. Second, VastTools can calculate the volumes of the volumetrically labeled segments and estimate their surface area during mesh exporting by summing the area of all generated mesh triangles for the object. This measurement is however likely to provide an overestimate of the true surface area because of the roughness of voxel models. A better estimate could be generated externally after smoothing the 3D models. Finally, VastTools can count the number of separately painted regions and export a list of centroid coordinates during particle cloud exporting.

The features discussed here are described in more detail in VASTs user manual.

### DISCUSSION

Volume Annotation and Segmentation Tool is a light-weight and versatile tool specialized for volumetric annotation and segmentation of objects in very large image stacks. It is a selfcontained program which is simple to set up and use even for inexperienced users. Typical applications include: Exploring very large EM and LM image stacks interactively; efficient manual segmentation of arbitrary structures to generate voxel training data for automatic segmentation algorithms; and sparse volumetric tracing of objects of interest, either manually or assisted by an automatic segmentation of the volume. VAST can also be used to proofread automatic segmentations, which is likely to become an important part of connectomic studies now that automated methods are becoming more commonplace.

Volume Annotation and Segmentation Tool is the only tool to our knowledge which can load multiple image and segmentation layers at the same time and use trans-layer masking to speed up manual segmentation and to proof-read automatic segmentations. For example, an automatic segmentation, even if it contains many errors, can be loaded as a separate layer and used to provide masks for painting objects with perfect outlines. Users can then volumetrically label neurites instantly while scrolling through the data set, while precise boundaries of the segmented objects are defined by the automatic segmentation. The 3D filling function speeds this process up further. Any defects caused by splits and mergers in the automatic segmentation can be corrected immediately by painting and filling. This process can also be seen as proof-reading a segmentation, in which original and proof-read versions are kept strictly separate because they reside in different layers.

In this way, volumetric voxel painting in VAST can reach speeds comparable to manual skeleton tracing based on placing nodes while scrolling through a 3D EM image stack. In both cases the speed is likely limited by either the rate at which images can be loaded and displayed, or by the speed at which humans can reliably follow a winding process through the neuropil. Voxel painting immediately generates filledin areas of object cross-sections, which makes it easier for human observers to spot splits and mergers. Also, it natively produces volumetric objects, which for skeleton tracing requires an additional step. Also, different from tools which perform segmentation proof-reading by agglomerating object parts, in VAST proof-read segments are represented as voxels, which gives the user full freedom to modify them by painting as needed.

With VAST, these operations can be performed on image stacks which can reach Petabytes, including datasets which are streamed from a network source. As datasets grow larger and more and more research labs get interested in reusing these data-rich EM image volumes to address various scientific questions, online hosting and remote access will become more commonplace.

## Limitations

However, there are data sets and segmentation tasks for which VAST in its current form is not suitable. First of all, VAST was designed as a single-user standalone program, and as such it is not a multi-user client-server solution in which multiple annotators can simultaneously contribute to a shared server-hosted segmentation. While VAST allows image stacks to be loaded dynamically from a server, segmentation layers are currently hosted in local files only. It is possible to combine the segmentation results from multiple users by merging segmentation files, but this is comparatively slow and inconvenient. Therefore it is tricky to use VAST in settings where numerous annotators are working together to segment or proof-read the same segmentation volume. A future version may implement a segmentation layer type in which the data is hosted on and synchronized with a server, for interactive multi-user editing. The server would also keep track of the changes each user makes (provenance tracking and version control), which becomes more important in multi-user settings. In VAST, provenance can be tracked rudimentarily by saving to a new file every time and keeping the old versions, and having different users keep their work separate.

Second, as a voxel painting program, VAST is not equipped to work with skeletons or surface mesh data directly. It is possible to use voxel painting to indicate skeleton lines of 3D objects, and to analyze these externally to generate skeletons (Morgan et al., 2016), but this is more a work-around than a true solution. VAST does not use surface meshes internally either; for example the 3D viewer uses a three-dimensional texture instead.

Surface meshes of segmented objects can however be generated in VastTools for external analysis and rendering. Measurement of lengths, diameters, axonal branching patterns etc. are also not included in VAST but have to be performed externally.

When using skeletons, synapses between two neurons can be represented by edges of a special 'synapse' type, bridging between the skeletons of two cells. This is not possible in VAST since it does not provide skeletons. However, voxel annotation of synapses can be (and has been) used to compute synaptic connectivity (Kasthuri et al., 2015; Morgan et al., 2016; Quadrato et al., 2017). If synapses are manually or automatically labeled so that the synapse region overlaps with the pre- and postsynaptic partner neurites volumetrically, the synapse label can then be used as a mask to extract the IDs of the connected neurites automatically. This can be done with an external script which uses either the API or exported segmentation image stacks.

While VAST has no problem allowing users to label all the synapses in a dataset, it is not a tool specialized for the analysis of connectivity structure or other more sophisticated analyses of tissue morphology. Data on synaptic partners can be exported and analyzed downstream with specific tools, as was done in (Kasthuri et al., 2015; Morgan et al., 2016; Quadrato et al., 2017). The possible future addition of explicit skeletons associated with the volumetric labels in VAST may make many of these tasks more straight-forward.

Currently, the 2D view in VAST is limited to XY sections, mainly because of our anisotropic ATUM data sets and the fact that using only XY mip maps saves time and storage space. Data can in principle be resliced externally if tracing at a different orientation is preferable. However, supporting different mip mapping and 2D slicing options, as well as improved capabilities of the 3D viewer, may be useful features in the future.

Finally, VAST is based on the Windows user interface and graphics system (Direct3D 11) for speed and simplicity, and cannot easily be ported to other operating systems. Also it is currently not an open source program, and feature additions and bug fixes depend on the developers. However, its API is fully documented and many functions can be accessed remotely for external processing and script-based automation to add custom functionality.

Though VAST is still being developed further and more features are added as needed, its main strength is manual and semiautomatic segmentation by voxel painting. Extensive import and export functions are provided, including an API, so that VAST can play its role as a powerful tool in a larger pipeline.

# REFERENCES


## DATA AVAILABILITY STATEMENT

No datasets were generated for this study. At the time of writing of this manuscript, the current version of VAST can be downloaded at: https://software.rc.fas.harvard.edu/lichtman/ vast/

#### AUTHOR CONTRIBUTIONS

DB wrote VAST. HS and JL supported the development and provided advice and feedback. DB and JL wrote the manuscript with input from HS.

# FUNDING

This work was supported by the Gatsby Charitable Foundation, the Howard Hughes Medical Institute, the Human Frontier Science Program, an Anonymous Donor, and the National Institutes of Health (HS); and NIH/NINDS (Sensorimotor processing, decision making, and internal states: toward a realistic multiscale circuit model of the larval zebrafish brain; U19 NS104653-01), IARPA/Department of the Interior (Algorithms for representation and inference informed by the acquisition of data from neuroscience experiments, ARIADNE; D16PC00002), NIH/NIMH (Developmental origins of mental illness: evolution and reversibility; Conte Center Award, P50MH094271), the Department of Defense (Columbia University, Imaging how a neuron computes; GG008784), and The Broad Institute (High resolution histological analysis of human brain tissue from patients with neuropsychiatric and neurodevelopmental disorders; 6600029- 5500000959) (JL).

#### ACKNOWLEDGMENTS

We would like to thank the Openconnecto.me team at Johns Hopkins University and the Google Brainmaps team for allowing VAST to connect to their image data servers; Xiaotang Lu, Tao Fang, and Hidde Ploegh for the example data in **Figure 3**; and Toufiq Parag, Lee Kamentsky, and Hanspeter Pfister for the automatic segmentation used as basis for **Figure 6C**.

Bock, D. D., Lee, W. C., Kerlin, A. M., Andermann, M. L., Hood, G., Wetzel, A. W., et al. (2011). Network anatomy and in vivo physiology of visual cortical neurons. Nature 471, 177–182. doi: 10.1038/nature09802



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Berger, Seung and Lichtman. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Methods for Mapping Neuronal Activity to Synaptic Connectivity: Lessons From Larval Zebrafish

Adrian A. Wanner\* and Ashwin Vishwanathan\*

Princeton Neuroscience Institute, Princeton University, Princeton, NJ, United States

For a mechanistic understanding of neuronal circuits in the brain, a detailed description of information flow is necessary. Thereby it is crucial to link neuron function to the underlying circuit structure. Multiphoton calcium imaging is the standard technique to record the activity of hundreds of neurons simultaneously. Similarly, recent advances in high-throughput electron microscopy techniques allow for the reconstruction of synaptic resolution wiring diagrams. These two methods can be combined to study both function and structure in the same specimen. Due to its small size and optical transparency, the larval zebrafish brain is one of the very few vertebrate systems where both, activity and connectivity of all neurons from entire, anatomically defined brain regions, can be analyzed. Here, we describe different methods and the tools required for combining multiphoton microscopy with dense circuit reconstruction from electron microscopy stacks of entire brain regions in the larval zebrafish.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Mario Treviño, Universidad de Guadalajara, Mexico Chen-Min Yeh, Salk Institute for Biological Studies, United States Joel C. Bornstein, The University of Melbourne, Australia

#### \*Correspondence:

Adrian A. Wanner adrian.wanner@princeton.edu Ashwin Vishwanathan ashwinv@princeton.edu

Received: 14 June 2018 Accepted: 28 September 2018 Published: 25 October 2018

#### Citation:

Wanner AA and Vishwanathan A (2018) Methods for Mapping Neuronal Activity to Synaptic Connectivity: Lessons From Larval Zebrafish. Front. Neural Circuits 12:89. doi: 10.3389/fncir.2018.00089 Keywords: zebrafish, connectome, olfactory bulb, hind brain neurons, electron microscopy, two-photon (2P), neural circuit

# 1. INTRODUCTION

The larval zebrafish has been gaining a lot of traction as a model system in systems neuroscience (Friedrich et al., 2010). From a model system point of view for neuroscience, the larval zebrafish sits in between the fly and the mouse, two of the most popular model systems. At larval stage, 4–7 days post fertilization (dpf), these fish have approximately 100,000 neurons in their nervous systems (Hill et al., 2003). The larvae are accessible to a variety of tools that include advanced genetic manipulation, high-throughput screening, behavioral assays, electrophysiology, and functional imaging. More importantly, at their larval stage they are optically transparent which makes them accessible for functional imaging and allows brain-wide monitoring of neuronal activity (Ahrens et al., 2013; Dal Maschio et al., 2017). To this end many studies use the larval zebrafish to study mechanisms by which activity in networks of neurons can lead to meaningful sensory processing and eventually behavior. In terms of behaviors the larvae display a rich set of behaviors like prey-capture, looming, and foraging that can be studied either in freely moving animals or in a virtual environment where typically the read out is in the form of eye and tail movements (Wyart et al., 2009; Ahrens et al., 2012; Bianco and Engert, 2015; Temizer et al., 2015; Dunn et al., 2016a,b; Naumann et al., 2016) and have thus been used extensively to investigate sensorimotor transformations (Mathuru et al., 2012; Barker and Baier, 2015).

From a systems neuroscience perspective, having access to the function, genetics, structure, and a wiring diagram of the neurons involved is key to understanding how fundamental computations are performed in the brain. Recent studies have shown how it is possible to extract single-cell transcriptomic data from entire brains or regions (Pandey et al., 2018; Raj et al., 2018). Similarly, it is also possible to functionally image the activity of the entire nervous system (Ahrens et al., 2013; Dal Maschio et al., 2017). However, detailed ultrastructural information on connectivity of the underlying neuronal circuits is still missing for most zebrafish brain regions. For a mechanistic understanding of neuronal computations and information flow it is essential to reconstruct the neuronal circuits at synaptic resolution. While low resolution electron microscopy (EM) and light microscopy is sufficient for mapping axonal projection patterns and somata locations (Randlett et al., 2015; Förster et al., 2017a; Hildebrand et al., 2017), a detailed mapping of neuronal circuits requires EM at synaptic resolution (Wanner et al., 2016b; Vishwanathan et al., 2017; Svara et al., 2018).

We present here a comprehensive step-by-step guide for neuronal circuit reconstruction in the larval zebrafish using serial block-face scanning electron microscopy (SBEM) or automated tape collecting ultra-microtome based scanning electron microscopy (ATUM-SEM). We outline the typical workflow for dense and targeted reconstruction of connectivity and activity of neurons using correlative light and electron microscopy. We exemplify the workflow by highlighting two recent studies in which significant parts of neuronal circuits have been reconstructed at synaptic resolution in the olfactory bulb and the hindbrain of the larval zebrafish, respectively (**Table 1**) (Wanner et al., 2016b; Vishwanathan et al., 2017). The main factors contributing to the quality of a volume EM stack are tissue preservation, voxel resolution, image contrast, and image registration. The most time consuming step and at the same time the major bottleneck in combining functional imaging and EMbased circuit analysis is the neuron reconstruction and synapse annotation. The accuracy and efficiency of image annotation depends highly on the quality of the underlying EM image stacks, thus it is crucial to optimize the EM preparation and image acquisition for the subsequent circuit reconstruction. Each of these steps takes significant time to optimize and to get right. By comparing the two methods that were used we hope to provide the reader a detailed overview of the methods and tools required for accomplishing such reconstructions.

## 2. FUNCTIONAL IMAGING

The recent advances in optogenetic tools and light microscopy (LM) have revolutionized population scale imaging of neuronal activity at cellular resolution. The advent of better transgenic tools (Halpern et al., 2008; Kimura et al., 2014; Marquart et al., 2015; Förster et al., 2017b), calcium reporters (Chen et al., 2013; Piatkevich et al., 2018), imaging techniques like light sheet microscopy (Ahrens et al., 2013; Panier et al., 2013) and two photon microscopy (O'Malley et al., 1996; Friedrich and Korsching, 1997; Ritter et al., 2001; Brustein et al., 2003; Niell and Smith, 2005; Yaksi and Friedrich, 2006; Orger et al., 2008; Ramdya and Engert, 2008; Sumbre et al., 2008; Naumann et al., 2010, 2016; Niessing and Friedrich, 2010; Blumhagen et al., 2011; Zhu et al., 2013; Kubo et al., 2014; Portugues et al., 2014; Candelier et al., 2015; Romano et al., 2015; Pérez-Schuster et al., 2016; Dal Maschio et al., 2017; Pietri et al., 2017) and data analysis (Miri et al., 2011; Freeman et al., 2014) have allowed for the imaging and interpretation of whole brain volumes at high spatial and temporal resolution. Typically, the temporal resolution of these experiments is on the order of few seconds to few miliseconds, enabling to measure neuronal activity with single spike resolution. It is also possible to image the entire brain during free swimming, more close to naturalistic behaviors (Kim et al., 2017). A detailed description of the factors that need to be considered for using two-photon imaging on larval zebrafish have been covered previously (Renninger and Orger, 2013). Instead, we highlight in the following somee important factors that need to be considered for combining functional imaging and EM-based circuit reconstruction.

The acquisition parameters of light microscopes are usually optimized for maximizing temporal resolution and signal-tonoise (SNR) of the activity measurements while minimizing the observable photo damage. At light microscopy level, photo damage is most prominently observable as photo bleaching (Magidson and Khodjakov, 2013). While a comprehensive study of photo damage at ultra structural level in combination with electron microscopy is still missing, several labs and researchers have observed and anecdotally reported that extended LM imaging prior to EM sample preparation can affect the tissue, ultra-structural integrity and staining quality in the subsequent EM steps, even if signs of photo damage are missing on the light microscopy level. It is therefore crucial to reduce photodamage beyond avoiding photo bleaching. On one hand, this can be achieved by decreasing the laser power under the objective which comes at the cost of sacrificing SNR. On the other hand, decreasing the photon dose by decreasing the dwell time and increasing the imaging rates seems also to reduce photo-toxic effects. The loss in SNR can be compensated partially by using improved transgenic or synthetic reporters.

# 3. STRUCTURAL IMAGING

Following functional imaging, the same larvae are prepared for EM. During this process the ultrastructure of the tissue is preserved and stained using a combination of fixatives and heavy metal stains.

#### 3.1. Immersion Fixation and Craniotomy

The tissue fixation is one of the most important steps toward good preservation and staining of cellular ultrastructure. The larval skull consists of soft cartilage covered by connective tissue and skin that hinders the penetration of aldehydes. This layer gets typically removed by a craniotomy. To allow for fast and homogeneous penetration of fixatives such as paraformaldehyde and glutaraldehyde we strongly suggest performing a craniotomy around the brain region of interest as follows (see Wanner et al., 2016b; Vishwanathan et al., 2017 for details on animal procedures):

1. Anesthetize the larva by putting it into a small drop of larval medium (E3 medium) and the anesthetic MS222 (0.1 mg/ml, Sigma E10521).

#### TABLE 1 | Dataset comparison.



TABLE 2 | Fixation and staining comparison.


# 3.2. Electron Microscopy Staining, Embedding, and Sectioning

For heavy metal staining we used conventional reduced Osmium (ROTO) based techniques that impart good contrast to the samples (Deerinck et al., 2010). Briefly, this involves staining with reduced Osmium followed by amplification with thiocarbohydroazine (TCH) and another round of Osmium. This is further amplified by en blocstaining of the samples with Uranyl acetate and Lead aspartate (**Table 2**). We used two different volume EM techniques to acquire large stacks of the zebrafish brain. The olfactory bulb dataset (**Figure 1**) was acquired with a serial block-face scanning electron microscope (SBEM) (Denk and Horstmann, 2004), whereas the hindbrain dataset (**Figure 2**) was acquired using automated tape collecting ultra-microtome based scanning electron microscopy (ATUM-SEM) (Schalek et al., 2011). During this process, we have encountered few failure modes as listed in **Table 3**.

During the staining of the tissue we observed poor penetration or precipitation of the stain in the form of large contrast gradients in the images. In both cases, beginning with good craniotomies was able to mitigate these problems. Occasionally, reducing the amount of TCH helped to reduce the occurrence of precipitates. Depending on the acquisition method, different resins should be used for embedding. In SBEM, surface charging due to electrons that accumulate in regions with low conductivity/heavy metal content is a common problem (Wanner et al., 2015; Titze and Genoud, 2016). Besides the image saturation due to accumulating electrons, the charging can impair the cutting quality and even more importantly, it can lead to non-linear, non-stationary distortions in the images, which can complicate the stitching of a mosaic of overlapping tiles tremendously. These effects are in particular problematic for zebrafish samples, because there is typically empty resin surrounding the brain tissue. One way to cope with this problem is to use variable pressure SEM (Griffin, 2007). This technique reduces charging artifacts by adding a gaseous agent into the recording chamber (e.g., water or nitrogen) whose molecules get ionized and neutralize excessive electrons on the block surface. These agents typically compromise the vacuum in the chamber and scatter electrons which can severely affect the resolution and SNR in the images. However, a promising new focal gas injection-based charge compensation seems to largely mitigate the charging problems without compromising the SNR (Deerinck et al., 2017). Another, technically more challenging method is to introduce a sputtercoating device into the recording chamber that coats the sample surface after each cut with a thin layer of Chromium or Palladium and makes its surface perfectly conductive (Titze and Denk, 2013). An alternative approach is to improve the grounding of the tissue and the sample block by adding conductive material such as carbon black (Nguyen et al., 2016) into the otherwise empty resin space surrounding the tissue. In the case of the larval OB image stack (Wanner et al., 2016b) an alternative embedding method called Epo-tek and Epon (EE) embedding was developed, in which the tissue was surrounded with a silver-filled epoxy glue before curing the Epon. EE-embedding effectively resolves the surface charging problems during backscattered electron imaging and is therefore suitable for blockface imaging in high vacuum mode. This also results in an order of magnitude increase in both, SNR and acquisition speed (Wanner et al., 2016b). In the following we give a step-by-step description of the procedure for EE-embedding of a resin-immersed tissue sample:


For ATUM-SEM, since the sample is collected prior to imaging, the resin had to be customized in order to facilitate good cutting characteristics. Embedding the samples in most typical resins resulted in the formation of micro-folds and compression of the tissue at the tissue-resin interface. This kind of folds can typically be attributed to the change in the density at the interface between the tissue and the resin. In order to overcome these problems, one approach is to re-embed the sample with resin that has been made more dense by the addition of tissue slurry that acts to homogenize the resin. Another similar approach was to embed the larvae inside a larger piece of tissue that then serves to homogenize the resin (Hildebrand et al., 2017). Our approach was to design a low-viscosity resin, that was able to withstand the compression at the interface while retaining good cutting characteristics (**Table 2**). This resin allowed for collection of 1000's of fold free sections from zebrafish larvae and murine tissue.

# 4. NEURON RECONSTRUCTION

We have employed two different methods for neuron reconstruction. In the case of the larval OB, neurons were skeletonized manually by a cohort of more than 30 professional image annotators (Wanner et al., 2016b), whereas in the case of the hindbrain, crowd-sourced players and professional image annotators proofread an automated, volumetric reconstruction (Kim et al., 2014). There are pros and cons to each of these methods, and here we list some of them, based on our experiences.

FIGURE 1 | Workflow for SBEM based pipeline: First, two-photon calcium imaging was performed in the OB and the telencephalon over multiple planes to record neuronal activity while delivering different odor stimuli. Next, the same sample was prepared for EM and a complete stack of the OB and parts of the telencephalon was acquired with a SBEM. Subsequently, all neurons in the OB have been reconstructed by manual skeleton tracing (Wanner et al., 2016b). After the co-registration of the EM stack and the two-photon planes, the neuronal activity can be mapped onto the reconstructed neurons for detailed structure to function comparison and analysis.

(Continued)

FIGURE 2 | registration of EM images, correspondences between LM and EM are used to register LM images onto EM images to locate somata in both volumes. (C) Automatic image segmentation using neural networks is used to generate affinities from raw images, that are then segmented to distinguish neurites from each other. Alternatively, another neural network is used to detect synapses in the entire volume. (D) Dense segmentation is agglomerated to produce entire neurites (left). These neurites are proof-read and corrected for mistakes such as false terminations and mergers to reconstruct entire neurons (middle). Colors represent different classes of neurons. (E) Accuracy of crowd sourced players reported as F1 scores when proof-reading neurons either the first time (round 1) or the second time (round 2). Each gray dot represents an individual player. Black dot and line is the average.

#### TABLE 3 | Failure modes.


#### 4.1. Skeleton Based Reconstructions

Despite the fact that automated image segmentation methods have made tremendous progress in the last few years, manual neuron reconstruction is still the preferred and often more economical approach for small and intermediate sized reconstruction projects involving few hundreds to few thousands of neurons. While manual volumetric annotation is extremely time consuming, skeleton tracing of neurites usually is sufficient for many circuit neuroscience related questions and is orders of magnitudes faster (Helmstaedter et al., 2011). Neurons are traced manually by placing connected nodes onto crosssections of neurites in the image data, many such nodes are then connected to form entire neurons. This is typically done using open-sourced software packages such as Catmaid (Saalfeld et al., 2009), KNOSSOS (Helmstaedter et al., 2011), and PyKNOSSOS (Wanner et al., 2016a). These software tools are specifically designed for high-throughput, multi-user, 3D image annotation and neuron reconstruction. Typically, skeleton tracing is performed by cohorts of students or researchers. A motivated researcher or student can learn a lot about the underlying data while manually annotating neurons, but it is probably not the best use of their talents to trace neurons for several thousands of hours (Helmstaedter et al., 2011). However, crowd sourcing neuron reconstruction and synapse annotation is intrinsically difficult. First, tracing neurons is not trivial and it requires 10–40 h of training for a naive student to become a good annotator (Helmstaedter et al., 2011). Second, neuron tracing is relatively monotonous and only few people are willing to do this kind of work over a prolonged period of time with the necessary care and accuracy. Helmstaedter et al. developed a redundant-skeleton consensus procedure (RESCOP) that can be used for reliable neuron reconstruction with cohorts of weakly trained students. RESCOP was used to densely reconstruct 950 neurons in the inner plexiform layer of a mouse retina (Helmstaedter et al., 2013). However, manual tracing is an error-prone process, even if performed by expert annotators. A single expert annotator misses on average at least 10 percent of the true neuronal arbor (Wanner et al., 2016b). Therefore, expert revision and/or redundant annotation is typically used to leverage the accuracy of the resulting reconstruction (Helmstaedter et al., 2011; Schneider-Mizell et al., 2016; Wanner et al., 2016b). In the case of the mouse inner plexiform layer connectome, RESCOP required an average redundancy of 6 for ganglion cells and 4 for amacrine and bipolar cells.

While tracing straight neurites is typically faster than tracing branching neurites with complex morphology, the tracing speed of a single annotator is on average 2– 15 h per mm neurite length (Helmstaedter et al., 2011; Wanner et al., 2016b; Boergens et al., 2017). Hence, redundant reconstruction can be time consuming and costly. Therefore a new iterative consensus procedure called CORE ("COnvergence by Redundancy and Experts") (Wanner et al., 2016b) was developed to reconstruct >1,000 neurons in the larval zebrafish OB. CORE leverages redundant reconstruction with focused expert input. For the OB reconstruction CORE achieved very high accuracy (F1 score >0.989 for mitral cells) with just a three-fold redundant reconstruction together with local re-tracing at mismatch points and focused expert inspection. Thereby the bulk skeleton tracing was outsourced to professional annotators (www.ariadne.ai).

## 4.2. Segmentation Based Volumetric Reconstructions

Volumetric reconstructions generally mean "coloring" entire neurons, including intracellular regions. In contrast to skeletons, this method effectively captures detailed morphologies of the neurons, including spine architecture and gives an accurate representation of the changes in the thickness of the neurites that originate from the somata. Manual volume annotation, although accurate, is very laborious, time consuming and about 50 times slower than skeletonization (Helmstaedter et al., 2011). Recent advances in machine learning tools such as deep convolutional networks (CNNs) have been developed to segment entire images based on human generated groundtruth annotations (Chklovskii et al., 2010; Jain et al., 2010; Kreshuk et al., 2011; Andres et al., 2012; Berning et al., 2015; Kaynig et al., 2015; Lee et al., 2015; Dorkenwald et al., 2017; Staffler et al., 2017). Typically the process requires (**Figure 2**):


Similar approaches can be employed for other features of interest, for example neural networks can be trained to identify synapses, mitochondria etc. Using the above described methods, we have automatically segmented the hindbrain dataset in order to reconstruct entire volumetric profiles of neurons. To validate and to correct the mistakes that are made by these machine learning algorithms, we use a crowd-sourcing platform where experienced players check the validity of the algorithms and override in regions where the AI makes mistakes (Kim et al., 2014). The typical workflow for a single neuron requires:


To ensure high accuracy, this process is performed twice in a "wikipedia" like manner, where the first player proof-reads and checks for errors in round 1 and a second player then checks that intermediate result in round 2 with the latest player having veto privileges. Finally, the entire neuron is checked by experts, who have >5,000 h of expertise to mark the neurons as complete. Using this process on average we can accurately reconstruct 3–4 neurons per day, with an average of 1.6 mm neurite length per day with a coverage factor of 3, which means each neuron had been reconstructed by 2 players and proof read by 1 expert. The crowd sourced players have F1 scores on average >0.8 as compared to expert tracers.

#### 5. CORRELATION OF FUNCTION TO STRUCTURE

A long standing question in neuroscience is whether and how the structure of neuronal circuits determines their function. A directly related question is to what extent knowledge about circuit structure can predict circuit function (Lichtman and Sanes, 2008; Seung, 2009; Bargmann, 2012; Morgan and Lichtman, 2013). Although fundamental, these questions remain unresolved for many circuits, largely because the detailed analysis of circuit structure, or connectivity, is still a major challenge. The first step involves mapping the neuronal activity from calcium imaging to the reconstructed neurons from EM based circuit reconstruction. To do this it is necessary to precisely register the calcium imaging planes to the electron microscopy image stack. Typically, this is done by iterative point matching and 3D alignment between the LM and EM data. Corresponding landmarks such as prominent blood vessel patterns or unique soma locations can be identified in both datasets. These landmarks can be used to calculate a spatial transform between the LM and the EM data. Tools for performing point matching are available on open sourced platforms (**Table 1** software) and can be easily scripted using built-in functions in Python (www.python.org) and Matlab (www.mathworks. com). The reconstructed connectivity or wiring diagrams can be used for hypothesis testing of circuit models. Using two recent larval zebrafish circuit reconstruction studies, we provide two examples of hypotheses that can be tested from such connectomes.

**Example 1:** In the hindbrain, eye position encoding neurons persistently fire action potentials during eye fixations (McFarland and Fuchs, 1992; Aksay et al., 2000). These neurons transform eye velocity signals to eye position signals and are called velocityto-position-integrator (VPNI) neurons. Theoretical models suggest that persistent activity could be induced by recurrent connectivity between VPNI neurons (Cannon et al., 1983; Seung, 1996). To test this hypothesis we reconstructed the connectivity between functionally identified VPNI neurons (Vishwanathan et al., 2017). From these reconstructions we found that the VPNI neurons are not a homogeneous class of neurons. We observed at least three classes of neurons, two excitatory and one inhibitory, that differed in their morphology, synaptic distribution and axonal targets. We further observed that only the excitatory class of neurons were recurrently connected, which supports the idea of positive feedback as one of the mechanisms by which persistent activity can be implemented.

**Example 2:** In the OB chemically similar odors tend to activate overlapping sets of olfactory glomeruli. This activity is decorrelated and normalized, presumably by interactions between interneurons (INs) and mitral cells (MCs), the major output neurons of the OB (Friedrich and Laurent, 2001; Yaksi et al., 2007; Niessing and Friedrich, 2010; Zhu et al., 2013). However, a mechanistic understanding of these population-level computations is lacking. By the dense reconstruction of all OB neurons we found that most MCs are largely uniglomerular (Wanner et al., 2016b). In contrast, INs tend to innervate multiple glomeruli and the glomerular IN innervation is governed by glomerular identity. Moreover, the examined INs did not have specific input or output glomeruli, implying that interglomerular interactions have a strong non-directional component. As a consequence, selective interglomerular connectivity may support differential preprocessing of odor information that is routed to specific target regions and that is relevant for different behaviors. Moreover, the specific projections between glomeruli may favor inhibitory interactions between processing channels with specific tuning properties which in turn could be an efficient solution for decorrelating activity patterns between small groups of neurons. This kind of questions can only be tested with experiments in which both, connectivity and activity, are measured exhaustively with single neuron resolution.

#### 6. CONCLUSIONS

Detailed anatomical maps and wiring diagrams can be a very powerful tool not only for gaining a mechanistic understanding of brain function, but perhaps even more importantly as a source of inspiration for new models and hypotheses for circuit functions. Here, we presented some of the tools, methods and examples that are required for large scale circuit reconstruction,

### REFERENCES

based on our work in the larval zebrafish. We hope that this article helps lowering the threshold for combining synaptic resolution circuit reconstruction and functional imaging. We highlighted two different sets of methods that were used to study the larval zebrafish. Both highlighted methods have advantages and disadvantages that the end user should consider before embarking on similar studies. Other important factors that have to be considered for large-scale volume EM projects, such as image acquisition speed, have been extensively discussed in previous reviews (Briggman and Bock, 2012; Wanner et al., 2015). Many of the tools that were used in the studies presented here are available in the form of open sourced software (**Table 1**) with more tools becoming available every day, ultimately making it possible to routinely analyze wiring diagrams.

#### ETHICS STATEMENT

All zebrafish larvae experimental procedures for the hindbrain dataset were approved by Weill Cornell Medical College's Institutional Animal Care and Use Committee. All animal procedures for the olfactory bulb dataset were approved by the Veterinary Department of the Canton Basel-Stadt (Switzerland).

## AUTHOR CONTRIBUTIONS

AV acquired and analyzed the larval zebrafish hind brain dataset and wrote the manuscript. AW acquired and analyzed the larval zebrafish olfactory bulb dataset and wrote the manuscript.

#### ACKNOWLEDGMENTS

We would like to thank our funding sources for supporting this work and R. W. Friedrich and H. S. Seung for many fruitful discussions and comments on this manuscript. AW thanks Christel Genoud for support in developing suitable EM protocols. The work of AW was funded by HFSP (RGP0015/2010), SNF (310030B\_152833/1, CRSII3\_130470/1) and the Novartis Research Foundation. AV would like to thank Emre Aksay, Alexandro Ramirez for optimizing tissue preparation, Jingpeng Wu, Dodam Ih, Kisuk Lee, Nicholas Turner, and Nico Kemnitz for support with alignment and automated segmentation pipeline. AV also thanks players from the EyeWire community and support form EyeWire for help with the reconstructions. The work of AV was funded by NSF CRCNS grant 1208088, NIH grant R01027036, NIH/NINDS award 5R01NS076467 and ARO MURI award W911NF-12-1- 0594.

Ahrens, M. B., Orger, M. B., Robson, D. N., Li, J. M., and Keller, P. J. (2013). Wholebrain functional imaging at cellular resolution using light-sheet microscopy. Nat. Methods 10, 413–420. doi: 10.1038/nmeth.2434

Aksay, E., Baker, R., Seung, H. S., and Tank, D. W. (2000). Anatomy and discharge properties of pre-motor neurons in the goldfish medulla that

Ahrens, M. B., Li, J. M., Orger, M. B., Robson, D. N., Schier, A. F., Engert, F., et al. (2012). Brain-wide neuronal dynamics during motor adaptation in zebrafish. Nature 485, 471–477. doi: 10.1038/nature11057

have eye-position signals during fixations. J. Neurophysiol. 84, 1035–1049. doi: 10.1152/jn.2000.84.2.1035


zebrafish exploratory locomotion. eLife 5:471. doi: 10.7554/eLife. 12741


spontaneous activity is independent of visual inputs. Cell Rep. 19, 939–948. doi: 10.1016/j.celrep.2017.04.015


**Conflict of Interest Statement:** Part of the results disclosed herein have been included in European patent application EP14736451 and US patent application US14897514. AW is owner and CEO of Ariadne-Service Gmbh.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Wanner and Vishwanathan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Fully-Automatic Synapse Prediction and Validation on a Large Data Set

Gary B. Huang\*, Louis K. Scheffer and Stephen M. Plaza

Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, United States

Extracting a connectome from an electron microscopy (EM) data set requires identification of neurons and determination of connections (synapses) between neurons. As manual extraction of this information is very time-consuming, there has been extensive research efforts to automatically segment the neurons to help guide and eventually replace manual tracing. Until recently, there has been comparatively little research on automatic detection of the actual synapses between neurons. This discrepancy can, in part, be attributed to several factors: obtaining neuronal shapes is a prerequisite for the first step in extracting a connectome, manual tracing is much more time-consuming than annotating synapses, and neuronal contact area can be used as a proxy for synapses in determining connections. However, recent research has demonstrated that contact area alone is not a sufficient predictor of a synaptic connection. Moreover, as segmentation improved, we observed that synapse annotation consumes a more significant fraction of overall reconstruction time (upwards of 50% of total effort). This ratio will only get worse as segmentation improves, gating the overall possible speed-up. Therefore, we address this problem by developing algorithms that automatically detect presynaptic neurons and their postsynaptic partners. In particular, presynaptic structures are detected using a U-Net convolutional neural network (CNN), and postsynaptic partners are detected using a multilayer perceptron (MLP) with features conditioned on the local segmentation. This work is novel because it requires minimal amount of training, leverages advances in image segmentation directly, and provides a complete solution for polyadic synapse detection. We further introduce novel metrics to evaluate our algorithm on connectomes of meaningful size. When applied to the output of our method on EM data from Drosphila, these metrics demonstrate that a completely automatic prediction can be used to effectively characterize most of the connectivity correctly.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Sacha Jennifer van Albada, Forschungszentrum Jülich, Helmholtz-Gemeinschaft Deutscher Forschungszentren (HZ), Germany Angel Merchán-Pérez, Universidad Politécnica de Madrid (UPM), Spain

#### \*Correspondence:

Gary B. Huang huangg@janelia.hhmi.org

Received: 20 July 2018 Accepted: 24 September 2018 Published: 29 October 2018

#### Citation:

Huang GB, Scheffer LK and Plaza SM (2018) Fully-Automatic Synapse Prediction and Validation on a Large Data Set. Front. Neural Circuits 12:87. doi: 10.3389/fncir.2018.00087 Keywords: connectomics, synapse prediction, deep learning, quantitative evaluation, Drosophila

# 1. INTRODUCTION

High-resolution EM imaging allows one to identify synapses, such as those shown in **Figure 1** below. In these examples, there is an electron dense region corresponding to the synapse at the pre-synaptic body. This consists of different transport apparatuses, such as vesicles, that abut the neuronal membrane. In a data set that contains numerous organelles of varying electron densities (i.e., imaging intensity) and neuronal membrane that intricately weaves throughout, identifying synapses can be challenging. When creating a connectome, an annotator will typically scan the data set or a traced neuron and manually identify and mark these sites. Even for organisms as small as a fruit fly, there are up to 100 million connections, making the process of manual annotation intractable.

Consequently, there have been recent research efforts to automate synapse detection using machine learning, which we discuss below in section 2. However, existing techniques for automated synapse detection have primarily been applied to detection in mammalian tissues. It is unclear, then, how well such approaches would translate when they are applied to synapse detection in Drosophila tissues. In contrast to synapses in the mammalian brain, which are predominantly monadic, involving a single presynaptic site and postsynaptic site, synapses in Drosophila are mostly polyadic, involving multiple postsynaptic partners for a given presynaptic site (Cardona et al., 2010), as can be seen in the examples shown in **Figure 2**. Even when the pre-synaptic site is given, these neuronal processes are often difficult to segment, which makes identifying the post-synaptic partners nontrivial.

With the exception of Staffler et al. (2017), prior work on automated synapse identification, as discussed below in section 2, has also focused only on the detection problem in isolation, with performance evaluated at the individual synapse level. However, synapse detection is one step in a larger pipeline, whose final goal is the extraction of a connectome from the electron microscopy (EM) data. Therefore, we are interested not only in individual synapse detection accuracy but also in how synapse detection integrates into this larger system and how errors in individual steps in this system combine when evaluating the final produced connectome.

For instance, one straightforward method for reliably using automated synapse detections in an EM pipeline is as hints for manual annotation, as done by Plaza et al. (2014). By manually verifying detections, errors in the final connectome are minimized but at the expense of human effort and time. An alternative would be to simply accept all detected synapses above a certain confidence threshold, but there has been limited prior work on whether such a prediction would result in a meaningful connectome (Dorkenwald et al., 2017). In particular, many connections between neurons are formed from a large number of synaptic contacts, and, therefore, one might hope that automated algorithms are capable of faithfully reconstructing such high strength connections, but there has been limited experimental testing in this direction.

Moreover, extracting a connectome is also dependent upon automatic neuron segmentation. In addition to possibly being outright incorrect, a segmentation may also be noisy along a border. Both cases may potentially cause errors in the connectivity graph when combined with the automated synapse identification output.

To our knowledge, these questions of evaluating synapse detection in a larger context have only been investigated in the recent work by Staffler et al. (2017). They find that many synapse detection errors occur near errors in automated segmentation and that manually fixing these segmentation errors is sufficient to correct nearly half of the synapse detections. They also give a theoretical analysis of individual synapse accuracy vs. binary neuron-to-neuron connection accuracy, assuming a distribution of synapses per neuron pair estimated from paired recordings in rodent cerebral cortex; additionally, they compute synapse accuracy and neuron-to-neuron level accuracy on a sparse local cortical connectome.

Therefore, in this paper, we introduce algorithms that enable fully automatic synapse prediction and evaluate the results of the end to end process from the standpoint of the final produced connectome. Specifically, key contributions and results of our approach include the following:


# 2. BACKGROUND

An automated approach for synapse identification in EM images using machine learning was first proposed by Kreshuk et al. (2011), who used a random forest (RF) classifier on hand-selected image features to detect synapses. In a subsequent study, Kreshuk et al. (2014) extend this method by applying graph cut on the synapse probabilities to obtain a segmentation of each putative synapse, extracting object-level features when the segmentation is given, and then applying a RF classifier to determine whether each segmented region is a synapse or not.

Becker et al. (2013) attempt to generate more informative features, by conditioning on the synaptic cleft, thereby, allowing features to be extracted from consistent spatial locations relative to the putative synapse. These features are then used as input for AdaBoost for synapse detection. Staffler et al. (2017) extend this by conditioning on presynaptic and postsynaptic regions separately, and use extracted features from these regions as input for LogitBoost for synapse detection, yielding improved results.

Jagadeesh et al. (2013) consider the problem of large-scale synapse detection in a large image volume. They first use a fast interest point detector based on image-thresholding to generate proposals for possible synapse locations. They then use feature descriptors hand-designed to extract information about relevant biological structures, namely vesicles, clefts, and ribbons. These features are used as input for a support vector machine (SVM) or a multiple kernel learner for patch-based synapse detection.

Biological preparation has also been considered as a means to aid automated synapse detection. Navlakha et al. (2013) apply a technique for the selective staining of synapses, leading to more pronounced opacity at synaptic sites, and leaving non-synaptic membranes unstained. They propose a high-throughput method for automated detection by first filtering down to a candidate set of patches and then applying an SVM to classify each patch as synapse or non-synapse. While this technique can be used to compute statistics on synapses such as density, since the membranes are left unstained, it cannot be used in conjunction

FIGURE 1 | Five examples of synapses in the Drosophila optic lobe (columns). Rows show three orthogonal views (xy, yz, xz slices) of each synapse. The presynaptic structure, referred to as a T-bar because of its shape, is centered in each image. Red dots mark segments containing postsynaptic densities (PSDs) that partner with the T-bar. Each image slice captures 1 µm<sup>2</sup> of data.

with segmentation and, therefore, cannot be directly used for extracting a connectome.

More recently, Roncal et al. (2015) also consider largescale synapse detection, presenting two different techniques. They propose a fast RF classifier using hand-selected features, including a filter designed for vesicle detection. This RF classifier yields results that are similar to the results of Becker et al. (2013) but with approximately half the run-time. They also propose a deep learning classifier for synapse detection, which yields results that are superior to the fast RF classifier but is approximately two orders of magnitude slower. Dorkenwald et al. (2017) also give a deep learning multiclass CNN for detecting synapses along with vesicle clouds and mitochondria and report improved results over Roncal et al. (2015).

The above approaches were evaluated on synapse detection in mammalian tissues, assuming a single postsynaptic site for each presynaptic site. Several approaches also make additional assumptions on the data, such as being able to reliably identify the synaptic cleft to extract spatially consistent features (Becker et al., 2013) or having feature descriptors hand-tuned for particular biological structures (Jagadeesh et al., 2013).

While manual annotation of synapses has been performed for sparse EM reconstructions (Zheng et al., 2018) and software tools have been created to facilitate manual synapse annotation (Boergens et al., 2017), a scalable alternative to facilitate reconstruction of larger connectomes is to make use of automated methods within a semi-manual workflow. For example, the method of Kreshuk et al. (2011) was adapted for presynaptic site detection by Plaza et al. (2014), where human proofreaders subsequently verified or rejected each automated detection, but the labeling of postsynaptic partners was performed manually with no automated guidance. Takemura et al. (2017) also took a similar approach, using the method of Huang and Plaza (2014) to generate presynaptic site proposals, with postsynaptic partner identification again being performed manually.

As mentioned in the introduction, synapse detection in Drosophila can be more challenging, owing to the polyadic nature of such synapses, where presynaptic sites have multiple postsynaptic partners and where postsynaptic processes can often be small and difficult to segment. To address this difficulty in Drosophila synapse detection, Kreshuk et al. (2015) specifically studied the problem of synaptic partner assignment. Conditioned on ground-truth neuron segmentation and synapse detection, they formulate a pairwise graphical model wherein nodes of the model, Pij, represent possible assignments between two neurons i, j at a putative synapse, for example, neuron i is presynaptic and neuron j is postsynaptic. Edges in the model, connecting Pij and Pik, encode biological priors on triplets of neurons i, j, k at a putative synapse, such as a preference for a one-to-many connection pattern over one-to-one. In a recent study, Heinrich et al. (2018) propose a deep learning CNN model for synaptic cleft segmentation in Drosophila, achieving state of the art results (evaluated on the CREMI challenge data set). With its specific focus on voxel-wise identification of the synaptic cleft, this work can be distinguished from ours in that we attempt to directly predict synaptic connectivity, which would require a nontrivial subsequent step conditioned on the cleft prediction output (for instance, by applying the method of Kreshuk et al. (2015).

In this study, we propose a complete system for automated synapse detection, capable of handling polyadic synapses as found in Drosophila. Our system uses a weakly-supervised deep learning approach and takes the simple point-wise annotations of presynaptic and postsynaptic sites as training data and, therefore, can be applied to new data sets with relatively minimal supervision. By comparison, existing methods as discussed above, which require hand-designed features to extract high-level information such as vesicles and ribbons, may not be appropriate for new data sets or may require significant manual effort to tune or redesign the feature descriptors. In contrast to Kreshuk et al. (2015), we evaluate our system on completely automated, noisy segmentation. Although our overall system was designed for synapse detection in Drosophila, in section 6, we discuss how the elements of our approach could be adapted for other domains such as mammalian tissue.

#### 3. AUTOMATED SYNAPSE DETECTION

Our system for automated synapse detection proceeds in two distinct steps. First, independent of any segmentation, we apply a classifier to automatically identify presynaptic sites in Drosophila, which are often referred to as T-bars, because of their Tlike shape, formed by a pedestal and platform structure. Next, conditioned on predicted T-bar locations and a segmentation, we apply a second classifier to predict partnering postsynaptic densities (PSDs) for the identified T-bars.

Owing to its distinct structure, we focus on first predicting T-bars in isolation, independent of both segmentation and PSD prediction, and delay the problem of determining the potentially multiple PSD partners until after segmentation, as PSDs are typically more ambiguous and difficult to identify. We note that this approach of splitting T-bar and PSD prediction into separate steps, with PSD prediction aided by segmentation, has also been employed for manual synapse detection (Plaza et al., 2014).

We describe each step in our pipeline in more detail in the next two sections. We have also released source code that implements the proposed methods<sup>1</sup> .

#### 3.1. Presynaptic T-bar Identification Algorithm

The first step in our automated synapse detection pipeline is to detect the presynaptic T-bar sites. Examples of T-bars can be seen in **Figure 1**.

For automated T-bar detection, we follow the approach described in Huang and Plaza (2014), except that we update

the voxel-wise classifier to be a 3D U-Net CNN (Ronneberger et al., 2015). We give an overview of our approach here; for more details, see Huang and Plaza's paper (2014).

Unlike the problem of image segmentation, which is naturally framed as a voxel-wise prediction problem (at each voxel, predict whether that voxel belongs to a cell boundary or not), T-bar detection is an object detection problem, which we formulate as predicting, for each T-bar, a point annotation, specifying the spatial coordinates of the center of the T-bar. To generate voxelwise training data for the U-Net, we simply consider any voxels within a certain radius of a T-bar point annotation to be a positive example and all other voxels to be negative examples. We find that the U-Net is able to successfully learn from this simple training data, allowing for less manual supervision effort relative to methods and tasks that require dense labeling. Our specific U-Net model consists of layers of convolution with 3<sup>3</sup> voxel filters and two downsampling and two upsampling layers, with a total receptive field size of 19<sup>3</sup> voxels.

To generate final T-bar point predictions from the voxelwise output of the U-Net, we spatially smoothen the voxel-wise predictions, selecting the voxels with highest confidence, and apply non-maxima suppression.

We make two notes concerning the evaluation of T-bar prediction, in the context of a larger connectomics pipeline. First, it is important to consider the precision/recall (PR) curve

<sup>1</sup>https://github.com/janelia-flyem/flypylib

for the automated predictions. Different applications may have different misclassification costs, leading to different thresholds along different points of the PR curve. For instance, if very high fidelity is required, one may need to select a threshold for high recall, at the expensive of precision, whereas if the final goal is to determine strong connections in the connectome with some tolerance for small errors, the optimal threshold may be to select for the PR break-even point.

Second, T-bar prediction accuracy can be computed by necessitating for instance, that predicted T-bars be within a specified distance of a ground-truth T-bar to be counted as a correct match, as described by Huang and Plaza (2014). However, ultimately, the exact location of a T-bar annotation will be abstracted as one end point of an edge in a connectomic graph, indicating the presynaptic body. Therefore, the primary concern is that annotation be placed in the correct neuron. Thus, when a segmentation is available, T-bar prediction accuracy should be computed by further necessitating that the predicted T-bar falls within the same segment as the ground-truth T-bar.

Owing to this interaction with the segmentation when evaluating T-bar performance accuracy, it may be beneficial to post-process the T-bar predictions. For instance, we find that our T-bar detection often places the annotation in the distinctive dark T-like structure itself, which, owing to its dark intensity, can cause problems for automated segmentation. We, therefore, find a benefit in slightly shifting T-bar predictions within a small radius to the brightest intensity voxel, helping the annotation to be placed in a nonambiguous region relative to the segmentation.

## 3.2. Segmentation-Aware Postsynaptic Partner Identification

Once we automate T-bar predictions and a (possibly automated) segmentation, we condition on this information in order to predict the PSDs that partner with each T-bar. For a given Tbar, we can consider all nearby segments as potentially possessing a partner PSD. More precisely, we use the set of segments that have a non-empty intersection with a sphere of a given radius, centered at a given T-bar, as the candidate set of bodies that may be postsynaptic to the T-bar. We exclude the segment containing the T-bar itself and, therefore, make no attempt at predicting autapses. Additionally, we do not attempt to identify cases where a single T-bar makes multiple connections to the same postsynaptic body, and, thus, any such biological multipleconnections will at most be predicted as a single synapse.

With this setup, we have a binary classification problem, where for each T-bar and each candidate segment, we wish to determine if the candidate segment contains an actual PSD and, thus, forms a synapse with the T-bar. For classification, we use a multi-layer perceptron (MLP) with a single hidden layer consisting of 50 hidden units, trained using cross-entropy loss. To generate the feature representation, we estimate the interface of the synapse between the T-bar segment and candidate segment, by dilating both segments by varying amounts and letting the estimated interface to be the intersection. We then pool a set of simple image features over the interface, computing statistics such as size of the interface and image intensity within the interface (such as number of voxels with intensity lower than some given threshold), giving a total input feature dimensionality of 135.

One important consideration is that PSD prediction performance will depend on both the accuracy of the PSD predictor itself as well as the performance of the algorithm used to generate the segmentation. Therefore, it may be necessary to tune the PSD predictor with an awareness of the behavior of the segmentation algorithm. For instance, we found that dark intensity values such as those found at a boundary, as well as at T-bars, would often present difficulties for the segmentation algorithm. This ambiguity could lead to, for instance, small parts of the T-bar being incorrectly assigned to a neighboring segment. Although such localized errors would not have a large effect on the topology of the segmentation (in terms of Rand error, for example), they could have a large effect on the proposed feature representation and, hence, the PSD classifier. Therefore, we attempt to make the classifier more robust to such errors by ignoring the segmentation at voxels with such dark intensity values.

# 4. METRICS FOR EVALUATION

As discussed above, to properly evaluate automated synapse detection performance in the context of a larger pipeline, it is important to consider the full performance curve as the threshold of classifier confidence is varied. This allows for synapse prediction to be evaluated at the appropriate threshold for varying misclassification costs, which will depend on the final application that is being considered. One straightforward metric for evaluating detection at the individual synapse level is to produce a (PR) curve. Under the view of the connectome as a graph, with directed edges between nodes (representing neurons) defined by synapses, we can consider two variations for computing PR. First, we can view the connectome as a weighted graph and compute PR by considering each individual synapse as a ground-truth label that is to be predicted. Second, we can consider the connectome as an unweighted graph and compute PR by considering each edge (formed by any number of synapses between a pair of neurons) as a ground-truth label that is to be predicted.

The above methods for computing PR are two ways of dealing with the finding from connectomic studies that many connections between neurons consist of multiple synapses (Takemura et al., 2013, 2015). This multiplicity may be a weight on the synapse strength or may be a mechanism for robustness. In either case, a general assumption in many connectomic efforts is that important biological connections will have some multiplicity greater than one. Therefore, we would like to consider a range of metrics that will better reflect whether a set of automated synapse predictions is actually good enough for use in connectomic studies.

Computing PR with a weighted graph requires that the automated predictions match the ground-truth precisely in terms of strength, without any regard to topology. For example, predicting an edge of strength 7 for a ground-truth edge of strength 9 is equivalent to missing an edge of strength 2 (in terms of impact on total recall value), which may be inappropriate if we fail to care about precisely determining the multiplicity of strong connections. Computing PR with an unweighted graph, on the other hand, evaluates the automated predictions solely in terms of unweighted topology. Therefore, no penalty is incurred for not correctly determining multiplicity, but missing an edge of multiplicity 1 is equivalent to missing an edge of multiplicity 7.

One simple modification that can be made to the unweighted graph computation is to consider the unweighted graph produced by thresholding the edge weights by some value t (in both the predicted and ground-truth connectomes). For t = 1, we have the original unweighted PR; for t > 1, we focus only on stronger predicted and ground-truth edges, with multiplicity of at least t. We can also examine performance of a given classifier over different sets of curves as we vary this threshold t.

## 4.1. Asymmetric PR, Connections Added/Missed

By thresholding the edge weights at some t > 1 and computing unweighted PR, we focus on the strong edge connections and ignore potentially noisy weak connections. However, there is still a strong boundary effect, where, for instance, a predicted edge of strength t − 1 for a corresponding ground-truth edge of strength t is counted as a false negative, the same as if the predicted edge strength had been simply zero. This harsh decision boundary may also be problematic from the standpoint of potential small errors in the manually annotated ground-truth. We would like a metric that focuses on identifying clear error cases in the automated predictions.

We, therefore, introduce an asymmetric variant of the above thresholded PR curve. Let the asymmetric t1, t<sup>2</sup> thresholded PR curve (with t<sup>1</sup> > t2) be defined as follows: consider the (weighted) ground-truth connectome graph g and the predicted graph p produced by applying some classifier threshold, and let g(e) be the weight of a given edge e in g and similarly for p(e). Recall is then computed as

$$\frac{\sum\_{e} [p(e) \ge t\_2 \land g(e) \ge t\_1]}{\sum\_{e} [g(e) \ge t\_1]},$$

where the square Iverson brackets equate to 1 if the condition inside is true and 0 otherwise. In other words, the total set of positive ground-truth instances consists of all edges with groundtruth weight greater than t1, but the subset of true positives allows for edges with predicted weight greater than the smaller t2. Conversely, precision is computed as

$$\frac{\sum\_{e} [p(e) \ge t\_1 \land g(e) \ge t\_2]}{\sum\_{e} [p(e) \ge t\_1]}$$

.

Here, the total set of positive predicted instances consists of all edges with predicted weight greater than t1, but the subset of true positives allows for edges with ground-truth weight greater than the smaller t2.

From the above PR definitions, it can be seen that the asymmetric t1, t<sup>2</sup> thresholded PR upper bounds the original symmetric thresholded PR at t = t1. This more lenient performance measure focuses on the more clear, egregious errors, where there is a strong edge in either the ground-truth or predicted connectome graph but a weak or no edge in the other graph. We can also report these types of errors directly as connections falsely added (false positives) and connections missed (false negatives). Let connections missed be the set of edges e such that g(e) ≥ t<sup>1</sup> ∧ p(e) < t2. The number of connections missed is an unnormalized version of 1 − recall. Let connections added be the set of edges e such that p(e) ≥ t<sup>1</sup> ∧ g(e) < t2. The number of connections added is an unnormalized version of 1 − precision. When plotting number of connections added vs. number of connections missed, we normalize these values by the number of edges in the ground-truth connectome after thresholding, that is, the number of edges e such that g(e) ≥ t1, to put curves with different values of t<sup>1</sup> on the same scale.

By using asymmetric thresholded PR and connections added/missed, we can focus on strong error cases when comparing sets of predictions and be robust to small amounts of labeling noise. These error measures also more clearly indicate to what extent strong biological connections are being missed or falsely introduced through prediction.

# 5. RESULTS

In this section, we present a case study of our proposed synapse detection system on data from the Drosophila optic lobe. The data set that we use comprises seven columns of the medulla, acquired using focused-ion beam milling scanning electron microscopy (FIB-SEM). The image data has a total volume of 40 × 40 × 80 µm, with an isotropic resolution of 10 nm per voxel. The manually annotated subset of the data that we use in this study consists of 27, 000 cubic microns and contains ∼56,500 T-bars and ∼336,500 PSDs. Our methods operate on the data at the original resolution. Additional details of the data can be found in the papers of Plaza et al. (2014) and Takemura et al. (2015), and the raw EM image data, FIB-25, is available online<sup>2</sup> .

We give results of the individual steps of our pipeline, full end to end results, results using the proposed error metrics focusing on clear error cases, comparison against a surface area contact baseline, and results in the context of preserving biological findings.

### 5.1. Performance of T-bar, PSD Detectors

We first train a T-bar detector using the system described above in section 3.1, using the ground-truth annotations contained in two 520<sup>3</sup> voxel subcubes of the total volume, containing a total of 325 T-bars. **Figure 2** gives the precision/recall curve for the automated predictions over the entire data volume. The plot highlights two important points that were made in section 3.1: First, T-bar prediction accuracy should ideally be assessed within the context of segmentation and the final produced connectome graph, rather than only considering the distance between predicted and ground-truth T-bar locations. A predicted T-bar that is very close to a ground-truth T-bar, but placed in the wrong ground-truth segment, will lead to errors in the

<sup>2</sup>https://www.janelia.org/project-team/flyem/data-and-software-release

connectome graph. This is highlighted in the difference between the dashed red curve and the dotted blue curve. Consequently, second, the T-bar detector may need to be aware of the behavior of the corresponding segmentation algorithm. In our case, we found that simply shifting the predicted T-bar locations slightly, toward brighter image voxels, would move the predictions away from dark image regions that are more difficult or ambiguous for the segmentation algorithm and, therefore, improve performance when applying the segmentation constraint.

Next, we evaluate the performance of the PSD predictor. We consider performance both under the scenario in which we have access to the ground-truth segmentation and in which we only have access to a predicted, fully-automated segmentation. We first make a note about the "ground-truth segmentation." This segmentation was produced by starting from an automated segmentation (separate and distinct from the fullyautomated segmentation we use for synapse prediction) and manually proofreading the segmentation by applying merge and split operations as necessary. This ground-truth segmentation, therefore, aims to get the correct general topology, but it is not refined to the point of necessarily assigning a correct label at the voxel level, and additionally this segmentation may have orphan fragments that were not merged into larger bodies. One important consequence is that, when we compute performance using this ground-truth segmentation, we typically ignore all predictions that fall into such orphan fragments, defined as segments that contain neither a ground-truth Tbar nor a ground-truth PSD. In other words, predictions that fall into such fragments are not counted when computing precision. Additionally, we shift PSD point annotations using the same criteria as those used when shifting T-bar annotations as mentioned above.

We first evaluate PSD prediction assuming that we have access to ground-truth T-bar locations, in order to evaluate the performance of the PSD detector on its own. This performance is given in the left plot of **Figure 3**. Next, we evaluate PSD prediction using predicted T-bar locations (using a conservative threshold on the T-bar confidence scores, aimed at achieving a high recall of 0.9). We compute precision/recall considering each PSD separately, corresponding to a weighted view of the connectome graph. Importantly, we note that although performance is best when the ground-truth segmentation is available during PSD prediction, our PSD predictor is still able to achieve close performance using the automated, predicted segmentation.

# 5.2. End to End Synapse Performance and Comparison

We now move from evaluating each of the detectors in isolation to giving a full end to end evaluation of our synapse detection pipeline, with respect to the final generated connectome. As determining an acceptable prediction accuracy is difficult without considering the particular connectomics application domain, we present a range of performance curves using our proposed error metrics. Additionally, we compare against a baseline using neuronal-body proximity/contact as a proxy for synaptic contact. For this baseline, we use the ground-truth segmentation. We randomly sample points at boundaries between ground-truth segments and then randomly select the direction of the synapse (presynaptic and postsynaptic bodies). For this proximitybased comparison, we also compute precision/recall using an undirected view of the connectome graphs, thereby, allowing for matches even if the predicted direction of synapse was incorrect.

The left plot of **Figure 4** gives the PR of our proposed system, using the fully-automated predicted segmentation. We fix a conservative threshold for T-bars, accepting all T-bars above this threshold, and vary the threshold for the PSD detector to generate PR curves, under both a weighted and unweighted view of the connectome graph edges. The right plot shows a comparison against the baseline using body-proximity as a proxy for synaptic contact. Even after using the ground-truth segmentation and computing the undirected edge PR this baseline performs much worse.

Next, we evaluate synapse detection performance using our proposed variants to PR, as shown in **Figure 5**. We give curves when thresholding the edges at different values t, that is, a (unweighted) edge is preserved in the connectome graph if the original edge weight is greater than t. If t = 1, then the curve is equivalent to the above unweighted graph PR. We also give curves using our proposed asymmetric thresholded t1, t<sup>2</sup> PR. We again compare with the baseline of using body-proximity.

We give another view of synapse detection performance, using our metrics of connections strongly added and missed, in **Figure 6**. For the case of thresholding with t<sup>1</sup> = 10, t<sup>2</sup> = 5, we have a total of about 2000 edges in the ground-truth connectome with a weight of at least t<sup>1</sup> = 10. Using our proposed system, we can recover more than 99% of these edges (less than 1% connections missed) while introducing less than 1% falsely added connections. By comparison, from the right plot in **Figure 6**, we can see that by using body-proximity as a proxy for synaptic connection, when thresholding by t<sup>1</sup> = 10, t<sup>2</sup> = 5 and considering the directed graph, the normalized number of connections added and missed is approximately 50/50%. Therefore, even with this error metric that focuses on clear, unambiguous errors, this baseline approach is missing half the ground-truth connections and adding in approximately the same number of false connections.

Lastly, we present plots comparing automatic vs. manual synapse counts when restricting edges to a core set of bodies and connectomes, used in a study by Takemura et al. (2015). **Figure 7** gives scatter plots, where each point gives the automated and manual synapse count for an edge in the connectome. As mentioned above in section 3.2, our proposed system has limitations in that it does not attempt to predict autapses and predicts at most one connection from a T-bar to a given postsynaptic body. Therefore, we also give a comparison of automatic vs. manual counts, shown to the right in **Figure 7**, after removing autapses and collapsing multiple connections from a single T-bar to the same postsynaptic body.

We note that, for strong edges with a synapse count of 30 or above, our automated predictions fall within the indicated bounds of y = 2x and y = 1 2 x. We can also examine edges in the automated and ground-truth connectome for which the corresponding connectome has a synapse count of zero. We can, thus, see that, for all edges with a manual synapse count of at least four, we are able to recover the edge, in the sense that

FIGURE 3 | PSD precision/recall, where each PSD is considered separately (weighted view of connectome graph, see section 4). Performance is computed both when access to the ground-truth segmentation is available during PSD detection (ground-truth/gt segm), and when only the fully-automated, predicted segmentation is available during PSD detection (predicted/pd segm). (Left) Plot of PSD prediction performance in isolation, using ground-truth T-bar locations. (Right) Plot of end-to-end performance, using predicted T-bar locations.

FIGURE 4 | Global connectome graph precision/recall. (Left) The top blue curves show PR of the connectome graph, with the dashed curve computing PR using a weighted view of the graph edges and the solid curve computing PR using a unweighted binary view of the graph edges. These curves are computed using the filtered set of bodies in the ground-truth segmentation, as described in section 5.1. For reference, the bottom red curves show weighted and unweighted PR if all bodies (adding in orphan segments) are considered. (Right) Comparison against the baseline of using ground-truth body-proximity as a proxy for synaptic contact. All curves show unweighted PR on the filtered set of bodies.

FIGURE 6 | Thresholded global connectome graph connections added/missed. (Left) The curves show errors in terms of connections that were added and missed, using the same thresholds as the PR curves in Figure 5. (Right) Comparison against the baseline using ground-truth body-proximity, both directed and undirected edges.

the automated prediction gives a synapse count of at least one. Similarly, for all edges with an automated synapse count of at least five, the edge appears in the ground-truth connectome, as the manual synapse count is at least one.

#### 6. CONCLUSIONS

In this paper, we have proposed an end to end system for automatic synapse detection in EM image data, capable of handling the polyadic synapses found in Drosophila. We have additionally proposed a set of metrics to better assess the quality of a set of synapse predictions and whether such predictions are sufficiently accurate to be of use in connectomic studies. We evaluate our system on the Drosophila seven column medulla data set and show that it is capable of reconstructing high multiplicity synaptic connections, preserving biological pathways, while only making a small number of clear errors; we also show that our system greatly outperforms the baseline using body proximity as a proxy for synaptic connections.

By performing an evaluation on the entire end to end automatic predictions, we are able to assess both how each component contributes to the overall performance as well as how the components interact. For instance, by comparing performance of PSD prediction using ground-truth T-bars or ground-truth segmentation, we can estimate the expected gains from improving T-bar prediction or segmentation. At the same time, we are able to see that overall performance may be improved by taking into account noise in a previous component, such as the need to spatially shift the synapse predictions to be more robust to noise in the segmentation.

Although our proposed method is designed for synapse detection in Drosophila, we believe that the success of our presynaptic T-bar detector suggests that our weakly-supervised approach may be of value in other domains as well. For example, in synapse detection in mammalian tissues, rather than spending manual annotation time to obtain groundtruth labeling of synapses accurate at a voxel-level, as is common practice (Roncal et al., 2015), it may instead suffice to place two landmarks per synapse (to indicate location and directionality), allowing one to obtain more synapse annotations in an equal amount of time. Training on a larger, potentially more diverse set of synapses may lead to better accuracy and generalization across a large volume. A combination of the two approaches could also be used, wherein a small number of synapses is labeled at voxel-wise accuracy and a large number is labeled with landmarks, thereby, maintaining voxel-level performance with the added benefit of a larger, more diverse training set.

#### AUTHOR CONTRIBUTIONS

GH developed and wrote core methodology, LS provided core connectome data and analysis, SP provided automated segmentation.

#### FUNDING

This study was funded and supported by Howard Hughes Medical Institute.

### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Huang, Scheffer and Plaza. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Block Face Scanning Electron Microscopy of Fluorescently Labeled Axons Without Using Near Infra-Red Branding

Catherine Maclachlan<sup>1</sup> , Daniela A. Sahlender <sup>1</sup> , Shuichi Hayashi <sup>2</sup> , Zoltán Molnár <sup>2</sup> and Graham Knott <sup>1</sup> \*

<sup>1</sup>BioEM Facility, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland, <sup>2</sup>Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom

In this article, we describe the method that allows fluorescently tagged structures such as axons to be targeted for electron microscopy (EM) analysis without the need to convert their labels into electron dense stains, introduce any fiducial marks, or image large volumes at high resolution. We optimally preserve and stain the brain tissue for ultrastructural analysis and use natural landmarks, such as cell bodies and blood vessels, to locate neurites that had been imaged previously using confocal microscopy. The method relies on low and high magnification views taken with the light microscope, after fixation, to capture information of the tissue structure that can later be used to pinpoint the position of structures of interest in serial EM images. The examples shown here are td Tomato expressing cortico-thalamic axons in the posteromedial nucleus of the mouse thalamus, imaged in fixed tissue with confocal microscopy, and subsequently visualized with serial block-face EM (SBEM) and reconstructed into 3D models for analysis.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Nobuhiko Ohno, Jichi Medical University, Japan Daniel Raimund Berger, Harvard University, United States

> \*Correspondence: Graham Knott graham.knott@epfl.ch

Received: 22 June 2018 Accepted: 08 October 2018 Published: 06 November 2018

#### Citation:

Maclachlan C, Sahlender DA, Hayashi S, Molnár Z and Knott G (2018) Block Face Scanning Electron Microscopy of Fluorescently Labeled Axons Without Using Near Infra-Red Branding. Front. Neuroanat. 12:88. doi: 10.3389/fnana.2018.00088 Keywords: correlative light and electron microscopy (CLEM), axons, scanning electron microscopy, neuron ultrastructure, serial block-face electron microscopy (SBEM)

# INTRODUCTION

Scanning electron microscopy (SEM) has become the tool of choice for many investigations of cell and tissue 3D ultrastructure. Serial block-face SEM (SBEM; Denk and Horstmann, 2004), focused ion beam SEM (FIBSEM, Knott et al., 2008), array tomography (Micheva and Smith, 2007) and automated tape collecting ultramicrotome SEM (ATUM-SEM; Schalek et al., 2012) provide biologists with the means to image large volumes of biological material with enough resolution to see all organelles and membranes (reviewed by Briggman and Bock, 2012). The methods have proved particularly useful in neuroscience as significant portions of neural circuits can now be visualized and mapped. However, identifying different types of cells, or parts of cells, with EM is not straightforward. Many different immunocytochemical or tracing approaches can be combined with EM, but these often require the cellular material to be less stringently fixed or permeabilized in such a way as to allow marker molecules, such as antibodies, free access, or quite often both (Knott et al., 2009). This reduces the quality of the ultrastructure and leads to difficulties in interpreting the images. The vast array of molecular biology manipulations that can label molecules, cells and tissues with fluorescent markers offers a myriad of possibilities for light microscopy. Therefore, combining the two by imaging initially with light microscopy, and then subsequently finding the same structure in the serial EM images offers significant advantages. These include the opportunity to combine in vivo imaging data from the light microscopy (LM), with the ultrastructural analysis of the same structures that have been optimally preserved. This correlated approach is used frequently in single-cell experiments (Murphy et al., 2011), especially cells cultured as a single monolayer, but in tissue volumes this is more difficult, and various strategies have been used to locate the fluorescently labeled structures in the EM image stacks without using immunocytochemistry. In very small structures, such as the Drosophila neuromuscular junction, serial sectioning of the entire muscle is possible and allows single axonal boutons, previously imaged in vivo, to be found with EM (Zito et al., 1999). For structures located in much larger pieces of tissue such as the mouse brain, or retina, the vasculature is used to identify the region and this is then completely sectioned and imaged prior to identifying individual neurons of interest afterward (Bock et al., 2011; Briggman et al., 2011). Targeting small structures and reducing the volumes that need to be cut and imaged is possible by laser branding the fixed tissue to create fiducial marks (Bishop et al., 2011). This is a proven technique for localizing neurites that had previously been imaged in vivo (Maco et al., 2014). Small (approximately 10–50 micrometer) squares around the structures of interest can be seen in the resin embedded tissue, and in the EM images, giving the opportunity to indicate the position of the structures of interest. This has been used in a number of correlative studies with in vivo 2 photon microscopy (Grillo et al., 2013; Mostany et al., 2013; Cane et al., 2014). However, while this is an effective approach, a 2-photon laser system may not always be at hand, particularly when not in vivo imaging.

For these reasons, we developed a method, using SBEM, that does not require introducing any fiducial marks, or the need to section and image massive volumes of tissue to reliably find axons and dendrites previously imaged with light microscopy. The approach relies on the natural landmarks, such as blood vessels and cell bodies. It only requires low-resolution imaging, of the entire section, using transmitted light, combined with high-resolution confocal imaging of the structures of interest. Once the tissue section is heavy-metal stained and resin embedded, careful block preparation using the blood vessel pattern and trimmed edges, allows regions of interest to be accurately positioned ready for EM. SBEM imaging can then collect both high and low-resolution images that reveal the exact location of the relevant structures.

The reliability with which SBEM can collect serial images of structures that were previously imaged with light microscopy removes the need to convert fluorescent markers to electron dense stains. This gives opportunities to carry out combined light and EM analyses using a wide range of different types of fluorescence imaging. To demonstrate this method, we show how fluorescent cortico-thalamic axons, and their boutons that synapse with neurons in the posteriormedial thalamic nucleus, can be imaged with laser scanning confocal microscopy and then 3D reconstructed from serial electron micrographs using SBEM. The structure of axons communicating between the thalamus and cortex have been the focus of many ultrastructural studies. These have used a variety of labeling strategies to locate them including tracers such as lectins (Hoogland et al., 1991) or biotinylated dextrans (Li et al., 2003), lesions (Mathers, 1972), autoradiography (Ogren and Hendrickson, 1979) and immunocytochemistry against endogenous markers (Godwin et al., 1996; Groh et al., 2014) or fluorescent tags expressed in axons (Hoerder-Suabedissen et al., 2018a).

### METHODS

#### Tissue Preparation

The animal experiments were performed in the animal facilities of the University of Oxford (UK) under a valid Animals (Scientific Procedures) Act project license as well as with local ethical approval by the central Committee on Animal Care and Ethical Review (ACER) and the Animal Welfare and Ethical Review Body (AWERB) at the University of Oxford. Adult mice containing a Cre-recombinase expressing strain (Tg(Rbp4 cre)KL100Gsat/Mmucd (Rbp4-Cre; Jackson Laboratories) were crossed with B6;129S6-Gt(ROSA)26Sortm14(CAGtdTomato)Hze/J (Ai14) to label cortical layer 5 neurons. The axons of these Rbp4-Cre;Ai14 mice were visible in the posterior medial thalamic nucleus (POm; Grant et al., 2016; Hoerder-Suabedissen et al., 2018b). Mice at P18 were perfused with a buffered solution of 2.5% glutaraldehyde (Electron Microscopy Sciences, 16220), and 2% paraformaldehyde (Electron Microscopy Sciences, 15714) at pH of 7.4. The brain was then removed and embedded in agarose, and 80-micrometer thick sections cut with a vibratome in the coronal plane, at the level of the thalamus. Only sections containing the posterior medial nucleus were collected.

## Collecting of Fluorescence and Light Microscopy Images

Prior to confocal imaging, the sections were viewed under a dissecting microscope and using a scalpel the region of the thalamus was trimmed away from the rest of the brain (**Figure 1A**). This created a section of approximately 4 × 4 mm. These pieces were then imaged with both bright field and epi-fluorescent illumination to capture the entire section (**Figure 1A**) and then at increasingly high magnifications so that blood vessels could be seen in each image (**Figures 1B,C**), and also their proximity to the fluorescent axon terminals of interest recorded (**Figures 1D,E**). At this point, confocal microscopy was used to capture images of the fluorescent axons (LSM710; Zeiss). Image stacks were collected with 0.5 µm distance between images at using a ×63 objective. These are crucial for locating the fluorescent axons in the final EM image series. In the example shown in **Figure 1**, the blood vessel is seen at a depth of 30 microns (**Figure 1D**), however, the axonal bouton of interest lies only 8 micrometers below the surface (**Figures 1E,F**).

#### Preparation of Tissue for Electron Microscopy

The trimmed and imaged sections were then transferred to cacodylate buffer (0.1 M, pH 7.4), and heavy metal stained

with a protocol largely similar to that described by Hua et al. (2015). In brief, the sections were postfixed in 1.5% potassium ferrocyanide (Sigma 14459-95-1) and 2% osmium tetroxide mixed together (Electron Microscopy Sciences, 19110), then stained with 1% thiocarbohydrazide (Sigma, 101001342) followed by 2% osmium tetroxide and then further stained overnight in 1% uranyl acetate (Electron Microscopy Sciences, 22400). They were then washed in distilled water at 50◦C before being transferred to a lead aspartate solution at pH 5, at the same temperature. After 20 min the sections were rewashed in distilled water at room temperature and then dehydrated in increasing concentrations of ethanol followed by increasing amounts of Durcupan resin (Electron Microscopy Sciences 14040 Parts A, B and D replace C with DMP30 from Electron Microscopy Science 13600) until at 100%. After infiltrating overnight, the sections were placed between glass microscope slides coated in a mold separating agent (Glorex Inspirations, Switzerland; 6 2407 445) and the resin hardened at 65◦C for 24 h.

#### Preparation of Block Ready for SBEM Imaging

Once the resin had completely polymerized, the glass slides were removed, leaving a thin resin lamella containing the section (**Figure 2B**). However, at this point, the section is well impregnated with heavy metals and completely opaque to transmitted light. It is therefore difficult to see any histological features other than bright spots of light from the blood vessel lumens traversing vertically through the section (**Figure 2B**). To locate the region of interest, images of the entire section are overlaid and aligned with the image of the same section prior to embedding (**Figures 2A,B**). There is no shrinkage at this stage as little deformation of the second image (**Figure 2B**) is required to align onto the first (**Figure 2A**). On this first image of the unstained section, it is possible to pinpoint the region of interest, from the position of the blood vessel, and therefore identify the same region in the stained resin embedded section (highlighted with a yellow box in the **Figures 2A,B**). These two overlaid images are then used to indicate from where excess material can be trimmed. It is important to note that blood vessels lying horizontally in the section, and visible in the wet section, may not be visible in the resin embedded section. Therefore, the accurate alignment of the two images is important, using the edges, so that the region of interest can be identified. In this example, a large vessel is being used as a fiducial mark for the smaller fluorescent structures. Smaller vessels radiate away from such structures providing a unique pattern throughout the entire section. This allows any regions to be found back in the EM as the closeness of blood vessels means they are never too far from fluorescent structures. An indication of this is given in the analyses of vasculature in different regions of rodent brain (Schlageter et al., 1999). In

an image is taken and this is overlaid with the previous so that the yellow box can be positioned precisely, indicating the region of interest. The small yellow box shown in the 4th trim shows the final region that remains after trimming in the ultramicrotome. This region corresponds with the block face shown in Figure 3. (D) The schematic diagrams show the initial and final trimming of the block from the side that is mounted on the pin. The block is initially trimmed to leave only the region of interest of approximately 250 × 250 µm. This is then surrounded in the conductive glue again, and this is trimmed away again until some of the resin remains around the edge to help with conduction. Scale bar in (A,B) is 1 mm.

sections through the cerebral hemispheres the average distances between microvessels are between 17 and 26 micrometers. Although the smallest capillaries may not be clearly visible when focusing through wet, thick sections, in our experience a blood vessel is always seen within approximately 20 microns of a structure.

Using a razor blade, parts of the section are removed (**Figure 2C**). This is done by first cutting one side, and then recording an image of the remaining piece. This image is then overlaid with the previous one, showing exactly how much material was removed. This process is repeated for the adjacent side, and again the image overlaid with the previous. Repeating this for each of the four sides leaves a small block with the region of interest located in the center (**Figure 2C**), and importantly, its location in the original section known. Typically, the remaining piece of the section is approximately 1 mm × 1 mm. This can then be glued to the SEM pin (Micro to Nano place, 10-006003- 50) using electrically conductive epoxy resin (Ted Pella Inc., Redding, CA, USA 16043), and left to harden completely at 65◦C, overnight.

When the glue is completely hardened, the pin is mounted to the arm of an ultramicrotome and with glass knives a block face of approximately 250 µm × 250 µm trimmed. As before, an image is taken after each side is cut away and overlaid with the previous image to indicate the position of the remaining tissue. This allows the region of interest to be positioned close to the center of the block (**Figure 3A**). To avoid any confusion as to the orientation of the block, a trapezoid shape is cut. It is important to trim down through the entire thickness of the block, leaving at least 80-micron proud of the pin.

This block is then again surrounded with electrically conductive epoxy resin and left to harden at 65◦C overnight. The reason for this second application of glue is to maximize the amount of electrical contact between the tissue and the pin. The second glue layer is trimmed away, using a glass knife in the ultramicrotome, being careful not to remove more resin from the block, and not too deep as was previously trimmed (see schematic diagram in **Figure 2D**). This means that the resin embedded tissue is now surrounded by the conductive resin, which will help to disperse the charge when the sample is imaged. Any excess resin is then removed from the surface of the block until the surface of the tissue is exposed. At this point, the block is ready and is gold sputter coated with a 50 nm layer (Quorum Technologies; Q300T).

#### SBEM Imaging

The sample is loaded into the SEM microscope (Merlin, Zeiss NTS) fitted with the 3View cutting system (Gatan, Inc., Pleasanton, CA, USA). While the microscope chamber is open, the initial cutting of the face is performed to ensure that the

FIGURE 3 | The final trimmed block is scanned in the microscope revealing the position of cell bodies and blood vessels seen with fluorescence microscopy. (A) The block is carefully trimmed so that the region of interest lies just within its borders. An image of this final block can be overlaid precisely with an image of the entire region taken prior to trimming. Then, a box (yellow) is able to indicate the precise region where images can be taken. (B) The block is then placed in the microscope and once a large part of the face has been revealed with the diamond knife, an scanning EM (SEM) image is taken. (C) The region of interest is then confirmed by matching the position of cells and vessels (arrowheads) with those seen previously in the confocal image stacks. Their height in the image stacks is also noted. Scale bar in (B) is 50 µm and in (C; right-hand images) is 20 µm.

cutting window is in the correct position, the block is seated securely, and sections are cleanly removed. The door is closed and the microscope pumped down to high vacuum ready for imaging.

The first SEM image is then taken to see the entire block face. This image can be overlaid with all previous aligned images taken during the preparation (**Figures 3A,B**). In this way, the EM image can be positioned on the LM image of the section prior to any trimming revealing the location of the region of interest on the block face. After this alignment, the imaging window can be positioned on the block face. However, it is important to now look at the serial confocal images to understand which features will be visible in the EM. The most obvious ones are blood vessels and cell bodies which appear dark in the confocal stack (**Figure 3C**). Their depth in the block can also be estimated from this stack. This Z-depth value is important for indicating when the ROI will appear in the serial EM images. To reach the correct position more quickly, thicker sections can be cut (e.g., 200 nm) while imaging at lower magnification (1K× Magnification, 2 kV, 150 pa and 19 nm resolution) using the backscatter detector. As this approach is carried out it is important to compare each low magnification EM image with the different images in the confocal stack to make sure that the imaging is proceeding as expected and to better pinpoint where in the block the ROI will appear. When the block surface is estimated to be close to the structure of interest (2 µm in the case shown in **Figure 3**), the higher resolution images can be collected. For this the parameters are: 6000 × 6000—pixel images, 6.5 nm x and y resolution, 2 kV beam tension, 150 pA) and sectioning thickness of 50 nm. This gives a field of view of 39 µm (**Figure 4B**). The imaging then continues until the entire thickness containing the relevant structures have been imaged.

### Image Processing, Analysis and 3D Reconstruction

The final image series is aligned using the alignment functions in the TrakEM2 plugin of FIJI (Cardona et al., 2012) 1 . Segmentations are then made on suspected structures corresponding to those imaged with LM. These first drawings are done rapidly only to confirm the correct identities by checking their shape with those of the fluorescent structures in the confocal stack. As well as these features, blood vessels and cell bodies are also roughly segmented to orientate the different features in the two image stacks (**Figure 4**). When it is clear that the correct structures have been found, the same software is used to make the final reconstruction. The models are then exported in the OBJ file format that can be imported as meshes into the Blender software<sup>2</sup> . Within this software, any parts of the meshes can be manipulated and measured, using the NeuroMorph tools (Jorstad et al., 2015, 2018). This includes surface areas, volumes, and distances, as well as tools to see the original EM images combined with the final

<sup>1</sup>http://fiji.sc

<sup>2</sup>www.blender.org

models so that other features can be added, or their identities checked.

#### RESULTS AND DISCUSSION

This article demonstrates that fluorescent structures seen with confocal microscopy can subsequently be imaged with EM without using any additional specific markers, introducing any fiducial marks or collecting large EM stacks of entire samples to locate them. Our approach uses fixatives that ensure optimal preservation of the ultrastructure, i.e., glutaraldehyde. In the example shown here, the fixation attenuated the fluorescence of the tdTomato deeper into the tissue, but we were still able to collect reasonable images at a depth of 30 µm (**Figure 1D**). Imaging beyond 40 µm would have been very limited, but as the section was 80 µm thick it would have been possible to flip the section if features of interest were closer to the opposite surface.

We reconstructed axon terminal of layer 5 pyramidal neurons (Rbp4-Cre; Ai14) in PO in this study. The density of these axons in this region is reasonably high, but nevertheless, the shape of individual boutons could be easily distinguished from one another. The advantage of such a labeling is that cell body and blood vessels appear as unlabeled holes in the tissue, and easily seen (**Figure 4A**). Denser labeling would leave these structures even more evident but would make it difficult to isolate the individual axons. Sparser labeling would limit the ability to see the cell bodies and vessels, but in such a case a fluorescent stain such as DAPI could be used to highlight all the cell nuclei. This would not compromise the quality.

The results show how the imaging capacity of the SBEM is well suited to this correlative light and EM method. The ability to rapidly capture an image of the entire block face that approximately matches the field of view of the LM image (**Figure 3C**) makes it easy to correlate the different features of the tissue. As soon as the same structures, such as cell bodies, have been identified, the field of view in the SBEM can be narrowed, and the resolution increased, to the region with the structures of interest (**Figures 4A,B**). For this reason, the method is not as suitable for FIBSEM. Accurately milling such a large field of view, in just a few minutes, would be impossible with an ion beam. In addition, with FIBSEM the region of interest needs to be close (within 20 µm) to the edge of the milled surface so that milling aberrations, such as curtaining, do not occur. Therefore, a much smaller block would need to be prepared with the ultramicrotome, and the risk of removing important distinguishing features would be high. The advantages of the FIBSEM imaging are that lower amounts of heavy metals can be used to stain the tissue to produce images with high contrast. This means that the laser marks can be seen with transmitted light microscopy, even after staining and resin embedding. In the staining protocols that are used for SBEM imaging, the transmitted light is not able to traverse the section, so any laser marks are obscured. This means that the laser marks cannot be used to accurately guide the block trimming process.

The SBEM technique is also able to remove far larger volumes of resin from the block face than the FIB within only a few hours. This is a clear advantage if the region of interest lies deep inside the section. Accurately milling a window of the size shown in **Figure 4** would not be possible with FIBSEM.

Therefore, while the resolution of the FIBSEM is superior in the Z plane, resulting in image stacks with near isotropic voxels, it is limited in the size of images that are possible in the x and y plane. This means that collecting images of a similar size to those of the high-resolution light microscopy is impossible, making it hard to match the same feature seen in both modalities: LM and EM.

In summary, for many years neuroscientists have used EM to analyze the connectivity in the brain. With a range of labeling approaches, different types of cells and features can be specifically targeted for ultrastructural analysis. Cell-type specific Cre-recombinase expression enabled us to monitor selective populations of neurons using stop-floxed fluorophores. However, without using any labeling methods that may disrupt the tissue ultrastructure we instead use a correlative approach

#### REFERENCES


using landmarks within the tissue to locate precise regions in the EM images and find specific structures seen with light microscopy. By relying only on the light microscopy to provide details of the cell identities, rather than specific staining at the EM level, we show here how the ultrastructure can be maintained and structures such as axons easily targeted and serially imaged at the EM level to allow for 3D analysis, revealing uncompromised details of their morphology, and also that of their synaptic partners.

#### AUTHOR CONTRIBUTIONS

CM, DS and GK conceived the experimental method. CM and SH carried out the experiment. CM, DS, SH, ZM and GK wrote the manuscript. ZM and GK helped supervise the project.

#### FUNDING

ZM's laboratory is supported by Medical Research Council (G00900901). SH is supported from Daiichi Sankyo Foundation of Life Science and The Uehara Memorial Foundation.

innervates higher order thalamic nuclei in mice. Cereb. Cortex 28, 1882–1897. doi: 10.1093/cercor/bhy036


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Maclachlan, Sahlender, Hayashi, Molnár and Knott. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# NeuTu: Software for Collaborative, Large-Scale, Segmentation-Based Connectome Reconstruction

Ting Zhao\*, Donald J. Olbris, Yang Yu† and Stephen M. Plaza

Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, United States

Reconstructing a connectome from an EM dataset often requires a large effort of proofreading automatically generated segmentations. While many tools exist to enable tracing or proofreading, recent advances in EM imaging and segmentation quality suggest new strategies and pose unique challenges for tool design to accelerate proofreading. Namely, we now have access to very large multi-TB EM datasets where (1) many segments are largely correct, (2) segments can be very large (several GigaVoxels), and where (3) several proofreaders and scientists are expected to collaborate simultaneously. In this paper, we introduce NeuTu as a solution to efficiently proofread large, high-quality segmentation in a collaborative setting. NeuTu is a client program of our high-performance, scalable image database called DVID so that it can easily be scaled up. Besides common features of typical proofreading software, NeuTu tames unprecedentedly large data with its distinguishing functions, including: (1) low-latency 3D visualization of large mutable segmentations; (2) interactive splitting of very large false merges with highly optimized semi-automatic segmentation; (3) intuitive user operations for investigating or marking interesting points in 3D visualization; (4) visualizing proofreading history of a segmentation; and (5) real-time collaborative proofreading with lock-based concurrency control. These unique features have allowed us to manage the workflow of proofreading a large dataset smoothly without dividing them into subsets as in other segmentation-based tools. Most importantly, NeuTu has enabled some of the largest connectome reconstructions as well as interesting discoveries in the fly brain.

Keywords: NeuTu, connectome, electron microscopy, proofreading, segmentation

#### INTRODUCTION

Building the structural connectome of a brain is widely considered as an essential step of understanding the brain (Seung, 2012). Even if it is only a static snapshot of the brain without functional details, the information obtained from connectomes has been expected to provide unique and critical biological insights, as demonstrated in practice from the earliest efforts on Caenorhabditis elegans (White et al., 1986) to recent achievements on larger animals, such as Drosophila melanogaster (Takemura et al., 2013; Eichler et al., 2017), zebrafish (Wanner et al., 2016), and mice (Bock et al., 2011; Briggman et al., 2011; Lee et al., 2016; Morgan et al., 2016). Although the strategy of tracing neuronal skeletons has been widely used (Saalfeld et al., 2009;

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Daniel Raimund Berger, Harvard University, United States Hidetoshi Urakubo, Kyoto University, Japan

> \*Correspondence: Ting Zhao zhaot@janelia.hhmi.org

#### †Present address:

Yang Yu, Allen Institute for Brain Science, Seattle, WA, United States

Received: 21 July 2018 Accepted: 25 October 2018 Published: 13 November 2018

#### Citation:

Zhao T, Olbris DJ, Yu Y and Plaza SM (2018) NeuTu: Software for Collaborative, Large-Scale, Segmentation-Based Connectome Reconstruction. Front. Neural Circuits 12:101. doi: 10.3389/fncir.2018.00101

**94**

Boergens et al., 2017), segmentation-based reconstruction (Plaza et al., 2014; Kasthuri et al., 2015), or labeling every voxel in a volume, has its unique advantages. First, it is the most reliable way to get the complete reconstruction of a circuit, or at least it is easier to verify in dense reconstruction if some inconsistency is caused by reconstruction errors or biological randomness (Takemura et al., 2015). Second, it can leverage automated segmentation methods more easily to reduce manual work than skeleton-based sparse tracing can. Once segmentations are available, errors can be corrected by making local decisions without having to trace any long-range path of neuronal branches. For example, it is common that the border between two large segments to be merged is only a small fraction of the segments' overall surface area. Confirming merges by examining small contact regions can often produce a high-quality long segment with much less effort than manual skeleton tracing. Third, it provides more detailed information about neuronal morphology, which cannot only facilitate quality control, but also play an important role in simulation (Hines and Carnevale, 1997).

The workflow of segmentation-based connectome reconstruction typically involves EM image acquisition, image pre-processing, automated segmentation, and manual proofreading. While each of these steps is technically challenging, the last step, manual proofreading, usually consumes the most human labor, which can become extremely expensive and time-consuming as the dataset is scaled up to the whole brain. Even though there are significant efforts of improving automated segmentation to reduce the work load of manual proofreading (Beier et al., 2017; Januszewski et al., 2018), manual proofreading is still currently the primary bottleneck. Improvement on manual proofreading is usually generally applicable and expected to save tremendous resources regardless of what automated segmentation algorithm is applied in the pipeline.

Due to the necessity of manual proofreading and its complexity, it is not surprising that various software tools, including Raveler (Olbris et al., 2018), Knossos<sup>1</sup> , Dojo/Mojo (Haehn et al., 2014), Eyewire<sup>2</sup> and VAST (Berger et al., 2018), have been developed almost in parallel for correcting segmentation for dense or sparse reconstructions. While they have been successfully applied to produce local connectomes, recent advances in EM imaging (Briggman and Bock, 2012; Eberle et al., 2015; Hayworth et al., 2015; Xu et al., 2017) and segmentation (Beier et al., 2017; Januszewski et al., 2018) suggest new strategies and pose unique challenges for tool design to accelerate proofreading. For example, to the best of our knowledge, there is a lack of tools designed to operate on large segmented 3D objects freely without special constraints on the data, such as separating data into blocks (e.g., Raveler and Eyewire) or fixing errors slice by slice (e.g., Dojo and Mojo). Modifying, or mutating segmentation data in three dimensions is critical for providing a scalable solution for densely proofreading a large connectome.

Therefore, we have developed NeuTu to enable scalable proofreading on segmented datasets. Like many other proofreading tools, proofreading in NeuTu consists of a series of merges or splits. But unlike those tools, NeuTu has different approaches for scalable 3D object visualization and splitting, enabling intuitive operations in three dimensions. NeuTu has been used to both densely and sparsely proofread multiple regions of the fly brain, including connectomes of seven columns in medulla (Takemura et al., 2015) and the alpha lobe of the mushroom body (Takemura et al., 2017a).

# MATERIALS AND METHODS

**Table 1** provides terminology we used to describe our proofreading pipeline and software in this paper.

# Collaborative Proofreading Workflow Based on Segmentation

NeuTu has been designed and improved continuously based on our current workflow of proofreading large-scale segmentation results. Before developing NeuTu, we used Raveler to proofread connectomes, such as the single-column medulla reconstruction (Takemura et al., 2013). Raveler was developed to handle a block-based workflow, in which the whole data are divided into disjoint blocks and different proofreaders worked on these blocks in isolation (**Figure 1A**). With the rapid increase of image size and improvement of automatic segmentation, which benefited from both advances in deep learning and innovative imaging technologies, however, it is difficult to manage the block-by-block workflow without a proportional increase of the overhead cost of dividing and reintegrating the data. One major difference resulting from these changes is that a segment can occupy many blocks. Fixing an error in a given block, especially a false merge error, becomes cumbersome when only a small portion of the segment is visible in the block. Many errors need a larger context to identify. Therefore, it is critical to visualize or manipulate a 3D segment or body with a global context.

For a segmentation-based workflow, the input is a set of segments, each composed of a set of voxels, and proofreading will output a new set of segments by reassigning voxels. Although this basic assumption remains the same, the major change in the current work is that the input segments are produced from 3D segmentation directly instead of a two-step process of 2D segmentation and linking. It implies that the typical way of fixing errors on individual planes followed by updating linkages is no longer a suitable option. Since we are also dealing with isotropic data, such as images acquired from focused ion beam scanning electron microscopy (Xu et al., 2017), there should be no predefined principal direction for a 3D segmentation. The old pipeline relied on 2D segmentation slices with a preferred planar direction for slicing because anisotropic data acquired from the widely used transmission electron microscopy often has the XY resolution one order of magnitude higher than the Z resolution, naturally leading to data management that uses the Z-axis as the principle direction to see image

<sup>1</sup>https://knossostool.org

<sup>2</sup>http://eyewire.org

#### TABLE 1 | Terminologies used in our proofreading workflow.

fncir-12-00101 November 10, 2018 Time: 13:47 # 3


details more conveniently. A better workflow should be free of this constraint, encouraging human proofreaders to perceive a segment as a 3D object without worrying about its underlying representation.

More specifically, proofreading workflows that work on large datasets with high-quality segmentation requires the following functions, which NeuTu implements (**Figure 1B**):


### Architecture of NeuTu

fncir-12-00101 November 10, 2018 Time: 13:47 # 4

Since NeuTu is supposed to be used on multiple computers at the same time for proofreading a shared dataset, designing it as a client of a data service is a natural choice. Specifically, we built it as a client of a distributed, versioned, image-oriented dataservice (DVID)<sup>3</sup> , which provides a fast IO access to large-scale 3D image data. Many functions in NeuTu are tuned to exploit important features of DVID, such as versioning, and optimized for data formats that DVID provides. On the other hand, NeuTu is a GUI application, which allows the user to interact with data as intuitively as possible. To this end, we provide two visualization modes in NeuTu. The 2D View provides a slice-based display of image and annotation data, while the 3D View shows those data in the 3D space. These two views can control each other through a direct communication channel to facilitate navigation. A command issued by the user will be passed from the front-end view to a lower-level IO directly or through a data processing engine. Changes in the data will be returned to update the views accordingly. Information related to the change, such as which bodies have been modified, may also be sent to the Computing Service for updating non-critical but useful results such as skeletons, which are usually expensive to compute. Although the DVID service can receives inputs from multiple users, it does not prevent users from interleaving merge or split results inconsistently. Therefore, we introduced a service called the Librarian to coordinate the workflow of multiple users. The Librarian allows a user to lock a body to keep it from being modified by other users, and unlock it when it is done. This lock-based coordinating workflow is well-suited for managing segments because each segment has a unique ID for the Librarian to track its status. Whenever a user wants to manipulate a body through NeuTu, the NeuTu client will ask the Librarian to lock the body first. If the body has already been locked by another user, the Librarian will return an error message, keeping the client from modifying the body. The overall architecture is illustrated in **Figure 2**.

#### Feature Highlights Data Management and Flow via DVID

All major data, including the original EM volume images (grayscale data) are stored in DVID. In this sense, NeuTu is a client of the DVID server. Since NeuTu fetches data from DVID on demand, the difficulty in handling a large connectome on the client side can be minimized. For example, to merge two bodies, NeuTu only needs to send DVID a request containing the IDs of the bodies, without having to deal with actual voxels. Body size is not a big issue for merging because the computation is

as trivial as assigning a new ID to all voxels to merge, which is done by DVID and does not involve any inter-voxel relationship. In the case of visualizing or splitting a body, where big body size becomes a challenge for computation, we store the binary mask of each body separately as a sparse volume (i.e., only foreground voxels are recorded) in DVID. Each sparse volume is further compressed with run-length encoding (RLE). When a user wants to split a body, NeuTu can just download the binary mask and relevant grayscale data that contains EM signals. In DVID, images are stored as small fixed-sized blocks, with a typical size of 32 × 32 × 32 voxels or 64 × 64 × 64 voxels, for fast indexing. Retrieving a whole DVID block is usually faster than retrieving the same number of voxels distributed across multiple blocks. For optimal performance, NeuTu uses the same kind of block structure to manage grayscale data for a body. When a block contains voxels both inside and outside of the body, all the grayscale data in that block will be retrieved and stored in memory for further usage. Conforming to the block alignment by using a little more memory space to retrieve and store block-aligned data can lead to speedups since unnecessary data slicing is avoided in DVID.

NeuTu takes advantage of the sparse volume representation provided by DVID to allow manipulation of individual bodies. For example, when a body is selected, NeuTu downloads the binary mask of the body with RLE, which is typically much smaller than a list of individual voxel locations, allowing it to fit in memory. The downloaded body data can be used in two ways that are critical for 3D body manipulation. First, surface points can be extracted from the body data and converted into a convenient form for 3D visualization. The user can select any of the surface points in 3D to perform a further operation such as exploring grayscale data at the corresponding position or adding a bookmark at the position directly. Second, the sparse body data can be used as a mask for constraining the range of watershed-based split computation, thus reducing

<sup>3</sup>https://github.com/janelia-flyem/dvid

computational time significantly. Although the final watershed step has to be run over the bounding box around the body for computational efficiency, some pre- or post-processing steps such as downsampling or connected component analysis can be applied to the sparse form directly.

#### 3D Body Representation and Visualization

3D visualization is the most critical component of NeuTu to achieve the goal of intuitive operation in 3D. Showing a neuron in 3D allows the user to examine the reconstruction much more easily than displaying a slice. The user can click a surface point on the reconstruction and jump to that location in the 2D view. This is very helpful for examining problematic branches. For example, in some dataset, a branch terminal without synapses often means that a part of the branch is missing. The user can easily see the point and perform more careful examination in the 2D view.

In NeuTu, besides mesh visualization, 3D visualization of a body is also implemented by rendering 3D surface points of the body as a set of spheres with independent shading. Compared to other techniques such as mesh or volume rendering, rendering surface spheres has advantages in updating speed and intuitive interaction. Without the need to create faces and compute normals, surface points can be extracted more quickly than generating a mesh, so that loading a body into the 3D visualization engine takes shorter time when pre-computation is not an option. Overlapping spheres (**Figure 3A**) actually emulates surface shading (**Figure 3B**) naturally, thus saving the time of computing normals. Furthermore, the sphere representation provides an easy and intuitive interface for the user to select

a surface point in 3D, which only needs a click on the corresponding sphere. Once a position is selected, the user can then quickly navigate to grayscale image nearby or add a bookmark directly at that position. The user can even select a collection of spheres and adjust their sizes to highlight some morphological features in dense arborization (**Figure 3C**).

Instead of using pre-computed visualization primitives, NeuTu computes them to ensure consistency between the scene and its underlying data, which is subject to modification at any time. This requires fast computation of visualization data to reduce waiting time. Even though approximating with surface spheres helps, transferring and parsing data from the server can still be time-consuming for a large body. Therefore, we exploit the multi-scale data representation in DVID to allow a multi-scale updating strategy in NeuTu. When a body is selected for 3D visualization, its RLE data is fetched first from the lowest resolution representation, and then to the next higher resolution, until a certain size threshold is reached. Thanks to flexible version control in DVID, which allows us to set any checkpoint of proofreading results and create a new version from it, NeuTu is able to visualize body differences from different segmentation versions (**Figure 4**), which is particularly useful for tracking proofreading progress and for training.

#### Fast Interactive Segmentation

NeuTu uses a seeded approach for fixing false merges interactively. In this approach, the user needs to paint seeds on regions belonging to different neurons with different colors. The seeds can be painted in either the 2D view (**Figure 5A**) or the 3D view (**Figure 5B**). In the 2D view, a seed can be painted on any slice, and in the 3D view, a seed can be painted as a sequence of rays, each going through the target body from the first surface point to the last surface point it encountered on its path. The splitting results can be viewed in both views as well (**Figures 5A,C**).

To improve the speed of fixing false merges, we designed an efficient seeded splitting system on the client side. The system

is powered by a highly optimized implementation of the seeded watershed algorithm. According to our benchmark test, our seeded watershed implementation is about as twice fast as that in Insight Segmentation and Registration Toolkit (ITK)<sup>4</sup> , a widely used image processing library. Because membrane voxels are generally darker than cytoplasm voxels, the watershed computation is applied on grayscale data directly by assuming that lower intensity has a higher "water level." Although this may not be as accurate as using boundary maps or affinity graphs, especially when there are dark organelles near the boundary, saving computational overhead of edge enhancement leads to a good accuracy-speed trade-off in practice.

To further reduce computational time significantly for splitting a sparse volume, watershed computation is constrained to the body foreground. However, even excluding background voxels is often not enough for a quick turn-around. Therefore, NeuTu provides options of adding further constraints. The user can quickly check splitting results locally by trigging a local computation that only covers an area around the seeds. This provides a fast feedback for the user to adjust seeds accordingly, making less accurate segmentation more tolerable by making it easier to correct. Even though correct local splits do not guarantee correct global splits, it is a reasonable indicator of seed quality. Alternatively, the user can explicitly define a bounding area for splitting by painting a rectangle (**Figure 5A**). NeuTu can produce correct results if the bounding box contains the whole merging border, which is often constrained in a region much smaller than the bounding box of the body. Any piece outside of the local box will be attached back to the local split regions after connected component analysis (**Figure 5D**). Although it is not common, there may be multiple false merging spots that are far away from each other. The user can decide to split the segment progressively in this case.

It is possible that a body can have multiple disconnected components. Because watershed never crosses from one component to another, splitting such a body is the same as running watershed on each component independently and then joining regions that have the same watershed labels. If a component has no seed point on it, it stays with the origin body.

#### Synapse Editing

Besides neuron segmentation, synapse identification is also essential for building a connectome. Each synapse has a pre-synaptic element and a post-synaptic element to define a directed connection. In our system, synapses are stored in DVID as a kind of annotation data that can be queried by their coordinates. NeuTu reads synapses from DVID and displays them in both 2D (**Figure 6A**) and 3D (**Figure 6B**). The user can add a synaptic element at any position, connect/disconnect an element from/to another, or remove/move an element. Since synapse editing starts from automatic predictions, NeuTu adds some special visualization hints to the glyph of a synapse to indicate its confidence level or verification status.

#### Data Annotation for Workflow Management

Like any large-scale workflow, the workflow of large-scale proofreading should be organized to avoid duplicated work or blind spots. NeuTu provides various data annotation tools for such a purpose. In NeuTu, the user can annotate a body by giving it a biologically meaningful name and/or specifying its status. For example, the user can annotate a body with one of the seven pre-defined statuses, including "not examined," "traced," "traced in ROI," "partially traced," "orphan," "hard to trace," and "finalized." We can assign bodies to different proofreaders according to the statuses. For example, if a body is annotated as "hard to trace," we can assign it to an expert for further examination. The assignment is often done as a separate process outside of NeuTu. To help examine the bodies sequentially or as a group, NeuTu provides a table widget called the "sequencer" in which bodies can be filtered by regular expression or sorted by their properties such as synapse counts.

<sup>4</sup>https://itk.org

Those body-specific annotations do not tell where the problematic sites are. Therefore, we added another kind of annotations called bookmarks (**Figure 7A**). A bookmark is defined as a 3D point with a type and a comment. For example, the user can add a bookmark with a type "false merge" to specify that the corresponding location has a potential false merge error. A comment can be any text with more detailed information about the site. While bookmarks are handy for tagging interesting locations, they are only visible to their owners and are not necessarily used in proofreading. A special kind of location-specific annotation called reviewing marks is designed to show potential spots to proofread. Visible to all the proofreaders, those annotations have two statuses, to-do or done. To-do can be further labeled as "to merge" or "to split." They serve as a check list in the proofreading workflow. When a user displays a body in the 3D window, he/she is able to see all the reviewing marks in the 3D visualization (**Figure 7B**), with different colors to distinguish to-do or done statuses. The user can add, delete or modify an annotation in 3D directly. The synchronization between annotations and body IDs is managed by DVID. Whenever a body ID is changed by merging or splitting, all annotations associated with the old ID will be updated automatically to use the new ID.

#### Implementation

NeuTu is mainly written in C++, initially built upon the visualization and interaction engines from neuTube, software for tracing neurons in light microscope images (Feng et al., 2015). Important development updates introduced in NeuTu include replacing Qt4 with Qt5, allowing C++11 syntax to take advantage of modern C++ features, as well as using Conda Package Management<sup>5</sup> for cross-platform deployment. The code is publically available on Github<sup>6</sup> .

## RESULTS

Designed to be cross platform, NeuTu has been built and tested on several modern Linux systems (Fedora 16+, Scientific

<sup>5</sup>https://conda.io <sup>6</sup>https://github.com/janelia-flyem/NeuTu

Linux 7) and Mac OS X (10.12.6+). **Figure 8** shows a typical GUI of NeuTu on Mac OS X. The user can interact with 2D and 3D windows side by side, assisted by glyphs in the 3D view to infer relative positions of selected bodies and the field of view in 2D. As shown in **Table 2**, our focus on scalable segmentation-based reconstruction and feedback from in-house proofreaders have led to a combination of unique features in NeuTu compared to other available proofreading software. More details about NeuTu functions can be found in our online user manuals, including a short manual for quick start<sup>7</sup> and a long manual for full details<sup>8</sup> .

We evaluated how particular designs and strategies in NeuTu could help improve user experience and proofreading efficiency. The testing results obtained from 27 bodies showed that the multi-scale updating strategy described in Section "3D Body Representation and Visualization" could greatly improve response time (**Figure 9A**), which is measured by how long it takes to convert a body in DVID into geometric primitives for 3D rendering. Sampled from our 7-column medulla dataset (Takemura et al., 2015), the 27 bodies have 59 × 10<sup>6</sup> voxels on average, with a range from 15 × 10<sup>6</sup> to 16 × 10<sup>7</sup> voxels. A typical body among them can be displayed in real time (∼100 ms latency) at the lowest resolution, which is good enough to show the overall shape. To test if 3D visualization is important for proofreading, we chose 10 incomplete bodies from a superset of the 7-column medulla dataset (Shinomiya et al., unpublished) and asked 10 proofreaders to trace from each body with or without 3D visualization within 2 min. Bodies were assigned randomly to each proofreader without duplication. The results showed that 3D visualization

FIGURE 8 | Overall GUI of NeuTu on Mac OS X with 2D and 3D windows side by side to enable efficient proofreading by showing global information as well local details. For more details, please refer to the user manual (see text footnote 8).

TABLE 2 | Feature comparison for segmentation-based proofreading tools shows that NeuTu has a unique combination of features tuned to proofread large-scale dense connectomes.


<sup>1</sup>High: 3D visualization, bookmark, proofread directly in 3D, localization; Medium: 3D visualization, localization; Low: 3D visualization. <sup>2</sup>Slow computation, unable to handle large body.

<sup>7</sup>https://github.com/janelia-flyem/NeuTu/blob/master/neurolabi/doc/user\_manual/ neutu/quick\_start.pdf

<sup>8</sup>https://github.com/janelia-flyem/NeuTu/blob/master/neurolabi/doc/user\_manual/ neutu/manual.pdf

could almost double the efficiency of locating false splits by finding 4.40 ± 1.05 false spits per minute, compared to 2.25 ± 0.80 false splits per minute (p < 0.01) without the help of 3D visualization (**Figure 9B**). Note that 2D and 3D visualizations are more complimentary than exclusive. In some cases, such as in a severely over-segmented region, examining the 2D view may lead to more corrections than using 3D visualization only. 3D visualization becomes more heavily used when false splits are more sparsely distributed because of its advantage of showing global morphology. For example, one important feature to identify a false split is a branch terminal without synapses, which is difficult to miss in 3D visualization.

We have applied NeuTu to aid in the reconstruction of two EM datasets, the 7-column medulla dataset and the MB dataset, acquired from an optical lobe and a mushroom body of the fly brain, respectively. **Table 3** summarizes the data and reconstruction results. More details of data acquisition and processing methods can be found in Takemura et al. (2015, 2017a). Most of the work for the 7-column medulla dataset was done in Raveler, while NeuTu was still under development. The initial connectome was later refined by sparsely tracing more than one hundred neurons in NeuTu, which has significantly better 3D visualization for finding false merges efficiently. The extra tracing made the neurons become more complete and reliable for biological analysis, allowing us to add seven new neurons to the NeuroMorpho database<sup>9</sup> along with 525 neurons proofread by Raveler previously.

For the MB dataset, automatically computed segments and synapses were initially proofread in Raveler, and then imported into DVID. A second round of proofreading, which involved real-time collaborative work of multiple proofreaders, was performed in NeuTu to create the final connectome. We used focused proofreading (Plaza, 2016) available in Raveler to proofread 903,309 potential false splits and then used NeuTu to trace 24,480 bodies potentially with false splits that were often trickier to identify automatically than manually. NeuTu was also used to correct false merges in 9,870 bodies, which was challenging for Raveler. The scale of either connectome is significantly larger than the one previously produced by Raveler alone, which has skeleton length amount to about 105 mm and 8637 synapses (Takemura et al., 2013). The number of synapses in each connectome is also at least one order of magnitude bigger than any other connectome that has been published, such as 4,657 synapses in Wanner et al. (2016) and 1,700 synapses in Kasthuri et al. (2015).

# DISCUSSION

We have developed NeuTu for addressing emerging demands for connectome proofreading, such as managing big data smoothly,

<sup>9</sup>http://neuromorpho.org

FIGURE 9 | Experimental results showed that particular designs and strategies in NeuTu could help improve user experience and proofreading efficiency: (A) the waiting time for converting a body in DVID to geometric primitives for 3D rendering decreases exponentially as the downsampling scale increases, leading to real-time response (109 ± 66 ms) at scale 5 (downsampled by 32 along each dimension); (B) using 3D visualization can accelerate proofreading significantly, such as doubling the productivity in fixing false splits.


TABLE 3 | Summary of published connectomes proofread by NeuTu.

<sup>1</sup>The number of neurons for the 7-column dataset is more than we reported in Takemura et al. (2015) because unidentified neuronal segments are also included here.

leveraging high-quality segmentations, and simultaneous collaborative proofreading. Our software has specific functions tailored to these needs. For example, the highly interactive 3D visualization provided by NeuTu allows the user to trace a neuron quickly with terminal examination. NeuTu has helped two large connectome reconstructions for deciphering vision and memory, respectively, in the fly brain. Both of them are one order of magnitude more complex than any other connectome in the literature in terms of the number of connections. In the connectome involved in visual processing (Takemura et al., 2015), accurate neuronal morphologies of the T4 neurons traced in NeuTu has revealed more details of the motion-detection circuit (Takemura et al., 2017b) and led to accurate prediction of motion-detection cells in in silico simulation (Gruntman et al., 2018). The other circuit, which is located in the memory center of the fly brain, is the most detailed connectome related to memory and learning to date. From that highly detailed connectome, we have not only confirmed the surprisingly random connections in sense information encoding, but also found some new connections that had never been observed before (Takemura et al., 2017a).

NeuTu is mature enough to be employed as a research proofreading tool without any further development, assuming automatic segmentation is available along with its registered EM data. However, some improvements may provide better user-friendliness and efficiency. One major limitation of NeuTu is that it is bound to DVID. This means that relevant data need to be imported into DVID first for NeuTu to work. Currently there is no easy plug-and-play interface for a user to proofread their data that are commonly stored as image files or in their own database format. This problem can be alleviated by providing a single script for converting common data formats into a DVID repository. Likewise, exporting proofreading data from DVID back to the format preferred by the user will also be useful. A more fundamental solution regarding this issue is to add an abstract layer to separate NeuTu from the actual APIs of a specific database. With such a layer, adding support of a new database to NeuTu would not require changing NeuTu itself. Such flexibility would help NeuTu scale too. For example, the number of users working on the same dataset is limited by the capability of DVID in our practice. Choosing more performance backends would be a reasonable option to match specific application goals that need to go beyond these limits.

Regarding performance on the client side, one challenge for NeuTu, or any other similar proofreading software, is efficient processing of big bodies (>100 M voxels). Even though we have employed strategies such as sparse volume representation, bounded splitting, and multi-scale updating to enable NeuTu to manipulate big bodies smoothly in most cases, there are some bottlenecks like data fetching and flood filling that are proportional to the body size. In practice, we used downsampling to limit the bounding box of a body to 1G voxels for responsive visualization and splitting. The major side effect of downsampling is the loss of morphological details, which can be addressed by using hybrid resolution scales for the same body. While we have not encountered any memory issue for common operations such as splitting or merging, which usually involves only a small number of big bodies, some unusual operations such as loading many big bodies (say, hundreds) into 3D visualization can indeed cause the machine to run out of memory. Potential solutions to this problem include optimizing related data structures, limiting memory usage, and automatically suggesting the user to visualize skeletons instead.

Future work could involve adding more functions for biological analysis, such as querying neurons and local circuits freely and allowing the annotation of more biological details (e.g., synapse sizes and subcellular structures). These features would be particularly useful at the late stage of proofreading, when the focus is shifting from an image-guided process to extracting biologically relevant circuits.

Being more intelligent is another important direction of NeuTu development. This will involve significant research work, like making suggestions based on global shape priors, as well as some simpler tricks, such as presenting nearby orphans segments automatically. But for most of those intelligent strategies, one common challenge will be harnessing fast computation to create a pleasant user experience. Tackling the challenge by optimizing code or algorithms for real-time computation is not generally practical because data involved are usually big. Extensive precomputation will be necessary. Pre-computed results should be relatively light so that they can be uploaded into the database quickly without taking too much bandwidth and space. The results can be organized at different levels, such as skeletons at a lower level and skeleton similarity at a higher level, and do not need to be up to date. The NeuTu client should leverage such information, which can be noisy but has meaningful statistical patterns, to generate useful hints for the user.

Developing a proofreading tool does not only involve software development, it is also about designing proofreading strategies and workflows. Interestingly, this synergy between software design and reconstruction goals resulted in NeuTu software that can be used beyond the initial target application. For example, the flexible annotation system in NeuTu is directly shaped by the need of organizing collaborative proofreading, even though our main goal is to correct segmentation errors.

# AUTHOR CONTRIBUTIONS

TZ and SP designed the project and wrote the paper with inputs from other authors. TZ led the software development and wrote most of the code. DO and YY developed part of software.

# FUNDING

This work was supported by Howard Hughes Medical Institute.

# ACKNOWLEDGMENTS

We thank our proofreaders led by Pat Rivlin for helpful feedbacks and providing proofreading statistics, Stuart Berg for package management, William Katz for DVID instructions, and Linqing Feng for helps with 3D visualization.

# REFERENCES

fncir-12-00101 November 10, 2018 Time: 13:47 # 11


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Zhao, Olbris, Yu and Plaza. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Analyzing Image Segmentation for Connectomics

#### Stephen M. Plaza\* and Jan Funke

*Howard Hughes Medical Institute, Ashburn, VA, United States*

Automatic image segmentation is critical to scale up electron microscope (EM) connectome reconstruction. To this end, segmentation competitions, such as CREMI and SNEMI, exist to help researchers evaluate segmentation algorithms with the goal of improving them. Because generating ground truth is time-consuming, these competitions often fail to capture the challenges in segmenting larger datasets required in connectomics. More generally, the common metrics for EM image segmentation do not emphasize impact on downstream analysis and are often not very useful for isolating problem areas in the segmentation. For example, they do not capture connectivity information and often over-rate the quality of a segmentation as we demonstrate later. To address these issues, we introduce a novel strategy to enable evaluation of segmentation at large scales both in a supervised setting, where ground truth is available, or an unsupervised setting. To achieve this, we first introduce new metrics more closely aligned with the use of segmentation in downstream analysis and reconstruction. In particular, these include synapse connectivity and completeness metrics that provide both meaningful and intuitive interpretations of segmentation quality as it relates to the preservation of neuron connectivity. Also, we propose measures of segmentation correctness and completeness with respect to the percentage of "orphan" fragments and the concentrations of self-loops formed by segmentation failures, which are helpful in analysis and can be computed without ground truth. The introduction of new metrics intended to be used for practical applications involving large datasets necessitates a scalable software ecosystem, which is a critical contribution of this paper. To this end, we introduce a scalable, flexible software framework that enables integration of several different metrics and provides mechanisms to evaluate and debug differences between segmentations. We also introduce visualization software to help users to consume the various metrics collected. We evaluate our framework on two relatively large public groundtruth datasets providing novel insights on example segmentations.

Keywords: image segmentation, evaluation, metrics, connectomics, electron microscopy

# 1. INTRODUCTION

The emerging field of EM-level connectomics requires very large 3D datasets to even extract the smallest circuits in animal brains due to the high resolution required to resolve individual synapses. Consequently, at typical nanometer-level resolution single neurons in even a fruit-fly brain typically span over 10,000 voxels in a given orientation. An entire fly dataset which is less than 1mm<sup>3</sup> requires over 100TB of image data (Zheng et al., 2017).

#### Edited by:

*Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan*

#### Reviewed by:

*Stephan Gerhard, Friedrich Miescher Institute for Biomedical Research, Switzerland Shang Mu, Princeton University, United States*

#### \*Correspondence: *Stephen M. Plaza*

*plazas@janelia.hhmi.org*

Received: *21 July 2018* Accepted: *25 October 2018* Published: *13 November 2018*

#### Citation:

*Plaza SM and Funke J (2018) Analyzing Image Segmentation for Connectomics. Front. Neural Circuits 12:102. doi: 10.3389/fncir.2018.00102*

These dataset sizes pose several challenges for automatic image segmentation, which aims to automatically extract the neurons based on electron-dense neuron membranes. First, image segmentation algorithms struggle with classifier generalizability. For a large dataset, there are greater opportunities for anomalies that are significantly outside of the manifold of training samples examined. Even with advances in deep learning (Funke et al., 2018; Januszewski et al., 2018), the size and high-dimensional complexity of neuron shapes allow even small segmentation errors to result in catastrophically bad results as shown in **Figure 1**. Independent of dataset size, image segmentation struggles in regions with image contrast ambiguity, inadequate image resolution, or other image artifacts. This is particularly prominent for small neurites where synapses often reside (Schneider-Mizell et al., 2016). In **Figure 2**, the synapses for the neuron reside on the small tips of the neurons.

It should follow that image segmentation should be evaluated on large datasets with additional consideration for the correctness of small neurites critical for connectivity. Unfortunately, this is not the case. The authors are aware of no publications for new segmentation algorithms that emphasize this. Recent work (Maitin-Shepard et al., 2016; Januszewski et al., 2018) have evaluated segmentation on large datasets, such as Takemura et al. (2015). But these works do not consider synaptic connectivity explicitly, which is the ultimate application of the image segmentation. Neither SNEMI (Arganda-Carreras et al., 2015) nor (CREMI, 2016) segmentation challenges use datasets that span large sections of neurons. While they have been instrumental to meaningful advances to the field, they are ultimately limited by their small size and can under-represent problems as shown in **Figure 1**. This occurs because the actual cause of the error is in only one small region, but the impact is observed in many more regions.

There are reasons large-scale, connectivity-based evaluations are uncommon. Importantly, evaluating large datasets requires considerable ground truth that is time-consuming to produce. The groundtruth dataset in Takemura et al. (2015) is an order-of-magnitude bigger than the other public challenges but took 5 years of human proofreading and is still over three orders of magnitude smaller than the whole fly brain. We believe connectivity-based metrics have not been readily adopted because (1) it requires the annotation of synapse objects which is an independent step of the typical segmentation workflows, (2) the segmentation optimization objectives used in classifier training focus on lower-level, local topology (Rand, 1971; Meila, 2003 ˘ ), whereas connectivity is more global, (3) there are no proposed connectivity metrics that are widely adopted, and (4) there are no sufficiently large challenge datasets to meaningfully capture neuron connectivity. For algorithm designers, it is probably disconcerting to achieve poor evaluation scores based on connectivity that cannot be directly optimized in segmentation objectives without clever engineering and heuristics. While the training and local validation of segmentation is both practical and leading to significant improvements to the field, ignoring the higher-level objectives could lead to an over-estimation of segmentation quality and missed opportunities for more direct improvements for the target applications. We will show later that evaluating segmentation around synapses more directly results in less optimistic scoring compared to traditional metrics like (Meila, 2003 ˘ ). Recent work in Reilly et al. (2017) also introduced a metric that more appropriately weighs the impact of synapses on segmentation, though it does not explicitly consider connectivity correctness between neurons.

To address these issues, we propose a segmentation evaluation framework, which allows one to examine arbitrarily large datasets using both traditional and newly devised applicationrelevant metrics. Our contributions consist of (1) new evaluation metrics, (2) novel mechanisms of using metrics to debug and a localize errors, (3) software to realize these evaluations at scale, and (4) visualization to explore these metrics and compare segmentations.

We advocate an "all-of-the-above" philosophy where multiple metrics are deployed. In addition, we provide an approach to decompose some of these metrics spatially and per neuron to provide insights for isolating errors. This overcomes a limitation in previous challenge datasets that mainly produce summary metrics over the entire dataset, which provides no insight to where the errors occur. By decomposing the results, our framework is useful as a debugging tool where differences between segmentations are highlighted. While ground truth is ideal for evaluating different segmentations to know which one is better in an absolute sense, these debugging features highlight differences even if directly comparing two test segmentations without ground truth. This is critical for practically deploying segmentation on large datasets. The best segmentation can often be discerned by quickly examining the areas of greatest difference. While this provides only a qualitative assessment, this information is useful for identifying areas where new training data could be provided. Also, if one samples some of these differences, potential impact on proofreading performance can be discerned. For instance, such analysis might reveal that the most significant differences are due to one segmentation having a lot of large false mergers, which tend to be time consuming to fix.

Beyond decomposing metrics in new ways, we introduce the following evaluations:


The above is deployed within a scalable, clusterable software solution using Apache Spark that can evaluate large data on cloud-backed storage.

We evaluate this ecosystem on two large, public datasets. Our parallel implementation scales reasonably well to larger volumes, where a 20 gigavoxel dataset can be pre-processed and evaluated on our 512-core compute cluster in under 10 min with minimal memory requirements. The comparison results emphasize the importance of considering the synapse connectivity in evaluation. We also show that groundtruth is not necessary to generate interesting observations from the dataset.

The paper begins with some background on different published metrics for segmentation evaluation. We then introduce the overall evaluation framework and describe in detail several specific new metrics. Finally, we present experimental results and conclusions.

# 2. BACKGROUND

Several metrics have been proposed for segmentation evaluation, where the goal is analyzing the similarity of a test segmentation S to a so-called ground truth G. We review four categories of metrics in this section: volume-filling or topological, connectivity, skeleton, and proofreading effort.

#### 2.1. Volume-Filling or Topological

Topological metrics measure segmentation similarity at the voxel-level, so that the precision of the exact segmentation boundaries is less important than the topology of the segmentation. For instance, if the segmentation splits a neuron in half, the similarity score will be much lower than a segmentation that mostly preserves the topology but not the exact boundary. Example metrics of this class include the Rand Index (Rand, 1971; Hubert and Arabie, 1985), Warping Index (Jain et al., 2010), and Variation of Information (VI) (Meila, ˘ 2003). Since VI will be discussed later in this work, we define it below as:

$$VI(\mathcal{S}, \mathcal{G}) = H(\mathcal{S}|\mathcal{G}) + H(\mathcal{G}|\mathcal{S}) \tag{1}$$

where H is the entropy function. VI is decomposed into an oversegmentation component H(S|G) and an under-segmentation component H(G|S). A low score indicates high similarity.

#### 2.2. Connectivity

Examining topological similarity using the above metrics can be misleading in some cases since small shifts in segment boundaries can greatly impact the scores as noted in Funke et al. (2017). Furthermore, as shown in **Figure 2**, the synaptic connections are often on the harder-to-segment parts of a neuron that only make a small percentage of overall neuron volume. One potential solution is to define S and G in Equation 1 over a set of exemplar points representing synapses, instead of all segmentation voxels as done in Plaza (2016) and Plaza and Berg (2016). A similar strategy of measuring groupings of synapses was introduced in Reilly et al. (2017), which additionally breaks down results per neuron making the results more interpretable. While these metrics better emphasize correctness near synapses, it is not obvious how to interpret error impact to connectivity pathways.

### 2.3. Skeleton

Similar to topological metrics, the works in Berning et al. (2015) and Januszewski et al. (2018) describe metrics based on the correct run-length of a skeleton representation of a neuron. This class of metric provides an intuitive means of interpreting data correctness, namely the distance between errors. In (Berning et al., 2015), the run length can be very sensitive to small topological errors if one tries to account for synapse connectivity since synapses can exist in small neuron tips or spine necks where segmentation errors are more prevalent due to the small size of the processes. While this can be useful to emphasize synaptic-level correctness, it can also under-value a neuron that is mostly topologically correct. Januszewski et al. (2018) proposes an expected run length metric (ERL) that proportionally weights contiguous skeleton segments. While ERL is the most topologically intuitive metric, it conversely suffers from under-weighting correctness for small process such as at dendritic neuron tips in Drosophila or spine necks seen in mammalian tissue.

#### 2.4. Proofreading Effort

Tolerant-edit distance (Funke et al., 2017) and estimates of focused proofreading correctness time (Plaza, 2016) provide another mechanism to measure segmentation quality. Good segmentation should require few proofreading corrections (shorter edit distance) than bad segmentation. A segmentation that splits a neuron in half would be better than one with several smaller splits, since the former would only require one merge and the later several mergers. Designing interpretable edit distance formulations are challenging because different proofreading workflows could lead to very different proofreading reconstruction times.

The usefulness of the above metrics often depend on the application. For practical reasons, mathematically well-formed metrics like VI and ERL that have few parameters are often favored. Metrics that better reflect connectivity are harder to define since they depend more on the target application or require the existence of synapse annotation which is currently predicted in a separate image processing step from segmentation.

Finally, there has been only limited exploration in using segmentation metrics as debugging tools. Presumably, this becomes a bigger concern when evaluating larger datasets. Notably, the authors in Reilly et al. (2017) recognized this challenge and describe a metric that allows intuitive insights at the neuron level. In Nunez-Iglesias et al. (2013), the authors decompose the VI calculation to provide scores per 3D segment. For instance, the over-segmentation VI score H(S|G) can be decomposed as a sum of oversegmentation per ground truth neuron g:

$$H(\mathbb{S}|G) = -\sum\_{\mathbb{g}} P(\mathbb{g})H(\mathbb{S}|G=\mathbb{g})\tag{2}$$

Presumably, other metrics like ERL, can be used to provide neuron-level information for finding the worst segmentation outliers.

#### 3. METRIC EVALUATION ECOSYSTEM

We introduce a metric evaluation ecosystem that is designed to assess the quality of large, practical-sized datasets. To this end, we propose evaluation paradigms that emphasize interpreting and debugging segmentation errors that make comparisons between two different segmentations. While having ground truth is mostly necessary to quantify whether one segmentation is better than another, meaningful comparisons are possible without laboriously generated ground truth since the metrics highlight differences and these differences can be readily inspected. In the following few paragraphs, we will discuss the overall philosophy of our efforts. Then we will explore in more detail novel metrics and the software architecture.

In this work, we do not advocate a specific metric, but instead recommend an "all-of-the-above" framework where for each dataset multiple metrics are used to provide different subtle insights on segmentation quality. While not every popular metric is implemented, our framework is extensible and can support customized plugins.

We provide feedback on segmentation quality at different levels of granularity: summary, body, and subvolume.

#### 3.1. Summary

Each segmentation sample is evaluated with several scores applied to the whole dataset. These scores do not provide insight to where errors occur but provide a simple mechanism to compare two segmentation algorithms succinctly. VI and Rand index are two such examples. Section 3.4 introduces several new connectivity-based metrics.

#### 3.2. Body

We provide per segment (or body) statistics with respect to segments from both datasets S and G (G need not be ground truth). For example, this includes the per-body VI score defined in Equation 2, which provides insights on where over and undersegmentation occur in the volume. We highlight a couple new body metrics in section 3.4.

#### 3.3. Subvolume

When appropriate, metrics that are computed for the whole dataset are also applied to a regular grid of subvolumes that partition it. In this manner, the quality of segmentation can be assessed as a function of its location in the volume. This is useful for potentially detecting regions in the dataset where a classifier fails to generalize. For example, the framework runs VI on each subvolume. To partially disambiguate errors that originate in one region but propagate to another, distant region, we apply a local connected component algorithm to treat each subvolume as an isolated test segmentation<sup>1</sup> .

The evaluation framework can run over several distinct sets of comparison points. By default, segmentations are compared at the voxel level, i.e., the comparison points are all segmented voxels. If other sets of important points (such as synapses) are provided, analysis is similarly applied over these sets. The evaluation provides a mechanism to compare against oneself (no ground truth or alternative segmentation). We discuss metrics that enable self-evaluation in **Figure 3**. Comparisons to ground truth can be restricted to sparsely reconstructed volumes or dense labeling.

#### 3.4. Metrics

In the following, we highlight a few novel metrics for evaluating segmentation, which is a subset of all metrics implemented in the framework. These new metrics are divided into the categories of summary, per-segment, and self-comparison.

#### 3.4.1. Summary

We propose a metric to assess the connectivity correctness (CC) of the given segmentation S compared to ground truth G. At a high level, CC(S|G) defines the percentage of connections that match the ground truth connections. A connection is defined as an edge between two segments (neurons) that represents a synapse. There can be multiple connections between the same two segments. More formally:

$$\text{CC(S|G)} = \frac{\sum\_{\langle \mathbf{g}\_i \mathbf{g} \rangle \in G} |\varkappa(A\_S(\mathbf{g}\_i), A\_S(\mathbf{g}\_j)) \cap \varkappa(\mathbf{g}\_i, \mathbf{g}\_j)|}{\sum\_{\langle \mathbf{g}\_i \mathbf{g}\_j \rangle \in G} |\varkappa(\mathbf{g}\_i, \mathbf{g}\_j)|} \tag{3}$$

where x returns the set of synapse connections between two segments. AS(gi) determines the optimal assignment of groundtruth segment g<sup>i</sup> to a segment in S (e.g., using the Hungarian matching algorithm). The matching is one-to-one and if there is no match x will be an empty set. In practice, an algorithm that greedily finds a set of matches by using greatest segment overlap with ground truth is likely sufficient since one would not expect the set of intersecting segments in S to a given segment in G to greatly overlap with intersection sets to other segments in G in a manner that would require joint optimization. This is true by construction in the scenario where every segment in S is either a subset of a given segment G or equal to a set of g.

This metric is sensitive to both false merge and false split segmentation errors. If there is a false split, there will be fewer matching connections compared to ground truth. If there is a false merge between g<sup>1</sup> and g2, the one-to-one assignment A<sup>S</sup> ensures that AS(g1) 6= AS(g2) meaning that there will be no matching connections involving either g<sup>1</sup> or g2.

Additionally, we introduce a thresholded variant of the connectivity metric to emphasize the percentage of connection paths that are found with more than k connections. We modify Equation 3 to include this threshold and decompose into recall and precision components as defined below:

$$recCC\_k(\mathsf{S}|G) = \frac{\sum\_{\langle \mathcal{G} : \mathcal{G} \rangle \in G} I(|\mathsf{x}(A\_S(\mathcal{g}\_i), A\_S(\mathcal{g}\_j)) \cap \mathsf{x}(\mathcal{g}\_i, \mathcal{g}\_j)| > k)}{\sum\_{\langle \mathcal{G} : \mathcal{G} \rangle \in G} I(|\mathsf{x}(\mathcal{g}\_i, \mathcal{g}\_j)| > k)}\tag{4}$$

$$preCC\_k(\mathbb{S}|G) = \frac{\sum\_{\{\emptyset, \emptyset\} \in G} I(|\mathbb{x}(A\_S(\varrho\_i), A\_S(\varrho\_j)) \cap \mathbb{x}(\varrho\_i, \varrho\_j)| > k)}{\sum\_{\{s\_i s\_j\} \in S} I(|\mathbb{x}(s\_i, s\_j)| > k)}\tag{5}$$

The above metrics to measure the similarity between two connectomes have advantages over using a more general graph matching algorithm. First, by requiring an initial assignment of each segment to a groundtruth neuron (if a distinct match exists), the CC metric aims to better constrain the problem of measuring the similarity between two connectivity graphs, thereby avoiding the need for the computational complexity typical in general graph matching algorithms. Second, the CC metric allows one to express the matching in terms of individual neurons and number of connections preserved, which is more biologically intuitive compared to a general edit distance score.

In addition, to CC<sup>k</sup> , we define a class of statistics that analyzes the fragmentation of S compared to G based on the simple formula:

$$Frag = |\mathbf{S}| - |\mathbf{G}|\tag{6}$$

where a high score indicates that S consists of many more segments than G. While very simple, this provides a lower-bound on the number of edits (or segments to "fix") to transform S into G. In practice, we find that S is typically an over-segmented subset of G and Frag provides a reasonable edit distance estimate. We can extend Frag by extracting a subset of S and G, S ∗ and G ∗ , that represent a less-than-100% correct segmentation. More specifically, we define a thresholded fragmentation score, where S ∗ and G ∗ are the smallest set of segments whose cumulative size reaches a specified size threshold, where size can be number of voxels or synapses. This trivially computed measure allows us to discern the number of segments required to produce a connectome that is X% complete.

<sup>1</sup> In cases of serious false merging that results in incidental contact between segments far away from the error site, connected components within a subvolume containing this site will be ineffective.

#### 3.4.2. Body

As described in Equation 7, VI can be decomposed to provide insight about the fragmentation of a given segment. If this score is applied with respect to segment g, it provides an over-segmentation score of g. If this applied with respect to segment s, it provides an under-segmentation score of s. We can alternatively decompose the VI calculation to report the over and under-mergers that intersect a given segment. We define the under and over segmentation score for g as:

$$H(\mathbf{S}|\mathbf{g}) + H(\mathbf{g}|\mathbf{S}) = -P(\mathbf{g})H(\mathbf{S}|\mathbf{G}=\mathbf{g}) - \sum\_{\mathbf{s}} P(\mathbf{s})H(\mathbf{G}=\mathbf{g}|\mathbf{S}=\mathbf{s}) \tag{7}$$

where P is the the probability of g (or percentage of g in G). This metric is useful to provide a simple score for the neuron that has the worst segmentation. This metric works most naturally over a densely labeled G since the impact of the false merging can be more accurately assessed.

Additionally, we modified the metric in Equation 3 to provide a score for each g the percentage of connections that are covered. We further note which bodies are the most correct by simple overlap, which is conceptually similar to examining the largest error-free run lengths often used in skeleton-based reconstructions.

#### 3.4.3. Self-Compare

As mentioned, the ability to decompose the metrics at segment level allows one to compare two different segmentations. However, it is often useful to have some information on segmentation reliability when no comparison volume is available. One simple statistic that can be extracted is the number of segments that are needed to reach a certain volume threshold (as defined previously), which provide insights in regions that are relatively over-segmented compared to others. However, this metric can be misleading since neuropil regions vary in neuron packing density.

We introduce two metrics to better assess segmentation in the absence of ground truth: orphan segments and segmentation loops. Biologically, one does not expect a neuron to be a small fragment below a certain size K. A count of the number of segments below this threshold, provides a crude error measure. This will not uncover potential under segmentation errors. To find potential under segmentation errors, we note that neurons should have few connections to itself (self-loops). By counting the number of autapses or finding the segments that have a lot of autapses, we can detect potential false mergers. As segmentation gets better the effectiveness of using autapses as a proxy for false-merge errors is limited since such connections due exist in practice, such as in the Drosophila medulla connectome in Takemura et al. (2015). Therefore, the loop detector should be viewed as a mechanism to detect outliers due to either segmentation error or biological design and serve as a good entry point for analyzing a segmentation. Depending on the organism and the extent of the region being evaluated, additional metrics could be considered, such as ensuring that each segment has both inputs and outputs. We only formally consider orphans and self-loops in this work.

#### 3.5. Architecture

We introduce an Apache Spark-based system for comparing two, large segmentations at scale. The implementation is built over the framework described in Plaza and Berg (2016) and is available at https://github.com/janelia-flyem/DVIDSparkServices as the EvaluateSeg workflow . The segmentation and synapse data is stored using DVID (Katz and Plaza, 2018). In general, segmentation compresses to a small fraction of the original EM data size and we do not observe fetching segmentation to be a bottleneck in the analysis workflow. However, evaluating on datasets that are significantly larger than the 1 gigavoxel datasets common in SNEMI and CREMI necessitates a framework that can compute metrics on a large-memory, multi-core, cluster environment.

An overview of the software workflow is shown in **Figure 3**. We partition the dataset into disjoint, equal-size subvolume for a region of interest (ROI). A local connected component algorithm is computed for each subvolume and other filters are applied, such as (1) dilating groundtruth segment boundaries to reduce the impact of small variations in the exact boundary between segmentation and (2) filtering out neurons that are not groundtruthed for sparse evaluation. If the ROI being analyzed is part of a larger segmentation, one can run a global connected component algorithm which ensures that segments that merge outside of the ROI are treated as separate objects within the ROI. The global connected component algorithm is computed by examining the boundaries between all subvolumes in parallel and determining which components have a connecting pathway through the ROI.

For each subvolume, we compute a contingency table between segments in S (when not doing a self-comparison) and G (where G is treated like ground truth unless otherwise specified). The overlaps computed between S and G allow many of the metrics to be computed per subvolume and then combined into global summary and body stats. This is done over the set of voxels and optionally any available synapse (or other point) data. In the current workflow, one of the largest, non-parallelized compute components is this final grouping of results. Future work to further reduce these non-parallel points is possible but not currently necessary for the experimented data sizes.

The framework allows additional plugins that conform to the API to be added without changing the surrounding framework. In circumstances where this partitioning and combination strategy will not solve a given metric algorithm, it is possible to define a completely custom workflow based on the input segmentation. The current framework does not implement ERL or other skeleton-based metrics, but our ecosystem should admit for its straightforward inclusion.

The statistics from this computation are collected into a file that can be easily parsed. However, the myriad of metrics can make interpreting results overwhelming, so we designed a single web page application in Javascript as shown in **Figure 4** to improve accessibility. The web application groups similar stat types together displaying the list of summary stats and per-body breakdowns for provided metrics. A visualization tool shows a heat-map highlighting subvolume to subvolume variation in segmentation quality. The application also allows one to compare the summary results of two different segmentation evaluations. The web page application is available at https://github.com/ janelia-flyem/SegmentationEvaluationConsole.

# 4. EXPERIMENTS

We demonstrate our evaluation framework on two large, public datasets: a portion of the Drosophila medulla (Takemura et al., 2015) and mushroom body (Takemura et al., 2017). The medulla dataset segmentation and grayscale can be accessed at http://emdata.janelia.org/medulla7column, and the mushroom body dataset can be accessed at http://emdata.janelia.org/ mushroombody. Both datasets are around 20 Gigavoxels in size and contain over 100,000 synaptic connections. Since neither dataset is 100 percent accurate, we filter small orphan segments in the ground truth using options in the metric tool and we dilate ground truth neuron boundaries with a radius of two pixels. We compare these ground truths to initial segmentation generated using a variant of the algorithm developed in Parag et al. (2015). <sup>2</sup> A smaller portion of the optic lobe segmentation is also compared against a more recent segmentation algorithm (Funke et al., 2018). The purpose of the following experiments is to demonstrate the breadth of provided metrics, as well as, some insights that might impact how one analyzes segmentation results.

#### 4.1. Summary Results

The evaluation service produced a series of summary stats. A subset of these are depicted in **Figure 5**. The stats are split into two broad categories: voxel-based and synapse-based. The voxelbased stats provide volume-relevant information. The synapsebased stats emphasize only the exemplar points that define each input and output for a synapse.<sup>3</sup>

In both the mushroom body and medulla, we notice that there are very few false merge mistakes indicated by merge VI. Notably, the split VI is much higher when focusing near synaptic regions. The comparably higher values in the mushroom body highlight both the conservative segmentation used and the presence of very small, hard-to-segment processes. The thresholded segment count shows that to examine 50 percent of the synaptic points, a relatively small number of segments need to be examined compared to achieving 90 percent coverage. For

<sup>2</sup>We only have an archival version of mushroom body segmentation available where a few catastrophic false mergers were already eliminated.

<sup>3</sup> In Drosophila tissue, synaptic connections are polyadic meaning multiple postsynaptic targets for one pre-synaptic region. For this analysis, each pre and post-synaptic site represented by a single exemplar point is considered as one connection endpoint.


FIGURE 5 | Select metrics for the medulla and mushroom body dataset. The data shows voxel-based metrics like VI and less-common, but more useful synapse-based metrics. The histogram metric shows the many more segments are required to reach X percent of the total volume.

both datasets, the connectivity correctness defined by Equation 3 is very low, in particular in the mushroom body where the neurites are very small. This indicates that the automatic segmentation is far from being useful for biological analysis without proofreading.

The summary results also report the worst body VI score and the segment ID number corresponding to this body. We show one example from the medulla in **Figure 6**. The evaluation service reports the biggest overlapping segments. Notice that the top 10 biggest fragments only cover a small portion of the complex neuron arbor.

We compare the baseline segmentation with a newer segmentation approach in Funke et al. (2018) for a subset of the medulla dataset in **Figure 7**. As expected, Funke et al. (2018) achieves a better score across all reported metrics. While the VI scores indicate significant improvement, the fragmentation thresholds and synapse connectivity clearly show the advantages for the newer segmentation. There are far fewer segments to consider to reach different levels of completeness as seen in Frag thres. Perhaps more significant is the much greater percentage of neuron connections found with the new segmentation. The CC metrics are sensitive to large neurons being correct in addition to the small synapse processes being correctly segmented. Metrics less sensitive to this level of correctness, like the VI numbers reported, might, in effect, over-rate the quality of inferior segmentation.

### 4.2. Unsupervised Evaluation

The previous results show comparisons between test segmentations and ground truth. As previously explained, the metric service is useful for comparing two segmentations directly even if one is not ground truth since there are many stats that highlight differences useful for debugging. For instance, while the VI between two test segmentations fails to suggest which one is better, it does indicate the magnitude of the differences, can indicate whether one segmentation is oversegmented compared to the other, and gives a list of bodies that differ the most, which can then be manually inspected to determine segmentation errors. But we also introduced stats that do not require a comparison volume. We evaluated both medulla

and mushroom body in this way. In **Figure 8**, we see a heatmap highlighting the small orphan segmentation density over the subvolumes that partition both datasets. We define orphan as any segment with fewer than 10 synaptic endpoints. Visually, the diagram shows more errors in the alpha 3 lobe and proximal region of the mushroom body and medulla respectively. If we evaluate these regions separately against the ground truth, we observe that the supervised VI scores are consistent with the unsupervised visualization.

We were also able to find one neuron in the medulla dataset that had many autapses, which suggests a potential false merge. This worst neuron in the un-supervised analysis corresponds to the fourth worst body in the supervised analysis. This suggests that the autapse count can reveal false merge errors.

#### 4.3. Performance and Scaling

These datasets are much larger than previous challenge datasets but are still much smaller than the tera to peta-scale datasets that are being produced. One obvious solution to handling larger datasets is to run the framework on a larger compute cluster.

We show the scalability of our framework by evaluating our two sample datasets with varying numbers of cores. The charts in **Figure 9**, shows a breakdown of runtime between the top parallelizable portion of the code and the bottom, sequential small overhead. As the number of cores increase we observe a speedup that is slightly less than linear to the number of added cores (indicated by the trendline). We observe that the sequential overhead indicated by the lowest two section of each bar is roughly constant and a small portion of this time (the lowest section) could potentially be partially parallelized with future optimizations.

The results in the table suggest that 512 cores can roughly process around 20 gigavoxels in around 5 min, or over 60


FIGURE 7 | Comparing two segmentations from a subset of the medulla dataset. Unsurprisingly, the more recent segmentation from Funke et al. (2018) performs better on all metrics (indicated by the highlighted boxes). In particular, Funke et al. (2018) achieves much higher CC scores finding 33 percent of all neuron connections with weight greater than or equal to ten synapses, compared to only 9 percent for the baseline.

megavoxels of data per second, or 1 TB in a little over 4.5 h. Note that the comparison framework requires two datasets to be processed and this analysis includes the global connected

FIGURE 8 | Orphan density map. For both the medulla and mushroom body sample, the orphan count density (an unsupervised statistic) appears greater (darker) in regions with worse synaptic VI compared to the other regions.

bottom two sections of each bar. This non-parallel time is around 129 and 187 s from the mushroom body and medulla respectively.

components analysis, which is not necessary if segmentation is completely contained within the defined region. Also, note that medulla and mushroom body ROIs do not perfectly intersect the subvolumes, so more data is actually fetched to retrieve the entire 20 gigavoxel ROI.

In practice, we expect additional bottlenecks if there are a lot of small segment fragments which could lead to more computation in the sequential parts of the code and in shuffling data around on the network. Future work should aim to improve the performance when dealing with a large number of small fragments since its relevance to analysis is mostly in the aggregate and not at the individual fragment level. We do not observe slowness fetching the segmentation data, but the data could always be partitioned between multiple servers to allow for higher cumulative read bandwidth.

To further improve performance, we consider downsampling the segmentation. (A multi-resolution segmentation representation is available in DVID and does not need to be computed.) **Figure 10** shows both datasets at original resolution and downsampled by a factor of 2, 4, and 8 along each axis. One might expect that downsampling the dataset considerably would greatly change the statistics particularly related to fragmentation due to presumably small synaptic processes. Perhaps surprisingly, a few key metrics have a consistent value when downsampling by 4x suggesting that significant computation reduction is possible since full resolution is unnecessary. For example, the fragmentation scores in these datasets, which provide a rough estimate of the number of merge edits required, is similar (within 20 percent) to full resolution. Once the resolution starts getting worse than 40x40x40nm, there is considerable impact on the synaptic VI and the number of thresholded segments. However, the significant differences reported between the two segmentations in **Figure 7** are preserved even at the lowest resolution tested.


FIGURE 10 | Stability of various metrics when downsampling the dataset. When the voxel resolution is higher than 40 × 40 × 40 nm, the results are fairly consistent. When the voxel resolution is too low, several synapses on smaller neurites are missed. The 50, 75, and 90% connections number refers to the number of segments required to cover the specified percentage of connection endpoints.

# 5. CONCLUSIONS

In this work, we demonstrate a metric evaluation framework that allows one to analyze segmentation quality on large datasets. This work necessitated diverse contributions: new metrics that provide novel insights in large connectomes, a software framework to process large datasets, and visualization software to enable intuitive consumption of the results. All of these contributions, in synergy, were critical to enable segmentation evaluation in practical settings.

We implemented multiple metrics to provide different insights on segmentation. In particular, we introduced new connectivity-based metrics that clearly show that significant improvements are still needed to produce fully-automatic reconstructions, which seem to correctly reflect our observations in practice. Furthermore, we note that for purposes of comparison, it is possible to downsample the data significantly without significant impact on important metrics. Finally, we introduced the possibility of comparing two segmentations without ground truth, where evaluation can be done by manually inspecting the largest segmentation differences revealed by decomposing the metrics in different ways and providing useful visualizations, such as showing segmentation quality variation as a function of region location. We believe that this work should help accelerate advances in image segmentation algorithm development and therefore reduce bottlenecks in large connectomic reconstructions.

The diverse set of statistics produced by our workflow could make the task of comparing segmentations overwhelming, as one desires to know which is the best metric. This paper has taken an agnostic position to the best metric largely because it depends on the application. If one is concerned about optimizing proofreading performance, edit distance measures make the most sense. However, this is complicated because edit distance costs depend on the proofreading methodology. The fragmentation scores provide a very intuitive, parameterfree measure of segmentation quality if one has mostly tuned the algorithms to over segment, since the number of segments is a guide for the number of mergers required. To assess whether the segmentation can be used in a biologically meaningful way, our new connectivity metric will provide the best insight on the quality of the resulting connectome. For assessing general neuron shape correctness, ERL (which we do not currently implement) or VI can be used.

We expect additional improvement is needed to further parallelize sequential portions of the framework. Also, we believe that additional metrics should be invented that provide interesting insights for evaluating the connectivity produced from the segmentation. We have introduced a few metrics to this end in this paper. We advocate the inclusion of more metrics in evaluation to better understand the failure modes of segmentation, which will hopefully lead to the implementation of better algorithms.

# AUTHOR CONTRIBUTIONS

SP devised and implemented the core methodology, JF contributed some image segmentation and metric discussions.

## FUNDING

This study was funded and supported by Howard Hughes Medical Institute.

# ACKNOWLEDGMENTS

Stuart Berg helped with the experimental setup and was instrumental in implementing the DVIDSparkServices ecosystem, where our metric service was implemented. Bill Katz provided support for the Big Data infrastructure through the software DVID. Alex Weston helped in developing the web application used for visualization. Lowell Umayam helped to collect data necessary for experimental analysis. We would especially like to thank the FlyEM project team at Janelia Research Campus for general discussions and support.

## REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Plaza and Funke. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Heterocellular Coupling Between Amacrine Cells and Ganglion Cells

Robert E. Marc† , Crystal Lynn Sigulinsky† , Rebecca L. Pfeiffer, Daniel Emrich, James Russell Anderson and Bryan William Jones\*

Moran Eye Center, Department of Ophthalmology and Visual Sciences, The University of Utah, Salt Lake City, UT, United States

All superclasses of retinal neurons, including bipolar cells (BCs), amacrine cells (ACs) and ganglion cells (GCs), display gap junctional coupling. However, coupling varies extensively by class. Heterocellular AC coupling is common in many mammalian GC classes. Yet, the topology and functions of coupling networks remains largely undefined. GCs are the least frequent superclass in the inner plexiform layer and the gap junctions mediating GC-to-AC coupling (GC::AC) are sparsely arrayed amidst large cohorts of homocellular AC::AC, BC::BC, GC::GC and heterocellular AC::BC gap junctions. Here, we report quantitative coupling for identified GCs in retinal connectome 1 (RC1), a high resolution (2 nm) transmission electron microscopy-based volume of rabbit retina. These reveal that most GC gap junctions in RC1 are suboptical. GC classes lack direct cross-class homocellular coupling with other GCs, despite opportunities via direct membrane contact, while OFF alpha GCs and transient ON directionally selective (DS) GCs are strongly coupled to distinct AC cohorts. Integrated small molecule immunocytochemistry identifies these as GABAergic ACs (γ+ ACs). Multi-hop synaptic queries of RC1 connectome further profile these coupled γ+ ACs. Notably, OFF alpha GCs couple to OFF γ+ ACs and transient ON DS GCs couple to ON γ+ ACs, including a large interstitial amacrine cell, revealing matched ON/OFF photic drive polarities within coupled networks. Furthermore, BC input to these γ+ ACs is tightly matched to the GCs with which they couple. Evaluation of the coupled versus inhibitory targets of the γ+ ACs reveals that in both ON and OFF coupled GC networks these ACs are presynaptic to GC classes that are different than the classes with which they couple. These heterocellular coupling patterns provide a potential mechanism for an excited GC to indirectly inhibit nearby GCs of different classes. Similarly, coupled γ+ ACs engaged in feedback networks can leverage the additional gain of BC synapses in shaping the signaling of downstream targets based on their own selective coupling with GCs. A consequence of coupling is intercellular fluxes of small molecules. GC::AC coupling involves primarily γ+ cells, likely resulting in GABA diffusion into GCs. Surveying GABA signatures in the GC layer across diverse species suggests the majority of vertebrate retinas engage in GC::γ+ AC coupling.

Keywords: amacrine cell, ganglion cell, gap junction, GABA, retina, neural circuitry, transmission electron microscopy, computational molecular phenotyping

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Benjamin E. Reese, University of California, Santa Barbara, United States Karin Dedek, University of Oldenburg, Germany Citlali Trueta, Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz (INPRFM), Mexico

#### \*Correspondence:

Bryan William Jones bryan.jones@m.cc.utah.edu

†Co-first authors

Received: 23 May 2018 Accepted: 28 September 2018 Published: 14 November 2018

#### Citation:

Marc RE, Sigulinsky CL, Pfeiffer RL, Emrich D, Anderson JR and Jones BW (2018) Heterocellular Coupling Between Amacrine Cells and Ganglion Cells. Front. Neural Circuits 12:90. doi: 10.3389/fncir.2018.00090

# INTRODUCTION

fncir-12-00090 November 14, 2018 Time: 16:6 # 2

Retinal ganglion cells (GCs) are the signal outflow cells of the vertebrate retina: a network layer that integrates bipolar cell (BC) and amacrine cell signals and passes them to CNS targets. Like BCs, most GCs are part of a unidirectional synaptic chain, not evidencing any direct feedback to the preceding input stage. However, early physiological studies established the ability of a GC to excite amacrine cells, other GCs and even itself (Matsumoto, 1975; Marchiafava, 1976; Marchiafava and Torre, 1977; Mastronarde, 1983; Sakai and Naka, 1988; Sakai and Naka, 1990). This excitation was always sign-conserving and occurred with short latency, yet electrical synaptic transmission was often dismissed due to a lack of anatomical evidence, in stark contrast to many other retinal neurons (Vaney, 1994). Later, intracellular biotinylated tracer injection studies (Vaney, 1991, 2002; Vaney and Weiler, 2000) showed tracer diffusion patterns between ganglion and amacrine cells that were interpreted as coupling mediated by gap junctions (e.g., Bloomfield and Xin, 1997; Massey, 2008), and more recently confirmed with gap junction protein knockout mice (e.g., Schubert et al., 2005a,b; Pan et al., 2010).

Gap junctions are intercellular channels that mediate the flux of small molecules and ions and, therefore, are the anatomical basis for electrical synaptic transmission in the nervous system. Like chemical synapses, gap junctions are extremely diverse structures mediating intercellular signaling. The primary proteins of gap junctions are drawn from a large family of connexins with four transmembrane spanning domains, cytosolic domains that usually (though not always) provide predominantly homotypic or bihomotypic binding even if the junctions are heteromeric (Li et al., 2008; Rash et al., 2013), and intracellular domains that mediate recognition and binding of other gap junction proteins. In general, it is thought that the peak open conductance of a single connexon is principally related to its pore diameter (this is not always true) with complex modulation enabled by a range of mechanisms (Ek-Vitorin and Burt, 2013; Hervé and Derangeon, 2013) including connexin phosphorylation (Pereda et al., 2013; O'Brien, 2017), methanesulfonate-analog (taurine) binding (Locke et al., 2011), and many different adapter protein interactions (e.g., Zou et al., 2017). Light-induced changes in gap junctions are currently understood to modify the open conductance of a connexon through these mechanisms, but will not change the presence or absence of gap junctions at contact sites with coupling partners. That said, photopic or scotopic changes may alter gap junctional sizes.

Modes of coupling in the retina can be grouped into broad categories such as homocellular (coupling between the same "types" of cells) and heterocellular (coupling between different cell types). But what do we mean by "type" in the context of retina? Our terminology is based on computational classification theory where a class is the ultimate level of granularity (Marc and Jones, 2002). In this terminology, mammalian rod photoreceptors, blue cones, rod BCs, and AII amacrine cells, are all classes. In contrast, the categories of photoreceptors, bipolar, amacrine and GCs are all superclasses, as they contain collections of classes or larger intermediate groups often defined ad hoc (see **Supplementary Table S1**). So what we really mean by heterocellular coupling is that it occurs between superclasses with clearly different morphologies, such as between AII amacrine cells and ON cone BCs. Homocellular coupling occurs within classes or between intermediate groups with the same morphology. Thus CBb3n::CBb4 coupling, where :: denotes the presence of gap junctions between the pair, is homocellular (between BCs) but is cross-class coupling engaging two different BC classes (**Table 1**; also see Mills, 2001). GCs are unique among retinal cells in favoring heterocellular over homocellular coupling. While sparse ultrastructure studies support in-class homocellular coupling for some GC classes (e.g., Hidaka et al., 2004), tracer coupling surveys (Bloomfield and Xin, 1997; Völgyi et al., 2009; Pan et al., 2010) of many GC classes suggests that most participate in heterocellular coupling with amacrine cells. In-class homocellular coupling, appears rarely, although it is impossible to distinguish between direct GC::GC coupling and indirect GC::AC::GC coupling when the tracer-labeled cohort includes both amacrine and GCs. Here, we show that specific GCs in the retina exhibit common rules for heterocellular coupling with amacrine cells, ranging from none to extensive. We have yet to identify instances of GC in-class homocellular coupling and have no proven cross-class homocellular coupling.

While we know quite a bit about the general patterns of GC heterocellular coupling from tracer coupling studies, the network topology for the specific cell class partnerships involved and significance of coupling between the cell classes is elusive. Heterocellular coupling with amacrine cells subserves a circuit for synchronous GC firing (Brivanlou et al., 1998; Völgyi et al., 2013a), which may contribute to encoding aspects of the visual scene, such as direction (Meister and Berry, 1999; Ackert et al., 2006; Schwartz et al., 2007). There has also been discussion about whether coupling leads to maladaptive receptive field center expansion that would depress spatial resolution (Massey, 2008). However, two anatomical tools can assess the extent of coupling, enable precise definition of the partners and lead to more refined models of function: computational molecular phenotyping (CMP) and connectomics. While physiological analyses will always be definitive arbiters of global network functionality, connectomics can resolve network topologies that physiology cannot (e.g., Lauritzen et al., 2016). CMP allows quantitative specification of the small molecule signatures of


retinal neurons, especially GCs (Marc et al., 1995; Marc and Jones, 2002). Here, we simply asked: what is the network embedding (in the mathematical sense) of GC::AC motifs? The answer is that for two specific GC classes, transient ON (tON) directionally selective (DS) and OFF alpha, heterocellular coupling exclusively occurs with multiple classes of γ+ amacrine cells that enable diverse modes of network specificity depending on the topology of the coupled inhibitory network. For the tON DS GC network, excitation of the GC may lead directly to the inhibition of neighboring GCs of differing classes.

Diffusion of small molecules, such as dyes and biotinylated tracers, through gap junctions has long been used to identify coupling between retinal cells (Vaney, 1994). Glycine, a small metabolite, readily identifies ON cone BCs due to glycine diffusion through gap junctions with AII amacrine cells, as cone BCs neither synthesize nor transport it (Cohen and Sterling, 1986; Vaney et al., 1998; Haverkamp and Wassle, 2000; Deans et al., 2002; Petrides and Trexler, 2008). Other small molecules are also likely to diffuse through gap junctions and accumulate, such as GABA from the γ+ amacrine cells to which the tON DS GC and OFF alpha GC are coupled. We show that both cells contain intermediate levels of GABA. In mammals, many classes of GCs exhibit an intrinsic GABA signal superimposed on a classic high-glutamate, high-glutamine and low taurine GC signature, suggestive of heterocellular coupling with γ+ amacrine cells (Marc and Jones, 2002). We note that no known GABA transporters have been described in any GCs, much less in the adult rabbit retina (Hu et al., 1999), and there are no studies that definitively report GAD in the GCs (in contrast to the amacrine cells in the GC layer), though there are studies that report GAD mRNA in developing rat retina (Brecha et al., 1991; Dkhissi et al., 2001), no functional protein has yet been identified. It should also be pointed out that the presence of GABA in the GCs does not imply that they are themselves, inhibitory. That circumstance would depend upon GABA vesicular loaders being present at the GC terminals. Rather, we only hypothesize about GABA being present due to coupling of GCs to amacrine cells where that GABA derives. It should also be noted that GABA is a central carbon metabolite that can be utilized for a number of biosynthetic pathways. As we will show, that signal is not unique to mammals.

# MATERIALS AND METHODS

#### Samples

Over 40 years our laboratory has collected retinal samples from over 50 vertebrate species spanning all classes. All euthanasia methods followed institutionally approved procedures, some of which changed over the years IACUC oversight evolved. Aquatic vertebrates were euthanized via cervical transection and double pithing (pre-1995) or sedated in 0.2% methanesulfonate prior to cervical transection (post-1995). Reptiles were similarly euthanized by cervical transection and double-pithing (pre-1995) or IP injection with 10% urethane followed by cervical transection. Mammals were euthanized by urethane overdose and thoracotomy (rabbits) or decapitation (pre-2014, mice), deep isoflurane anesthesia and thoracotomy or decapitation (2015); or Beuthanasia <sup>R</sup> euthanasia and thoracotomy (rabbits, post-2015). The basic fixation method for all of them has been the same, as summarized in Marc (1999b): 250 mM glutaraldehyde, 1320 mM formaldehyde in either cacodylate or phosphate 0.1 M buffer pH 7.4, 3% sucrose, 1% MgSO<sup>4</sup> or 1% CaCl2. All tissues were embedded in Eponate resins (Marc et al., 1978), serially sectioned at 100–250 nm onto array slides, probed for small molecules (Marc et al., 1998), visualized by quantitative silver-immunogold detection (Marc and Jones, 2002), and imaged as described below. Some retinas were incubated for 10 min in either teleost saline (Marc et al., 1995) or Ames medium (Marc, 1999a,b) containing 5 mM 1-amino-4-guanidobutane (AGB) and either 1 mM NMDA or 0.05 mM kainic acid for excitation mapping of retinal GCs.

#### Immunocytochemistry

For the purposes of this paper, data from ≈20 years of post-embedding immunocytochemistry were analyzed and summarized. The same protocols and antibodies were used for all analyses. It is important to note that post-embedding immunocytochemistry for glutaraldehyde-trapped amines or imines is idempotent: once the sample is fixed and embedded, no detectable changes in immunoreactivity occur, even over decades. Indeed, tissues deriving from multiple species fixed in mixed glutaraldehydes and plastic embedded over 1980–1990 and published (Marc et al., 1990; Mills and Massey, 1992; Kalloniatis et al., 1996; etc. . .) have been directly compared with blocs of the same species (e.g., goldfish, rabbit, human, primate etc.) fixed in the past few years. They are indistinguishable. A good reference for this is Jones et al. (2003) where blocs of ≈30 individual transgenic rats had been prepared in the 1980s by Matthew LaVail. Rat retinas prepared post-2000 for this paper showed the same strength of GABA signals as blocs prepared in the 1980s. Signals were indistinguishable, and there is no published evidence showing any signal decline in resin embedded samples.

The key marker for heterocellular coupling is 4-aminobutyrate (GABA) detected in post-embedding immunocytochemistry (Marc, 1999b) using YY100R IgG (RRID:AB\_2532061) from Signature Immunologics Inc. (Torrey, UT, United States). Additional channels for cell classification (Marc et al., 1995; Anderson et al., 2009, 2011b) targeted AGB (B100R, RRID:AB\_2532053), glutamate (E100R, RRID:AB\_2532055), aspartate (D100R, RRID:AB\_2341093), glycine (G100R, RRID:AB\_2532057), glutamine (Q100R, RRID:AB\_2532059), and taurine (TT100R, RRID:AB\_2532060) from Signature Immunologics Inc. For ease of notation, the Greek nomenclature for amino acids is used: GABA (γ), Glutamate (E), Glutamine (Q), Aspartate (D), Glycine (G), and Taurine (τ). AGB is denoted with (B). The activity tracer 1-amino-4-guanidobutane (AGB) is used to map both endogenous and exogenous ligand-driven glutamatergic signaling in single cells. Guanidinium cations are permeable to a wide variety of non-selective cation channels. The Guanidinium analog, AGB has demonstrated the same non-selective cation channel permeability to that seen by guanidinium (Yoshikami, 1981; Qwik, 1985; Kuzirian et al.,

1986) and can be utilized as channel permeant markers by selectively activating glutamate receptors (Marc, 1999b; Marc and Jones, 2002), and allowing AGB to diffuse in along a concentration gradient. In essence, the tissue is incubated in a high concentration of AGB, which enters the cell through cation channels when the cell is activated. In the case of RC1, a flicker photopic light was used to drive neuronal classes allowing AGB entry via cation channel opening in response to glutamate receptor activation in neuronal classes (Marc, 1999a,b; Marc and Jones, 2002; Marc et al., 2005; Anderson et al., 2011b). All IgGs were detected with silver-intensified 1.4 nm gold granules coupled to goat anti-rabbit IgGs (Nanoprobes, Yaphank, NY, SKU 2300), imaged (8-bit monochrome 1388 pixel × 1036 pixel line frames) in large mosaic arrays with a 40× oil planapochromatic objective (NA 1.4) on a 100 × 100 Märzhäuser stage and Z-controllers with a QImaging Retiga camera, Objective Imaging OASIS controllers, and Surveyor scanning software (Anderson et al., 2011b; Lauritzen et al., 2016).

Raw signal is used to describe the original image acquired following staining without any image processing. Density mapped images are obtained from light microscopy of the silver intensified antibody labeled images. In these images, darkness of a region indicates a higher density of antibody labeling. Intensity mapped images are the inverted image of density mapped images, we invert these images to better facilitate the readers ability to interpret the small molecule mixtures within cells. Theme mapping is the assignment of a color to each cell class generated through k-means cluster analysis and overlaid in the same space as the original image to visualize which cells cluster together, and therefore have the same cell signature. Segmentation of cell classes using amino acid labeling was performed as previously described (Marc et al., 1995; Marc and Jones, 2002). In brief, IgG labeled sections were co-registered and clustered as N-dimensional images using k-means. Each separable cluster is made up of a distinct signature of concentrations of multiple amino acids unique to that cell class. The clustering results were then remapped in the same x–y dimensions as the original IgG image. This graphical representation of the cell classes is termed a theme map. Using the theme map as a mask, the underlying histograms can be evaluated for each cell class, where the histogram demonstrates the approximate log concentration of small molecule within the cell. For a more comprehensive review of these methods see Marc and Jones (2002). Image analysis, histogram thresholding, object counts and spacing measures were performed using ImageJ 2.0.0-rc-43/1.51w (Rueden et al., 2017) in the FIJI Platform (Schindelin et al., 2012) and Photoshop CS6 (Lauritzen et al., 2016).

### Connectomics in Rabbit Retinal Volume RC1

Connectome assembly and analysis of volume RC1 has been previously described (Anderson et al., 2009, 2011a,b; Lauritzen et al., 2012, 2016; Marc et al., 2013, 2014a) and only key concepts expanded here. RC1 is an open-access rabbit retina volume imaged by transmission electron microscopy (TEM) at 2 nm and includes 371 serial 70–90 nm thick sections, with six and twelve optical sections flanking the inner nuclear and ganglion, cell layers, respectively, containing small molecule signals and additional intercalated optical sections throughout (Anderson et al., 2011b). The retina was dissected from euthanized light-adapted female Dutch Belted rabbit (Oregon Rabbitry, OR) after 90 min (under 15% urethane anesthesia, IP) of photopic light square wave stimulation at 3Hz, 50% duty cycle, 100% contrast with a 3 yellow – 1 blue pulse sequence (Anderson et al., 2011b) with 13–16 mM intravitreal AGB. All protocols were in accord with Institutional Animal Care and Use protocols of the University of Utah, the ARVO Statement for the Use of Animals in Ophthalmic and Visual Research, and the Policies on the Use of Animals and Humans in Neuroscience Research of the Society for Neuroscience. Each retinal section was imaged as 1000–1100 tiles at 2.18 nm resolution in 16- and 8-bit versions, and as image pyramids of optimized tiles for web visualization with the Viking environment (Anderson et al., 2011a). Synapses and other intercellular relationships and intracellular structures were identified anatomically from TEM images and re-imaged at 0.27 nm resolution with goniometric tilt where necessary for validation. Neural networks in RC1 have been densely annotated with the Viking viewer (Lauritzen et al., 2016), reaching over 1.4 million annotations of 3D rendered volumetric neurons, processes, pre- and postsynaptic areas, locations in the volume with subnanometer precision (Jensen and Anastassiou, 1995), and explored via graph visualization of connectivity and 3D renderings as described previously. The volume contains ≈1.5 M annotations, 104 rod BCs, >145 classified, 24 unclassified, 10 classified partial arbors, 300 amacrine cells and 20 GC somas. This density of annotations belies the additional work required to validate, classify and scale. Each annotation is a size and location entity coupled to a full metadata log (Anderson et al., 2011a) and has been validated by at least two tracing specialists; many have been revisited 5–10 times, representing a total of 7 person-years of work. No current automated tracing tool makes fewer errors than a trained human annotator (even our own: Jagadeesh et al., 2013). Therefore, any time saved by automation is negated by the necessity for human cross-checking, validation and correction/re-annotation. Rendered neurons in RC1 were produced in Vikingplot (Anderson et al., 2011a,b) and VikingView (Lauritzen et al., 2016).

#### Mining Coupled Ganglion Cell Networks

Candidate GC coupling networks in RC1 were visualized and annotated by identifying GABA-positive (γ+) GC somas and dendrites in Viking<sup>1</sup> (RRID:SCR\_005986) in the intercalated GABA channels and by searching the RC1 database for coupling connections using network graph tools and database queries. All resources are publicly accessible via Viking and a range of graph and query tools are available at connectomes.utah.edu. All cells in this article are numerically indexed to their locations, network associations, and shapes. The data shown in every TEM figure can be accessed via Viking with a library of ∗ .xml bookmarks available at marclab.org/GCACcoupling. Each cell index number in the RC1 database can be entered into

<sup>1</sup> connectomes.utah.edu

different software tools for analysis, visualizations, or queries: Viking, Network Viz, Structure Viz, Info Viz, Motif Viz (Viz tools are based on the GraphViz API<sup>2</sup> , developed by AT&T Research, RRID:SCR\_002937), and VikingPlot developed by the Marclab; and VikingView developed by the University of Utah Scientific Computing and Imaging Institute. Further, Viking supports (1) network and cell morphology export into the graph visualization application Tulip<sup>3</sup> developed by the University of Bordeaux, France; (2) cell morphology for import into Blender<sup>4</sup> (RRID:SCR\_008606); and (3) network queries for Microsoft SQL and Microsoft Excel with the Power Query add-in to use the Open Data Protocol (OData.org) to query connectivity features. More efficiently, we discover and classify coupling networks in Tulip with TulipPaths: a suite of regex (regular expression) based Python plug-ins for network queries<sup>5</sup>,<sup>6</sup> , . Tulip networks can be directly exported from our connectome databases with a web query tool at connectomes.utah.edu and all data used in this article can be accessed via marclab.org/GCACcoupling.

#### Statistics

Small molecule signal comparisons across groups were done by both k-means clustering and histogram analysis using PCI Geomatica (Toronto, Canada) and CellKit based on IDL (formerly ITT, now Harris Geospatial, Melbourne, FL, United States) as described in Marc and Jones (2002). Various parametric and non-parametric analyses of feature sets (e.g., gap junction numbers, sizes) and power analyses were performed in Statplus:mac Version v6<sup>7</sup> (RRID:SCR\_014635) and R<sup>8</sup> (RRID:SCR\_001905).

#### Signatures

The signature hypothesis is the concept that each morphologically and functionally distinct cell would also possess distinct neurochemical compositions (Burnstock, 1976; Watt et al., 1984). We define the signature as quantitative differences in small molecule concentration mixtures as determined by k-means cluster analysis, indicating unique cell classes.

#### RESULTS

#### Phylogeny of Heterocellular Ganglion Cell Coupling With GABAergic Amacrine Cells

Our analysis of two γ+ GC classes in connectome RC1 demonstrates a mechanism by which the small molecule GABA could accumulate in GCs: heterocellular coupling via numerous small gap junctions with sets of γ+ amacrine cells. Thus, GABA signals superimposed on a classic high-glutamate, high-glutamine, and low taurine GC signature, can in turn be used to screen vertebrates for possible heterocellular GC::AC coupling. Specifically, cells in the GC layer with GABA signal histograms matching those of conventional amacrine cells (1–10 mM) are classified as displaced amacrine cells and those with intermediate signals (0.1–1 mM) are classified as provisionally coupled GCs (see Marc and Jones, 2002 for calibrations). In many species, we are also able to correlate these intermediate GABA levels with classical high glutamate signals of GCs and distinctly large GC sizes (e.g., Marc and Jones, 2002). Using the marclab.org tissue database we reviewed 53 vertebrate species spanning all vertebrate (**Supplementary Table S2**) classes to assess the scope of potential coupling. Importantly, evidence of GC heterocellular coupling with GABAergic amacrine cells occurs in every vertebrate class, even if other markers of comparative function vary: e.g., Müller cell GABA transport (limited to Cyclostomes, Chondrichthyes, Mammals and advanced fossorial ectotherms such as snakes), horizontal cell GABA transport (limited to most bony ectotherms) and horizontal cell GABA immunoreactivity (dominant in bony ectotherms and variable in mammals). The only vertebrate class we can say appears to clearly lack evidence of heterocellular GC::AC coupling is Testudines: turtles.

In every vertebrate class that shows a potential coupling profile, the GABA signal and GC types involved are diverse. **Figure 1** shows the spectrum of GABA signals in the rabbit GC layer, just below the visual streak, obtained by registering the glutamate (**Figure 1A**) and GABA (**Figure 1B**) channels of 2385 cells in the GC layer. The signals in **Figure 1B** reveal that GABA levels range from undetectable in many cells to levels that nearly match those of conventional amacrine cells, starburst amacrine cells in particular. In between are a range of concentrations far lower than any GABAergic amacrine cell (Marc and Jones, 2002) but much higher than background. Our previous assessments of the selectivity of the YY100R anti-GABA IgG (Marc and Jones, 2002) and competition assay results are shown in **Supplementary Table S3**, and range from 10<sup>4</sup> to 10<sup>6</sup> log units in concentration. Thus, the intermediate values cannot be due to cross reactivity with any plausible alternate biomarkers (e.g., L-alanine, β-alanine, taurine, etc.), else they would have to be present at levels of 1–100 M (100 µM low signal range × 104–10<sup>6</sup> cross-reactivity), which is physically impossible. Glutamate concentrations seen in GABAergic neurons is over a log unit lower than levels of glutamate found in presumptive glutamatergic cells. This range of glutamate immunoreactivity in GABAergic neurons has been described before (Marc et al., 1990, 1995) and it is likely that all GABA cells have at least some detectable glutamate given that glutamate is a central carbon skeleton metabolite and is the direct precursor to GABA synthesis via glutamate decarboxylase (GAD).

The intermediate ranges of GABA signals are associated with GC soma sizes ranging from some of the largest to some of the smallest GCs (**Figure 1C**), and the GC layer is separable by either clustering or histogram thresholding (Marc et al., 1995) into pure glutamate signal GCs (uncoupled), γ+ GCs (provisionally coupled) and starburst and minor displaced amacrine cell cohorts

<sup>2</sup> graphviz.org

<sup>3</sup> tulip.labri.fr

<sup>4</sup>Blender.org

<sup>5</sup>https://github.com/visdesignlab/TulipPaths

<sup>6</sup>https://docs.python.org/2/library/re.html

<sup>7</sup>www.analystsoft.com/en/

<sup>8</sup>www.r-project.org

FIGURE 1 | Glutamate and GABA colocalization in the rabbit ganglion cell layer; registered serial 200 nm horizontal sections through the plane of the ganglion cell layer with silver density visualization. (A) Raw glutamate signals of ≈2100 cells, density mapped; Scale, 0.5 mm. (B) Raw GABA signals of the same cells, density mapped. (C) Intensity mapped [inverse images of A,B)] registered channels with GABA (γ) signals encoded as a yellow (y) channel (R + G) and glutamate signals (E) as the blue (b) channel (γE = yb mapping) to create additive and quantitative concentration maps that reflect the free amine content of the cells. (D) Theme mapped data produced through GABA histogram segmentation. Magenta: high GABA content (5–10 mM) population containing mostly starburst amacrine cells and a few displaced amacrine cells. Yellow: medium GABA content population containing provisional γ+ ganlion cells (0.1–1 mM). Cyan: Ganglion cells with no measurable GABA content (<0.1 mM); small fragments represent portions of cross-sectioned cell dendrites. (E) Glutamate histograms of peak normalized pixel number (N) vs. pixel value (PV) for starburst amacrine cells (SACs), GABA-positive ganglion cells (γ+ GCs) and GABA-negative ganglion cells (GCs). The pixel value is the digital grayscale readout from the raw imagery, ranging from 0 to 255 and the peak normalized pixel number is the normalized maximum frequency of pixels in the image for a given pixel value. Pixel value reflects the quantitative amounts of small molecules which are log-linearly scaled with histogram pixel value representing an approximation of concentration (Marc et al., 1995). (F) GABA histograms of peak normalized pixel number (N) vs. PV for starburst amacrine cells (SACs), GABA-positive ganglion cells (γ+ GCs) and GABA-negative ganglion cells (GCs).

(**Figure 1D**). Importantly, all γ+ GCs show glutamate signatures indistinguishable from γ− pure glutamate GCs (**Figure 1E**), while starburst and other displaced amacrine cells display much lower glutamate contents similar to γ+ amacrine cell signatures in various vertebrate species (Marc et al., 1995; Marc, 1999a). The amacrine cell cohort is unique in quantitative glutamate and

GABA signatures, and soma size, while the γ+ GCs and γ− GCs are not discriminable in glutamate signatures, or soma size and are separable as an intermediate class only by the modest GABA signatures of γ+ GCs.

GABA signals in GCs are not unique to mammals. Teleost fishes represent the Actinopterygii, a vertebrate class with ≈400 Mya divergence from class Sarcopterygii, while infraclass Teleostei is of even more modern origin (≈310 Mya) with a massive post-Mesozoic, early Cenozoic expansion (Friedman, 2010) contemporaneous with mammalian speciation. The emergence of the cyprinids (goldfish and zebrafish) is extremely modern with estimated peak speciations in the Miocene and even early Holocene (Dubut et al., 2012), postdating the divergence of anthropoid primates (Pozzi et al., 2014). Arguably, comparing mammals and teleosts is one of the most diverse spans that could be conceived in assessing a putative synapomorphy (specialization of a clade) such as coupling, with a last common ancestor in the Devonian. **Figure 2** shows varying GABA signals exhibited by cells within the GC layer of the goldfish Carassius auratus. There are clearly different classes of GCs with GABA

FIGURE 2 | Glutamate and GABA colocalization in the goldfish ganglion cell layer; registered serial 200 nm sections with silver density visualization inverted (with a logical NOT) to an intensity display (Marc et al., 1995). (A) GABA (γ), AGB (B), and glutamate (E) signals assigned to the red (r), green (g), and blue (b) channels, respectively, creating an rgb image reflecting the combined small molecule signature. AGB permeation was activated in vitro with 50 µM kainic acid (KA) in the presence of 10 mM AGB in Hickman's Teleost saline (Marc et al., 1995). This signature separates ON starburst amacrine cells (small circles) with a yellow signal mixture (high GABA and AGB, representing classic strong starburst amacrine cell KA responses) from cyan ganglion cells (high glutamate and AGB, representing strong ganglion cell responses to KA), light blue ganglion cells (high glutamate, low AGB, representing weak ganglion cell responses to KA), and deep blue spherical terminals of Mb ON cone bipolar cells (surrounded by a polygon), which lack ionotropic glutamate receptors. The northwest arrow labeled with the E symbol indicates a high glutamate content ganglion cell with modest GABA and high AGB signals. The northwest arrow labeled with the γ symbol indicates a low glutamate, high GABA, non-starburst amacrine cell. (B) Glutamate channel, intensity mapped as medium blue for visibility (R = 0, G ≈ 0.5B, B ≈ 0–240). Southeast arrows denote high glutamate ganglion cells that also have significant GABA signals. (C) GABA and glutamate channels mapped as γE :: rb, revealing γ+ ganglion cells as magenta cells. (D) GABA channel mapped as orange for visibility (R ≈ 0–240, G ≈ 0.5R, B = 0), clearly revealing weak GABA signals in a set of high glutamate ganglion cells. Scale, 100 µm.

signals that are below the amacrine cell signal range (**Figure 2**, northwest arrow labeled with the E symbol and southeast arrows). As a control, the population of goldfish starburst amacrine cells (small circles) forms a single signature cohort with GABA levels much higher than provisionally coupled GCs. Their glutamate, GABA and kainate-activated AGB signals show that they form a distinct, monolithic, inseparable signature group that cannot be drawn from any other population, while presumably coupled and uncoupled GCs have much weaker (or no) GABA signals and diverse signatures (**Figure 2**).

## Ultrastructural Evidence of Heterocellular GC::AC Coupling

Tracer coupling suggests widespread heterocellular GC::AC coupling in the vertebrate retina, and significant correlative evidence supports that view (Völgyi et al., 2013a). But what are these coupled amacrine cells and what networks do they comprise? What is the relationship between coupling and gap junction expression? This is where retinal connectomics can offer critical insights. To direct our tracing efforts, we took advantage of the integrated CMP in connectome RC1. Most amacrine cells utilize either GABA or glycine as a neurotransmitter, which likely diffuses through gap junctions into coupled GCs. Consistent with this, many GC classes exhibit a range of GABA signals, but well below that of conventional amacrine cells (Marc and Jones, 2002). The GC layer in rabbit retinal connectome RC1 contains the somas of 20 GCs and 7 ON starburst amacrine cells (**Figure 3A**). Several of the GCs show significant levels of GABA (**Figure 3B**), suggesting they may couple with γ+ amacrine cells. We have reconstructed the gap junction patterns of two major classes of γ+ GCs, the tON DS GC GC 606 and a length of dendrite within the inner plexiform layer of the OFF alpha ganglion cell GC 9787 (not shown in **Figure 3** as the soma is not contained within the RC1 volume). We traced most of the connections of both of these cells in connectome RC1 and demonstrate that both are extensively coupled to unique sets of γ+ amacrine cells.

# GC 606

GC 606 has a large, γ+, crescent-shaped soma with a maximum diameter of 35 µm positioned within the GC layer of connectome RC1 (**Figure 3**). Its GABA signal is strong, albeit at much lower concentrations than truly GABAergic amacrine cells such as ON starburst amacrine cells. Its dendritic arbor spans the entire RC1 volume, extending beyond its boundaries in all directions, and appearing to fully stratify within sublamina b of the inner plexiform layer, just distal of the ON starburst amacrine cell dendritic stratification within the inner plexiform layer (**Figure 4**). GC 606 is indisputably an ON GC. Its excitatory synaptic input exclusively arises from ON cone BCs. GC 606 heavily couples with at least two classes of γ+ amacrine cells, including an interstitial amacrine cell (IAC) consistent with the γ+ PA1 polyaxonal cell (Famiglietti, 1992; Wright and Vaney, 2004) with which it extensively co-stratifies (**Figure 4**). Due to this coupling, GC 606 cannot be an ON alpha GC (Hu and Bloomfield, 2003) nor a classic sustained ON directionally selective (DS) GC (Hoshi et al., 2011). Moreover, there are no starburst amacrine cell inputs, further supporting that it cannot be a classic sustained ON DS GC (Hoshi et al., 2011). The soma size, arborization level, γ+ coupling and lack of starburst inputs are all consistent with the classification of GC 606 as a tON DS GC, known to be tracer coupled to at least two classes of γ+ amacrine cells, one of which is clearly an IAC

cell (SAC) 4890 (red) in the inner plexiform layer of connectome RC1. Computer generated three dimensional rendering of Viking annotations generated with VikingPlot. Small dots indicate the relative sizes (scaled by a factor of 2 for visualization) and locations of presynaptic specializations (green), PSDs (orange), gap junctions (magenta). (B) XZ projection demonstrating lamination of GC 606, SAC 4890, and IAC 9769. Scale, 100 µm.

(Ackert et al., 2006, 2009; Hoshi et al., 2011; Massey, personal communication).

The initial stage of characterizing a neuron in a connectome is defining its excitatory, inhibitory and coupling drive (**Figure 5**). The drive for GC 606 extracted by data queries from connectome RC1 is summarized in **Table 2** for 1267 validated contacts. As in previous analyses of the inner plexiform layer (Marc and Liu, 2000), synaptic drive is dominated by inhibition with ≈3 inhibitory synapses per excitatory input and 5.5 µm<sup>2</sup> of inhibitory PSD area per µm<sup>2</sup> of ribbon PSD. By measuring dendrite lengths of representative cells from Hoshi et al. (2011), we estimate that GC 606 represents only 18% of the dendritic length of a complete tON DS GC. Thus, a complete tON DS GC should receive ≈1440 excitatory ribbon synapses driving ≈54 µm<sup>2</sup> of PSD area; ≈4350 inhibitory conventional synapses driving ≈290 µm<sup>2</sup> of PSD area; and make ≈1270 gap junctions summing to 35 µm<sup>2</sup> of coupling area across its arbor (assuming no dramatic influence of eccentricity on the frequency of these interactions). However, this comprises only about 6% of the gap junction density in the inner plexiform layer (Marc et al., 2014a) and since many of the gap junctions are suboptical, tracing them by fluorescence imaging (even super-resolution methods) could be challenging.

Excitation patterns are class-specific. GC 606 receives glutamatergic excitation exclusively from ON cone BCs as can be shown by querying the RC1 database with the TulipPaths plugin (see section "Materials and Methods"): e.g., query "CB.<sup>∗</sup> , ribbon, GC ON" which returns all the cone BC ribbon synapses onto specific ON GCs from identified BCs (**Figures 6**, **7**). Of the 259 ribbon complexes that drive GC 606 in RC1, 54% originate from one class of BCs, CBb4w (**Figure 6A**), and over 99% of the input excludes CBb5 BCs, which represent the primary drivers of ON starburst amacrine cells (**Figure 7**). This is largely due to stratification. CBb5 BCs stratify just proximal to CBb4w BCs in the inner plexiform layer with only marginal overlap of their axonal arbors (Lauritzen et al., 2016). Likewise, the ON starburst amacrine cell 4890 stratifies just proximal to GC 606 (**Figure 4**), consistent with previous descriptions of tON DS GCs (Hoshi et al., 2011). As a reference, the highly coupled IAC 9769 extensively commingles with GC 606 and samples many of the same BCs with an even narrower preference spectrum dominated by CBb4w and effectively excluding CBb5 (**Figure 7**). In contrast, ON starburst amacrine cells contact a different profile with over 90% of their inputs deriving from CBb5 and CBb6, and less than 1% from CBb4w (**Figure 7**). While ON starburst amacrine cells make numerous synapses onto GC dendrites in the RC1 volume, they make no synapses onto either GC 606 or IAC 9769, nor do they receive any synapses from IAC 9769.

In addition to its extensive excitatory cone BC input, GC 606 also collects 783 conventional inhibitory chemical synapses from amacrine cells. Of those that are neurochemically identified, 33 have been mapped to definitive γ+ amacrine cells and only two to G+ amacrine cells as they traverse GABA or glycine reference slices (see Anderson et al., 2011b).

The key feature that distinguishes GC 606 is its extensive and obvious coupling with amacrine cells and IACs (**Figure 8**). The morphology of retinal gap junctions is characteristic of vertebrate CNS, yielding multilaminar profiles at 0.27 nm/pixel resolution with spacing identical to those reported by Marc et al. (1988) using ≈0.1 nm resolution on film. In parsing the GC arbor contained within RC1, it is clear that GC 606 makes abundant small gap junctions with amacrine cell-like processes (**Table 2**). Of the 228 gap junctions, 61 have been successfully traced to specific source amacrine cells or IACs. The mean gap junction diameters for the entire cohort (181 ± 56 nm) are not significantly different from those of the identified amacrine cell subset (paired homoscedastic T-test, p = 0.43, dof = 284). The diameter range is 72–357 nm, and many gap junctions are thus sub-optical. All but two of the GC gap junctions in the entire volume RC1 are associated with amacrine cell processes and every GC::AC gap junction that is associated with a complete soma or traverses a reference slice arises from a GABAergic amacrine cell. **Figure 9** illustrates the arbor of GC 606 and its overlap with coupled partner IAC 9769 (**Figure 9A**) and an additional set of seven coupled amacrine cells (**Figure 9B**). Key locations where representative gap junctions are formed are marked as A1, A2, B1, B2, etc. and displayed in **Figure 10**.

While the arbor of IAC 9769 coarsely intertwines with GC 606 at several loci, fasciculation doesn't correlate with the occurrence of gap junctions, which typically appear at brief crossing points where the processes align for less than a few µm and even then gap junctions do not occur along the apparent alignment

TABLE 2 | Contacts of GC 606.


(**Figure 9A**). From a TEM perspective, gap junctions occur at loci where gaps in suboptical glial processes expose the target, similar to axonal ribbons in BCs (Lauritzen et al., 2012). IACs are not the only γ+ neurons that couple with GC 606. A set of conventional amacrine cells driven by CBb BCs are also coupled to GC 606 (**Figure 9B**). While their reconstructed fields are too limited to classify them all, they mostly appear to be wide-field (wf) γ+ amacrine cells, and there may be two or three classes that couple to GC 606. Representative validated gap junctions from IAC 9769 and the other γ+ amacrine cells are shown in **Figure 10**. At high resolution, it is clear that most gap junctions are not fasciculations but rather crossings (**Figure 10** Column 1). The resolution of RC1 (2.18 nm/pixel) is sufficient to reliably detect gap junctions and measure their areas (**Figure 10** Column 2) but is not adequate for complete validation as occasional adherens junctions can mimic gap junctions in oblique view (Marc et al., 2014b). High resolution (0.27 nm/pixel) reimaging with goniometric tilt allows validation of gap junctions by visualizing their characteristic multilaminar density profiles (**Figure 10** Columns 3, 4 and inset). Finally, all of these coupled

FIGURE 6 | (A,B) Are combined VikingPlot and VikingView renderings. GC 606 and its bipolar cell input. (A) The dominant synaptic ribbon drive (58%) arises from a single class, CBb4w, a coupled homocellular network of ON cone bipolar cells. Each cell is colored individually. (B) The entire CBb input cohort to GC 606. Scale, 100 µm.

neurons are GABAergic (**Figure 10** Columns 5, 6). Of course it is not possible to reimage every structure in every grid, but of the more than 2000 partner-identified gap junctions tagged in RC1 at 2.18 nm/pixel resolution, ≈20 have proven to be mistaken adherens junctions (<1% error) through tilt series recapture.

The real advantage of TEM connectomics database analysis is that we can take additional network hops and ask what the roles of the coupled interneurons might be. Every cell that is coupled to GC 606 is exported as a <sup>∗</sup> .tlp format and its embedding network

displayed in the Tulip framework<sup>9</sup> . All the amacrine cells coupled to GC 606 receive excitatory drive from CBb3, CBb3n, CBb4, and/or CBb4w ON cone BCs but not from the CBb5 and CBb6 cells that drive ON starburst amacrine cells and sustained ON and transient ON-OFF DS GCs. Thus, all are ON γ+ amacrine cells with matched input cone BC drive to that of GC 606.

Many ON γ+ amacrine cells are predominantly feedback amacrine cells that target ON cone BCs. Consistent with this, the density of feedback synapses in the ON cone BC networks in the entire connectome RC1 appears ≈3:1 higher than feedforward synapses: 2359 feedback synapses from amacrine cells onto BCs, 336 feedforward synapses by amacrine cells onto GCs and 564 feedforward synapses by amacrine cells onto other amacrine cells. This lumped analysis masks the exceptional specificity of various well-known cells. For example, rod BC-driven A<sup>I</sup> amacrine cells are exclusively feedback amacrine cells, and the cohort of A<sup>I</sup> amacrine cells in RC1 make 837 feedback synapses onto BCs and 0 feedforward synapses to either GCs or other amacrine cells. In contrast, the specific cohort of ON γ+ amacrine cells coupled to GC 606 also shows direct feedforward to GCs other than GC 606 with morphologies and circuities inconsistent with the tON DS GC classification. For example, IAC 9769 has a strongly reversed bias (>10:1 feedforward:feedback), targeting 38 amacrine cells and 13 GCs but only 3 cone BCs (**Figure 11**).

Feedforward does not send inputs recursively into the upstream network like feedback does, allowing for strong channel isolation even if the interneuron is involved in feedback. An excellent example is ON γ+ AC 598 (**Figure 9B**) which engages in both feedforward and feedback, transferring sign-conserving coupled signals from GC 606 via sign-inverting GABA synapses to another ON GC (**Figure 12**).

<sup>9</sup> tulip.labri.fr

IAC 9769 is visible as a unique dense line over the apposed membranes of the two cells (bracketed by arrowheads). This is the basic identification schema for identifying gap junctions in the RC1 volume at its native 2.18 nm/pixel. Note that the gap junction can be "zoomed" to subpixel image levels in practice for annotating it (Anderson et al., 2011a). (B) TEM reimaging of the same gap junction and ribbon complex visualized at high resolution (0.27 nm/pixel) and goniometrically tilted 5◦ to optimize the multilaminar gap junction structure (inset). Reprinted by permission from (Marc et al., 2013).

The coupled set of identified γ+ IACs/ACs and additional unclassified ACs form over 200 gap junctions with GC 606 in the RC1 volume, implying that the complete cell forms over 1000 gap junctions, thereby comprising a massive coupling path between the inhibitory and excitatory networks of the retina. Cross-class inhibitory feedforward driven by coupling to GC 606 converges

FIGURE 9 | Selected sites of heterocellular coupling between inhibitory amacrine cells and GC 606. (A) Two loci of coupling (A1, A2) between IAC 9769 and GC 606 viewed as a horizontal field. Lower image, vertical overlay. (B) Seven loci of coupling between displaced amacrine cell (DAC) 10559; γ+ amacrine cells with somas in the RC1 volume (YACs) 5481, 5442, 5481, and 598; and wide-field γ+ amacrine cell (wfAC) processes 55403 and 55517 arising from somas outside the volume. Horizontal and vertical overlays. High resolution analyses of these loci are shown in Figure 10.

on pure ON GCs (ID 7594, 15796) and ON-OFF GCs (ID 5107, 6857). ON GC 7594 is also γ+ (**Figure 3**), albeit at lower levels than GC 606, but none of the GC 606-coupled amacrine cells

appear to couple with GC 7594. ON-OFF GC 5107 is uncoupled and γ−, while ON GC 15796 is very weakly γ+ and GC 6857 is strongly γ+. Thus, this feedforward inhibition does not appear to discriminate GC classes. We can mathematically summarize this chain as: GC1 :: AC > <sup>i</sup>GC2 (where class 1 6= class 2, i.e., they are disjoint sets). Other GCs receiving feedforward input in connectome RC1 are too incomplete to classify as they arise from outside the volume and it is impossible to connect branches to exclude mixed polarity inputs. Those with pure OFF inputs (OFF GCs) remain a possibility. For example, GC 606-coupled γ+ AC 5451 is pre-synaptic to GC 28950, an unbranched process traversing the OFF layer with only OFF cone bipolar inputs. If we use the rough scaling for size obtained in **Table 2**, a target GC could receive at least 200 inhibitory synapses via a single amacrine or axonal cell, driven by a coupled GC of a different class. This must be a vast underestimate, since we cannot trace the majority of the coupled processes that arise from outside the volume.

Finally, we have found no proven homocellular gap junctions between GCs. This is consistent with findings in mouse retina that homocellular coupling is always in-class, never cross-class (Völgyi et al., 2009, 2013b; Pan et al., 2010). There are 2 candidate junctions out of many thousands of gap junctions in the RC1 connectome, but we cannot validate the processes as GCs. The lack of homocellular coupling in connectome RC1 does not mean it does not exist in rabbit, since the RC1 volume is too small to ensure discovery of coupling between the small overlap zones of GCs of the same class.

#### GC 9787

Among the full cohort of GCs, OFF alpha GCs in the rabbit retina are characterized by a number of key features. In peripheral retina (rabbit volume RC1) they are among the largest of retinal GCs with very large, simple dendrites of 1–2 µm diameter, dendritic arbors of ≈0.5–0.9 mm, somas approaching 30 µm in diameter and extensive heterocellular coupling to amacrine cells (Xin and Bloomfield, 1997; Marc and Jones, 2002; Peichl et al., 2004). The somas can protrude deeply into the inner plexiform layer and insert large dendrites into the OFF layer of the inner plexiform layer. Additionally, they receive extensive input from both OFF (CBa) cone BCs and AII ACs (Kolb and Famiglietti, 1975; Marc et al., 2014a). None of the GCs with somas positioned in the RC1 volume fit this profile. Due to the sparse but uniform coverage of OFF alpha GC dendrites, we presumed that the largest crossing dendrite of the OFF layer in volume RC1 was the most probable candidate to be from an OFF alpha GC: GC 9787 (**Figure 13**). GC 9787 is a large, 1.5 µm diameter process traversing the proximal half of the OFF stratum, while reference ON-OFF GCs, e.g., GC 5107 have their dendrites and input OFF BC terminals in the most distal portion of the inner plexiform layer. Because the dendrite exhibits a single branch along the entire stretch of its crossing volume RC1, it likely represents a cell with a much larger field than nearby bistratified diving GC (Lauritzen et al., 2012) and even tON DS GCs, and clearly excludes classification as a classical X-type sustained GCs and a range of W-type cells, even those that are coupled. Beyond size, four features suggest that this single large dendrite crossing the volume arises from an OFF alpha GC. First, it collects inputs only from a subset of OFF cone BCs (mostly CBa2), especially at multi-ribbon, large PSD sites (**Supplementary Figure S1**), but not CBa1 and CBa1w BCs and axonal ribbons of ON cone BCs in the OFF layer. In contrast, dendrites of bistratified diving GCs traverse the OFF layer collecting OFF-layer axonal ribbon inputs from ON cone BCs but never inputs from the resident OFF cone BCs (Lauritzen et al., 2012). Interestingly, GC 9787 appears to form large PSDs (up to 600 nm diameter) only at pre-synaptic ribbon sites (**Supplementary Figure S1**) and never at conventional (ribbonless) BC pre-synapse sites, which are, in fact, quite common in the OFF layer and formed by the same BCs onto different targets (Anderson et al., 2011b; Marc et al., 2013). For example, bistratified ON-OFF GC 8575 collects two conventional ribbonless OFF BC synapses for every OFF ribbon it contacts. Second, GC 9787 receives conventional inhibitory chemical synapses from every AII AC it encounters, six cells in all across the volume (**Supplementary Figure S2**). Third, the process traverses GABA-labeled slice z184 in the RC1 volume multiple times and is clearly γ+ (**Figure 14**), making it an excellent candidate for heterocellular coupling with γ+ amacrine cells. Finally, it forms distinct gap junctions with amacrine cells (**Figure 14C**).

Except for the southeast margin of the volume, GC 9787 is a smooth, unbranched dendrite very unlike the topology of GC 606 and represents only ≈0.3 mm of length. The entire passage of GC 9787 through the OFF layer collects 13 gap junctions with an area of 9142 nm<sup>2</sup> /µm of dendrite length, which is ≈46% of the gap junction density of GC 606. The frequency and range of size of gap junctions formed by GC 9787 (≈0.37 ± 0.19 µm<sup>2</sup> ) tend to be on the larger size of gap junctions in RC1, but this sampling is not significantly different from those gap junctions formed by GC 606 (≈0.28 ± 0.18 µm<sup>2</sup> ) as assayed by either parametric (unpaired, heteroscedastic t-test; F-test) or non-parametric (Kolmogorov–Smirnov) measures. However, as any power calculation is defined by the smallest sampling group (gap junctions in GC 9787), the calculated power only reaches ≈0.3 with α = 0.05, and the false negative rate β is very high at 0.7. So, it is very possible that the gap junction sizes between GCs are significantly different, especially since the GC 606 statistics are stable (due to the very large sample) and its coefficient of variation is stable to less than 25% of a decimated sample set.

The cohort of coupled amacrine cells exclusively receive input from OFF cone BCs. Whether the class distribution of these excitatory inputs matches that of GC 9787 will have to wait for more detailed classification of the OFF cone BC cohort, but like GC 9787, these amacrine cells exclusively receive this input via ribbon-containing pre-synaptic sites. The set of GC 9787-coupled amacrine cells includes two γ+ amacrine cells: a large, γ+ monostratified OFF AC (YAC 7859, **Figure 13A**) and a long, unbranched amacrine cell process whose soma lies outside the RC1 volume. While we cannot verify that every coupled process is γ+, there is no evidence that any glycinergic amacrine cell in RC1 is coupled to either GC 9787 or GC 606. While we previously identified a single candidate glycine- and GABA-coupled GC class in the rabbit retina (Marc and Jones, 2002), we have not yet encountered a valid instance of glycinergic amacrine cell coupling to GCs in RC1.

FIGURE 10 | High-resolution analysis of coupling loci in Figure 9 imaged as: Column 1, VikingPlot renders; Column 2, RC1 native TEM at 2.18 nm/pixel; Columns 3 and 4, Goniometric reimaging at 0.27 nm/pixel; Column 5, soma or major process TEM; Column 6, GABA (γ) signal from the nearest intercalated CMP channel (Anderson et al., 2011b). (A1) Gap junction between GC 606 (cyan) and IAC 9769 (yellow). (A2) Gap junction between GC 606 (cyan) and IAC 9769 (yellow). Inset in columns 5,6 show a normalized plot of membrane density spanning the entire junction starting from the paramembranal density in GC 606, crossing the trilaminar zone and ending in IAC 9769 (ImageJ). (B1) Gap junction between GC 606 (cyan) and γ+ amacrine cell YAC 5451 (pink). (B2) Gap junction between GC 606 (cyan) and γ+ amacrine cell YAC 598 (tan). (B3) Gap junction between GC 606 (cyan) and γ+ amacrine cell YAC 5481 (yellow). (B4) Gap junction between GC 606 (cyan) and γ+ amacrine cell YAC 5542 (lavendar). (B5) Gap junction between GC 606 (cyan) and displaced γ+ amacrine cell DAC 5451 (green). Note that the lamination of the gap junction can be visualized through the inadvertent stain debris in column 4. (B6) Gap junction and adjacent conventional synapse between GC 606 (cyan) and wf γ+ amacrine cell wfAC 55403 (red). (B7) Gap junction and adjacent conventional synapse between GC 606 (cyan) and wf γ+ amacrine cell wfAC 55517 (orange).

FIGURE 11 | Graph of synaptic and gap junctional connectivity from IAC 9769 to amacrine, bipolar and ganglion cells. Tulip query showing bundled (i.e., multiple synaptic paths between loci are represented as single lines) dendrograms. Each line in the dendrogram represents a connection from IAC 9769 to another cell. (A) Annotation-free view with (C) inset marked (box). (B) Key: Red symbols, IAC, γ+ ACs and unidentified ACs; green symbols, AII and GACs (glycinergic amacrine cells); blue symbols, bipolar cells; orange symbols, ganglion cells; small red symbols at left, IAC coupled ::ACs (coupled amacrine cells); red lines and arcs, synaptic outflow from IAC 9769; blue lines and arcs, synaptic input to IAC 9769 and instances of bipolar cell; yellow lines and arcs, gap junctions. (C) Enlargement of inset in (A). GC 606 is strongly coupled to IAC 9769 (circled yellow edge, GC::AC) and other γ+ amacrine cells (yellow arc). Massive coupling networks exist among ON cone bipolar cells (yellow box).

Similar to GC 606, there is feedforward signal flow from GC 9787 via coupled OFF γ+ amacrine cells to both GC 9787 itself and other short fragments of non-alpha GC dendrites in the OFF layer. Some non-alpha dendrites are themselves γ+. At least one of these does not form gap junctions with these same amacrine cells (ID 43716), implying that they may be coupled to different sets of amacrine cells, as is the case with GC 606. However, with both the GC and amacrine cell extending processes beyond the volume boundaries of RC1, it is possible that such coupling occurs elsewhere in their arbors. Two GC processes do couple with these same amacrine cells. Both (ID 28950, 5150) are also large-caliber single- or un-branched processes and receive frequent input from AII amacrine cell lobules, not inconsistent with OFF alpha dendrites. The high overlapping coverage of adjacent OFF alpha dendritic arbors (Völgyi et al., 2005) therefore makes it impossible to rule out that these are OFF alpha dendrites from the same or other OFF alpha GCs. While a complete tabulation of the connectivity of coupled OFF amacrine cells would require over a year's worth of dedicated annotation and classification time, Tulip queries reveal that some partners such as wf γ+ AC 7859 appear to be biased toward feedforward contacts, similar to IAC 9769 in the ON system, and support cross-class inhibitory feedforward to other GC classes.

# DISCUSSION

## GABA Signatures

GABA content is a useful signature for predicting coupling in the GC layer. There have been no known GABA transporters described on any GCs, much less GCs in the adult rabbit retina (Hu et al., 1999). Unlike coupled GCs, uncoupled cells have no GABA signal but all GCs have mathematically inseparable glutamate signatures (Marc et al., 1990, 1995; Marc and Jones, 2002), regardless of GABA content (**Figure 1E**). In support of this observation, we have found no gap junctions made by any γ− GCs in RC1. GCs display GABA levels ranging from undetectable to close to bona fide amacrine cell levels (Marc and Jones, 2002), with the majority centered around 300–600 µM, 10-fold lower than typical starburst amacrine cell levels (**Figure 1F**). Given that specific GCs show extensive heterocellular coupling with markers such as Neurobiotin (MW 322), it is not surprising that a molecule several times smaller, such as GABA (MW 103), is highly mobile through gap junctions, similar to quantitative measures of glycine coupling into ON cone BCs from glycinergic AII amacrine cells (Marc et al., 2014a). Glycine is present in appreciable amounts in BCs despite the lack of a synthesizing enzyme and transporter, explicitly due to AII amacrine cell coupling. Indeed, for other work, we use glycine as an index on ON cone BCs revealing their coupling to AII amacrine cells (Marc et al., 2014a). Importantly, GCs known to show heterocellular coupling such as rabbit OFF alpha GCs (Xin and Bloomfield, 1997) and tON DS GCs (Hoshi et al., 2011) uniformly show intermediate GABA levels (Marc and Jones, 2002). Importantly, Ackert et al. (2006, 2009) and Hoshi et al. (2011) also demonstrated that an axonal cell (axon-bearing "amacrine" cell) virtually identical in dendritic morphology to our IAC 9769 was both coupled to tON DS cells and γ+ by immunocytochemistry, and that other amacrine cells with

FIGURE 12 | Coupling flow from GC 606 through γ+ amacrine cell 598 to multiple targets. (A) Tulip query bundled dendrogram plot of all the sources and targets of amacrine cell 598: GC yellow circles, ganglion cells: GAC green dots, glycinergic amacrine cells; γAC red dots, GABAerigc amacrine cells; CBb cyan dots, ON cone bipolar cells; RB magenta dot, rod bipolar cell. Line color denotes the presynaptic source. Arrows denote presynaptic source in γAC to γAC paths. Each line represents a bundle of synaptic lines. (B) Validation of GABAergic identity for AC 598. (C) A gap junction between AC 598 (red) and GC 606 (yellow) delimited by arrowheads. (D) Synapse from AC 598 (red) to GC 38810 (arrow).

reference bipolar cells. ON–OFF GC 5107 is driven by both ON cone bipolar cells (e.g., CBb3n 6120 tangerine) and OFF cone bipolar cells (CBa 165 blue) that bracket the OFF inner plexiform layer and are distal within the inner plexiform layer to the rod bipolar cell terminals (e.g., RB 11031, magenta). GC 9787is driven by a separate set of more proximal OFF cone bipolar cells (e.g., CBa 473 gray). Ellipses delimit bipolar cell axonal fields. (B) Vertical view displaying the separate strata for rod (RB), ON cone (CBb) and OFF cone (CBa) bipolar cells. CBa 473 that drives OFF alpha GC 9787 is proximal to the OFF CBa 165 and similar bipolar cells that drive GC 5107.

differing arbor patterns were also coupled. In contrast, GCs that we know are definitively not dye-coupled, such as primate midget GCs (Dacey and Brace, 1992), never show GABA coupling (Kalloniatis et al., 1996).

This sets the framework for using glutamate and GABA as markers of GC coupling in other species (**Figure 2**), since antibodies targeting small molecules have no species bias. In surveying our library of all vertebrate classes and many vertebrate orders (**Supplementary Table S2**), we find that apparent heterocellular coupling between GCs and amacrine cells is widespread with only one group failing to show evidence of coupling: Trachemys scripta elegans (formerly Genus Pseudemys), Order Testudines, Class Reptilia. As all vertebrate classes show evidence of heterocellular GC::AC coupling, this argues for such signaling as a feature of primitive retina and perhaps even of its predecessor diencephalic primordia. Indeed, extensive coupling

containing a segment of GC 9787 (cyan) flanked by AII AC lobules (yellow–green). (B) The same TEM image from slice z184 overlayed with the neighboring intercalated GABA signal showing positive colocalization. Scale 5 µm. (C) Typical gap junction between GC 9787 (cyan) and an amacrine cell (AC 85607) at a crossing, non-fasciculated junction. Scale 1 µm. Inset. High resolution image of the gap junction, showing characteristic gap occlusion. Scale 100 nm.

and regions of high cell firing synchronicity is a hallmark of early mammalian brain differentiation (Niculescu and Lohmann, 2014). In sum, these considerations argue for heterocellular GC::AC coupling as a retinal plesiomorphy (a basal feature of ancient retinas), not a synapomorphy (specialization of a clade) among select classes, and that heterocellular coupling is foundational for the retina as argued by Völgyi et al. (2013a).

# Coupling and Feedforward

But what is the functional network role of heterocellular coupling? A fundamental clue arose when certain retinal GCs and downstream neurons in brain were found to show synchronized spiking across cells (Alonso et al., 1996; Hu and Bloomfield, 2003) and that the mostly narrowly correlated retinal firing persisted after global pharmacologic synaptic suppression, leading to the argument that it was mediated by coupling (Brivanlou et al., 1998). Subsequent analyses have refined these concepts to show that correlated spiking appears to occur within sets of the same GC class, including the OFF alpha GC class, and that

one essential pathway is heterocellular coupling (Völgyi et al., 2013b; Roy et al., 2017). Our findings are consistent with this view: (1) homocellular cross-class GC coupling is non-existent in connectome RC1, (2) heterocellular coupling between GCs and multiple classes of amacrine cells is abundant and robust, and (3) in instances where multiple GC processes couple to the same amacrine cell, the ganglion processes are not obviously of different classes. No evidence emerged to show that any amacrine cells in the coupling networks for one tON DS GC and one OFF alpha GC are shared: they seem completely separate. However, we do find sparse instances of coupling among the γ+ ACs coupled to GC 606, but these short fragments are impossible to classify and may reflect homocellular coupling between amacrine cells of the same class, which has been supported, for example, for the IACs due to their robust tracer coupling (Wright and Vaney, 2004) and Neurobiotin visualization using photochromic 2-stage intensification as described in Vaney (1992).

Interstitial amacrine cells and ON γ+ amacrine cells coupled to tON DS GCs share the same profile of excitation: a bias for class CBb4w ON cone BCs and against classes CBb5 and CBb6 cells that drive starburst amacrine cells (**Figure 7**). While our analysis of OFF cone BC populations is not yet as refined as for ON cone BCs, the excitatory drive to amacrine cells coupled to OFF alpha GCs shares similar biases: toward OFF Cba2 BCs and away from CBa1 BCs. Considering the high diversity of vertebrate amacrine cell classes (Wagner and Wagner, 1988; MacNeil et al., 1999), every instance of GC::AC coupling could easily involve unique sets of amacrine cells for each coupled GC class, though such class separation may not be essential.

But amacrine cells are not simply conduits for coupling. Every amacrine cell class that we know well is either GABAergic or glycinergic. Indeed, every amacrine or axonal cell involved with heterocellular GC::AC coupling whose signature can be retrieved is GABAergic. And connectomics can resolve the targets of these coupled amacrine and axonal cells. Importantly, both ON and OFF instances of GC::AC coupling demonstrate feedforward synapses directly from coupled ACs to different classes of GCs: cross-class inhibition. As schematized in **Figure 15**, heterocellular coupling allows an active GC to directly inhibit its neighbors: GC1::ON AC > <sup>i</sup> GC2; where ><sup>i</sup> denotes sign-inverting signaling, :: denotes coupling and classes GC1 and GC2 are disjoint. The essential feature is that inhibitory postsynaptic currents (IPSCs) should be generated in a halo of different GC classes closely synchronized with the spikes of a source GC. If these IPSCs were strong enough to suppress some incidentally coincident spikes in target GCs, this could create an improved signal-to-noise ratio (SNR) at the CNS downstream targets of the source GC compared to a parallel channel (**Figure 15**). Certain GCs (e.g., ON–OFF DS GCs) show Na-dependent dendritic spiking (Oesch et al., 2005; Schachter et al., 2010), potent spike veto by inhibitory processes (Sivyer and Williams, 2013) and postsynaptic current integration (Brombas et al., 2017). This argues that dendritic inhibition in GCs can be strong enough to suppress dendritic spiking. The bleed-through of excitation from the tON DS GC into a set of GABAergic neurons that target different GCs means that such heterocellular coupling likely has the ability to suppress activity in nearby disjoint populations.

While the potential for precise timing of both synchronized spikes and feedforward inhibition is clear, it is also certain that many wf γ+ amacrine cells (e.g., **Figure 12**) provide feedback to cone BCs of matched polarity: ON AC > <sup>i</sup> ON CBb and OFF AC > <sup>i</sup> OFF CBa. This provides a much broader fan-out of targets for the GC::AC inhibitory couple, amplified explicitly by the positive gain of BC ribbon synapses (e.g., Lauritzen et al., 2016).

FIGURE 15 | Signal flow through the tON DS GC :: γ+ AC network. Key in inset. (1) ON cone bipolar cell signals are collected by all cell classes at AMPARs or AMPARs + NMDARs. GC :: AC gap junctions connect networks of (2) γ+ IACs and wide-field (wf) γ+ ACs. IACs are predominantly feedforward, driving sets of ganglion cells including (3) the coupled tON DS GC, (4) local dendrites from GCs outside this coupled set, and (5) distant instances of tON DS GCs in the far surround via their axons. IACs also engage in (6) nested feedback with wf γ+ ACs, which are themselves mixed feedforward (not shown) and (7) feedback inhibitory neurons. This model may also support a directional bias for tON DS GCs with the preferred direction arising from the regions driven by the axonal field of distant IACs.

Heterocellular coupling between spiking projection neurons and local inhibitory neurons may be more widespread than appreciated. Like retina, olfactory bulb generates synchronized oscillatory excitation/inhibition interactions that are enhanced by Cx36-mediated coupling (Pouille et al., 2017) though the initial mechanism was modeled as homocellular coupling of mitral cell (MC) pools. However, detailed analysis of inhibitory intraglomerular networks provide strong evidence for heterocellular coupling between mitral cells and specific short axon GABAergic cells in olfactory glomeruli and that the coupling, at least, is part of the mechanism that truncates events to permit more precise excitation/inhibition coordination in mitral cells (Liu et al., 2016). This may be a common mechanism in many "transient" neurons as it is consistent with feedforward onto the coupled source in both instances of GCs, albeit with completely different inhibitory networks. Similarly, physiological evidence supports an analogous network for timing control in olfactory bulb. While little is known of the cell class distinctions among neighboring mitral cells in olfactory bulb, there is strong evidence for multiple projection classes, intrabulb short-range excitatory projections, and inhibitory classes including different classes of GABAergic short axon cells (Nagayama et al., 2014). We would predict that specific classes of short axon neurons (SANs) make heterocellular gap junctions with specific mitral cells and inhibit neighboring mitral cells where MC1 and MC2 are disjoint sets: M1 :: SAN > <sup>i</sup> M2.

Finally, coupling networks involving inhibitory neurons can take on very complex frequency-dependent operations, such as Golgi neurons in cerebellum (Vervaeke et al., 2010, 2012; Pereda et al., 2013). In a similar fashion, it is plausible that the IAC might not have effective dendritic spiking and are more passive cables, like cerebellar Golgi interneurons, but the high density of GC::AC coupling acts as an excitation repeater. Further, such networks could either enhance synchrony or desynchronize in different excitation modes (Vervaeke et al., 2010).

#### Arbor Size and Resolution

There is a major caveat arising from the conflicting demands of connectomics coverage and resolution. Once captured, one can downsample but never upsample, one can mine an area but never expand. Dedicating more bits to one mode steals from the other. Unlike small-field BCs and glycinergic amacrine cells, GCs and GABAergic neurons can have arbors much larger than a connectome. Ackert et al. (2009) showed the axonal cells coupled to ON DS GCs had fine extensions of their terminal dendrites that ascended into the OFF layer and co-stratified with the OFF ChAT+ starburst amacrine cells. As IAC 9769 has long straight dendrites that exit the full volume perimeter, such unusual morphologic features cannot be excluded. Thus, there appears at least two crossover paths between the IAC and the OFF layer. In addition to the apparent cholinergic layer ramification, it is clear (via Tulip queries) that crossover glycinergic amacrine cells driven by OFF cone BCs also synapse on IAC 9769. The data of Ackert et al. (2009) do not reveal a glycinergic path, though it clearly exists. Their data support the OFF starburst amacrine cell path: OFF CBa > OFF SAC > IAC :: ON DS GC; where > denotes sign-conserving synapses and :: denotes coupling. It is remarkable that, despite abundant opportunity (**Figure 4**), the ON γ+ IAC completely rejects interaction with the ON starburst amacrine cell arbors in favor of the OFF arbors.

# Direction of Motion

The coupled tON DS GC is a largely separate stream of directional signaling with little apparent engagement with the ON-OFF DS cohort (Ackert et al., 2006; Hoshi et al., 2011), but does contribute to the classic three-lobed orientation distribution of ON DS GCs reported by Oyster and Barlow (1967). Consistent with this model, we find not only complete synaptic separation of tON DS GCs and the starburst amacrine cell network, but also nearly total separation of each other's BC input profile. Nevertheless, like other DS cells, directional signaling by tON DS GCs is dependent on GABAergic inhibition and is suppressed after global GABA blockade (Ackert et al., 2006, 2009). Directional selectivity in visual cortex has been thought to be driven by asymmetries in excitation, although differing spatial distributions of excitation and inhibition clearly play a role (Li et al., 2017). IACs, as axonal cells, offer a built-in simple asymmetry that could be maximized for low velocity directional motion if: (1) their axons behave as classical axons and arborize into predominantly presynaptic terminals; (2) the axons do not form gap junctions; and (3) the axons target tON DS GCs. We simply don't have information on the latter in connectome RC1, but it is important to consider two quantitative points. First, a complete tON DS GC likely receives over 4,000 inhibitory synapses and the bulk of those will be GABAergic (15-fold more prevalent than glycinergic synapses onto GC 606). The fact that coupled amacrine cells make up a small fraction of that inhibitory input via feedforward simply means that the bulk of inhibition is driven by sets of wide field amacrine cells or IAC axons arising from outside the volume, displaced from the centroid of the GC 606 arbor. Importantly, prior work had shown these IACs (also known as axon-bearing amacrine cells) had long sparse axons (Ackert et al., 2006, 2009; Hoshi et al., 2011) but some of these are incomplete, as a more complex terminal arbor was demonstrated in Massey (2008). A more complete description of bona fide IACs in primate by Dacey (1989) describe each IAC as being surrounded by a sparse halo of axon terminals roughly 10x the diameter of the dendritic arbor and yielding perhaps over 100 fold greater coverage. Similarly, IAC 9769 in RC1 is identical to the PA1 polyaxonal cell in rabbit meticulously described by both Famiglietti (1992) and Wright and Vaney (2004). In any case, there is ample additional space in the tabulation of synapses to accommodate sparse inhibitory cells with very high coverage factors, in addition to IAC axons. Wright and Vaney (2004) also show that the net density (length) of axonal processes is ≈10× higher than dendritic density. If this translates to synaptic density and the axon has the same target preferences as the IAC dendrites, it is very likely that a large fraction of inhibition targeting tON DS GCs could arise from the IAC or PA1 polyaxonal cell. We have not shown that the non-IAC processes coupled to GC 606 correspond to the diffuse multistratified cell previously described (e.g., Hoshi et al., 2011), but presuming we are selecting for parts of their arbors, this cell may be even better suited than the IAC

for mediating cross-class inhibition. The second point is that a gap junction on GC 606 is never more than a micron away from a BC ribbon input, so the shunting path for any dendritic spikes is very short. This lays the framework, via IAC axons or wf amacrine cells, to provide narrowly shaped and time-locked, delayed feedforward IPSCs to other tON DS GC instances in the coupled neighborhood.

#### CONCLUSION

Physiological and tracer studies have firmly established heterocellular coupling as a norm in the mammalian retina. By combining small molecule markers and connectomics we provide some additional insights. First, heterocellular GC::AC coupling is likely a plesiomorphy and not a synapomorphy. Second, in the instances of GC::AC coupling we know well in the mammalian retina, one involving tON directionally selective GCs and the other engaging transient OFF alpha GCs, the coupled GABAergic amacrine and axonal cells clearly inhibit many neighboring cells, including feedforward inhibition onto neighboring GCs of different classes, outside the coupled set. Thus, an activated GC may inhibit neighboring GCs of different classes in a time-locked fashion, potentially erasing coincident dendritic spikes across GC classes. If we can now begin to tabulate and explore the detailed distributions of inhibition relative to the sites of coupling, we may uncover spatial asymmetries that convert to temporal delays necessary for encoding direction in this unique cohort of ganglion cells.

#### AUTHOR CONTRIBUTIONS

RM wrote, edited, created figures, annotated and analyzed content for this manuscript. CS annotated and analyzed the data, created figures, and edited the manuscript. RP annotated and analyzed the data and edited the manuscript. DE extensively annotated. JA created connectome volume builds, maintained and edited annotation and image data in the database. BJ directs the research lab and science, edited the manuscript, participated in data generation, annotation, analysis, read and approved the submitted version.

#### REFERENCES


#### FUNDING

This work was funded through the National Institutes of Health, R01EY015128, P30EY014800, T32EY024234, and EY02576 as well as an Unrestricted Grant from Research to Prevent Blindness, Inc., New York, NY, to the Department of Ophthalmology & Visual Sciences, University of Utah.

#### ACKNOWLEDGMENTS

We would like to thank and acknowledge the efforts of Dr. Shawn Mikula who invited us to participate in this special issue in his role as one of the editors of this special issue. Shawn died in July and will be missed by his family, friends and the wider neuroscience connectomics community.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fncir. 2018.00090/full#supplementary-material

FIGURE S1 | Multi-ribbon OFF cone bipolar cell inputs to GC 9787. Five serial TEM sections (z168–z172) are coded for GC 9787 (cyan) and CBa 165 (green). Over a span of 280–300 nm, at least 4 bipolar cell synaptic ribbons (R) dock presynaptically across from a single large postsynaptic density (bracketed by arrowheads). Arrows denote direction of synaptic flow (pre → post). Scale 1000 nm.

FIGURE S2 | GC 9787 dendrites (cyan) collect multiple synaptic inputs from glycinergic AII amacrine cell distal lobular appendages (green) across the volume. (A) Conventional synapses from GAC AII 7113 onto both GC 9787 and CBa 35696 (tan) which is presynaptic to GC 9787 at two other sites. (B) Convergent signaling from γ+ amacrine cells (pink γ+ AC), GAC AII 284, and a CBa bipolar cell (tan) onto GC 9787. GAC AII 7157 makes synapses onto 9787 in another section (not shown) but is also presynaptic to the CBa bipolar cell. (C) Single synapse from lobule GAC AII 8032 onto GC 9787. (D) Classical multiple presynaptic densities associated with a single GAC AII synapse. Scales 1000 nm.

TABLE S1 | Examples of retinal cell classes, intermediate groups and superclasses.

TABLE S2 | GABA immunocytochemistry species list.

TABLE S3 | Log10 relative ligand required to block tissue binding.

and summarization of large connectomics datasets. J. Microsc. 241, 13–28. doi: 10.1111/j.1365-2818.2010.03402.x




vertebrate electrical synapse. Neuron 79, 957–969. doi: 10.1016/j.neuron.2013. 06.037



Zou, J., Salarian, M., Chen, Y., Zhuo, Y., Brown, N. E., Hepler, J. R., et al. (2017). Direct visualization of interaction between calmodulin and connexin45. Biochem. J. 474, 4035–4051. doi: 10.1042/bcj20170426

**Conflict of Interest Statement:** RM is a principal of Signature Immunologics Inc., the source of some of the antibodies used for this research.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Marc, Sigulinsky, Pfeiffer, Emrich, Anderson and Jones. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Verifying, Challenging, and Discovering New Synapses Among Fully EM-Reconstructed Neurons in the Leech Ganglion

Jason E. Pipkin<sup>1</sup> \*, Eric Allen Bushong<sup>2</sup> , Mark H. Ellisman2,3 and William B. Kristan Jr. <sup>1</sup>

<sup>1</sup>Division of Biological Sciences, University of California, San Diego, La Jolla, CA, United States, <sup>2</sup>National Center for Microscopy and Imaging Research, University of California, San Diego, La Jolla, CA, United States, <sup>3</sup>Department of Neuroscience, University of California, San Diego, La Jolla, CA, United States

Neural circuits underpin the production of animal behavior, largely based upon the precise pattern of synaptic connectivity among the neurons involved. For large numbers of neurons, determining such "connectomes" by direct physiological means is difficult, as physiological accessibility is ultimately required to verify and characterize the function of synapses. We collected a volume of images spanning an entire ganglion of the juvenile leech nervous system via serial blockface electron microscopy (SBEM). We validated this approach by reconstructing a well-characterized circuit of motor neurons involved in the swimming behavior of the leech by locating the synapses among them. We confirm that there are multiple synaptic contacts between connected pairs of neurons in the leech, and that these synapses are widely distributed across the region of neuropil in which the neurons' arbors overlap. We verified the anatomical existence of connections that had been described physiologically among longitudinal muscle motor neurons. We also found that some physiological connections were not present anatomically. We then drew upon the SBEM dataset to design additional physiological experiments. We reconstructed an uncharacterized neuron and one of its presynaptic partners identified from the SBEM dataset. We subsequently interrogated this cell pair via intracellular electrophysiology in an adult ganglion and found that the anatomicallydiscovered synapse was also functional physiologically. Our findings demonstrate the value of combining a connectomics approach with electrophysiology in the leech nervous system.

Keywords: serial block face scanning electron microscopy, invertebrate neurobiology, leech, EM reconstruction, synapse

#### SIGNIFICANCE STATEMENT

The function of any nervous system depends on the arrangement of its component neurons into circuits. Determining this precise pattern requires an account of which neurons are linked by synapses, and where. Here, we use serial electron microscopy to confirm, challenge, and discover synapses in the neuropil of one ganglion from a juvenile leech. Relying on the homology of the ganglion from animal to animal, we demonstrate that we can identify synapses we knew existed from previous physiological work, and that we can confirm a new anatomically-discovered synapse by subsequently recording from the same neurons in a different animal. Here, we show how analyses

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Marcel Oberlaender, CAESAR Zentrum für fortgeschrittene europäische Wissenschaft und Forschung, Germany Antonia Marin-Burgin, CONICET Instituto de Investigación en Biomedicina de Buenos Aires (IBioBA), Argentina

#### \*Correspondence:

Jason E. Pipkin jason.e.pipkin@gmail.com

Received: 16 July 2018 Accepted: 18 October 2018 Published: 14 November 2018

#### Citation:

Pipkin JE, Bushong EA, Ellisman MH and Kristan WB Jr (2018) Verifying, Challenging, and Discovering New Synapses Among Fully EM-Reconstructed Neurons in the Leech Ganglion. Front. Neuroanat. 12:95. doi: 10.3389/fnana.2018.00095 of anatomical detail and physiologically determined interactions complementarily yield insight into how neural circuits produce behavior.

# INTRODUCTION

The behavioral repertoire of a given neural circuit is constrained in part by the connectivity pattern among its constituent neurons. To understand how circuits produce behavior, it is therefore necessary to know which neurons make synapses onto which other neurons. Deciphering this connectivity by means of exhaustive electrophysiology is possible in preparations involving relatively few neurons, as in the ∼25–30 neuron crustacean stomatogastric ganglion (Marder and Bucher, 2007). As the number of neurons increases however, an imaging-based anatomical approach is required to capture the full connectivity of all neurons within a given volume of tissue (Denk et al., 2012). The resolution necessary to reconstruct neurons and identify synapses among them is provided by serial electron microscopy. For instance, the C. elegans hermaphrodite nervous system was reconstructed from a set of overlapping serial electron micrographs from \_\_\_\_ individual animals, resulting in the first whole-organism ''connectome'' (White et al., 1986). Yet the time-consuming nature of this approach has, until recently, dissuaded attempts to apply serial EM to larger volumes of tissue. In the past decade the development of serial blockface scanning electron microscopy (SBEM; Denk and Horstmann, 2004) and refinement of serial section transmission electron microscopy (ssTEM, e.g., Bock et al., 2011; Kasthuri et al., 2015; Ohyama et al., 2015) has dramatically reduced the image acquisition time for large volumes of neural tissue. The resulting datasets have been used to provide insight into both existing and novel circuits. Among others, these results include discovering new features of a known retinal circuit (Briggman et al., 2011), the circuitry of the tail of male C. elegans (Jarrell et al., 2012), a new type of retinal bipolar cell (Helmstaedter et al., 2013), the complete visual circuitry of a polychaete worm (Randel et al., 2014), the elucidation of circuits responsible for turning behavior (Ohyama et al., 2015) in larval Drosophila as well as olfactory processing in both the larval (Berck et al., 2016; Eichler et al., 2017) and adult Drosophila (Takemura et al., 2017a; Tobin et al., 2017), the reconstruction of visual circuits in larval (Larderet et al., 2017) and adult Drosophila (Takemura et al., 2013, 2017b), and the full connectome of the central nervous system of the larval tunicate Ciona intestinalis (Ryan et al., 2016).

To link the connectivity information gleaned from SBEM or ssTEM datasets to models of circuit function, the anatomicallypredicted synapses must be testable physiologically. In C. elegans, the connectome has been essential for guiding cell manipulation, ablation, and functional imaging experiments (Bargmann and Marder, 2013). Similarly, calcium confirmed the existence of synapses identified by EM reconstructions of Drosophila circuitry (Ohyama et al., 2015; Takemura et al., 2017b). These applications rely on the ability to identify the same neurons from preparation to preparation—an advantage afforded by many invertebrate systems.

The utility of an anatomically-defined connectivity map is enhanced by the amenability of the preparation to electrophysiological techniques. A connectome specifies which neurons are synaptically connected, but subsequent physiological inquiry is needed to determine whether those connections are inhibitory or excitatory and how strongly a given presynaptic neuron influences its postsynaptic partners. The leech ganglion is particularly advantageous for this purpose as the positioning and size of its neurons render them accessible to sharp electrode intracellular electrophysiology in a way that neurons of C. elegans or Drosophila are not. In the medicinal leech, Hirudo verbana, behaviors are produced by a chain of homologous ganglia each containing approximately 400 neurons. To date, most of the work uncovering the circuitry responsible for given behaviors in the leech has relied on intracellular electrophysiology (e.g., Nicholls and Baylor, 1968; Ort et al., 1974) or optical monitoring of voltage-sensitive dyes (e.g., Briggman et al., 2005). These experiments have resulted in several well-characterized synapses and circuits (e.g., Ort et al., 1974; Stent et al., 1978; Lockery and Kristan, 1990a,b; Kristan et al., 2005), yet many neurons and their connectivity in the leech ganglion remain completely or partly uncharacterized (Wagenaar, 2015).

We applied SBEM to leech tissue to study known circuits and discover new synaptic connections. We previously reported on the distribution and pattern of synaptic sites in two SBEM datasets: one small volume of mature leech neuropil, and one entire ganglion taken from the smaller yet behaviorally-mature juvenile leech (Pipkin et al., 2016). Herein, we report on the connectivity uncovered within the juvenile ganglion dataset. To validate the approach, we first analyze the connections of well-characterized motor neurons that innervate the longitudinal muscles and participate in the swimming behavior. Second, we use the dataset to identify a previously uncharacterized synaptic relationship and subsequently verify it physiologically. Our results demonstrate the utility and potential of EM-based circuit reconstruction in the medicinal leech by linking anatomy and electrophysiology at the level of individual cell pairs.

#### MATERIALS AND METHODS

#### Animals

We used both adult and juvenile medicinal leeches (Hirudo verbana). Adult leeches were obtained from Niagara Leeches (Niagara Falls, NY, USA) and housed in aquaria on 12 h daily light/dark cycle at 15–16◦C. Juvenile leeches were obtained by harvesting cocoons produced by a breeding colony of adult leeches maintained in our laboratory. Leeches were allowed to mature within the cocoons at room temperature (RT) and collected once they had emerged. We then waited 2 weeks to ensure full development prior to dissection. We confirmed that the juveniles lacked any embryonic features using established staging criteria (Reynolds et al., 1998). For the juvenile sample, we stained and embedded several ganglia but eventually imaged only ganglion 11. The methodological description of this sample's preparation (below) and results of some analyses have been published previously (Pipkin et al., 2016).

## Sample Preparation for Electron Microscopy

We anesthetized the juvenile leech in ice-cold leech saline (4◦C) containing 115 mM NaCl, 4 mM KCl, 1.8 mM CaCl2, 2 mM MgCl2, 10 mM HEPES buffer (Nicholls and Purves, 1970). Midbody ganglia were then dissected from the nerve cord and pinned to the bottom of a Sylgard-coated dish. The ganglia were then fixed for 2 h at RT in 2% PFA, 2.5% glutaraldehyde, and 0.1 M phosphate buffer. After fixation the ganglia were rinsed in 0.1 M phosphate buffer and incubated in 2% OsO4/1.5% potassium ferrocyanide. For this step, the samples were microwaved in a scientific microwave (Pelco 3440 MAX) three times at 800 W with a duty cycle of 40 s on and 40 s off at a measured temperature of 35◦C and subsequently left to sit at RT for 30 min. Samples were then washed in ddH2O and microwaved three times at 800 W with a duty cycle of 2 min on and 2 min off at 30◦C. We found that this and subsequent brief microwave incubations facilitated staining penetration to the center of our samples and was necessary to gain sufficient image contrast. Samples were then incubated in 1% thiocarbohydrazide (Electron Microscopy Sciences) and microwaved three times at 800 W with a 40 s on and 40 s off duty cycle at 30◦C and subsequently left to incubate for 15 min RT. The samples were then washed again with the same microwave incubation as described earlier. Next, the samples were incubated in 2% aqueous OsO<sup>4</sup> and microwaved three times at 800 W with a 40 s on and 40 s off duty cycle at 30◦C and then incubated at RT for 1 h. After washing, the samples were then left in 1% uranyl acetate overnight at 4◦C. The next day, samples were incubated in a lead aspartate solution prepared by dissolving 0.066 gm of lead nitrate into 10 ml of 0.03 M aspartic acid with the pH subsequently adjusted to 5.5 using 1 N KOH. This incubation took place in a 60◦C oven for 30 min. The samples were then washed and dehydrated through a series of ethanol solutions (50%, 70%, 90%, 100%, 100%, 10 min each) at RT and incubated in acetone. Following this, samples were infiltrated with epoxy resin by first incubating them for 2 h at RT in a solution of 50% acetone and 50% Durcupan and then overnight in 100% Durcupan. The next day, samples were transferred to a freshly prepared 100% Durcupan solution and incubated at RT for 2 h. Samples were then incubated within a 60◦C oven for 3 days. Durcupan Araldite resin was made by mixing 11.4 g of component A, 10 g of component B, 0.3 g of component C and 0.1 g of component D.

#### Imaging

The resin-embedded ganglia were preserved within epoxy blocks trimmed until tissue was barely exposed. For the juvenile sample, the edges of the block were trimmed until very near to the external capsule of the ganglion to reduce charging in the outer image tiles that contained both tissue and empty plastic. Blocks were mounted onto aluminum pins to which they were adhered with conductive silver paint. The pin and block were then sputter coated with a thin layer of gold and palladium to further enhance conductivity.

We imaged ganglion 11 from a juvenile animal with a Zeiss MERLIN SEM equipped with a Gatan 3VIEW SBEM system. We collected montages of 8,000 × 8,000 raster tiles at 5.7 nm pixel size. We oriented the sample so that it was imaged from the dorsal surface to the ventral surface with sectioning occurring perpendicular to the dorsal-ventral axis. Montage size thus varied from 1 × 1 to 5 × 5 tiles depending on the area of tissue that was exposed to the surface of the block. We sectioned the block 2,203 times at 50, 100, or 150 nm thicknesses for a total z-distance of 138 µm. The 100 nm and 150 nm sections were taken in regions containing only cell bodies (at the top and bottom of the overall volume) as there are very few fine neuronal processes to trace here and thus imaging time could be reduced. Similarly, we varied dwell time throughout acquisition along a range of 0.8-µs to 1.5-µs with higher dwell times used in neuropil-containing sections. During the juvenile ganglion acquisition, an unexpected and gradual reduction of contrast occurred due to the premature degradation of the filament in the electron gun. As imaging proceeded from the dorsal surface towards the ventral, we therefore focused most of our analysis and reconstruction on cells whose arbors tended to fall within the dorsal half of the ganglion. Where processes from these cells entered the low-contrast region of the volume, we were likely to have missed some fine branches and any associated synapses in this area.

# Reconstruction and Annotation

In the juvenile ganglion volume, montages and sections were aligned in the TrakEM2 (fiji.sc/TrakEM2, RRID: SCR\_008954, Cardona et al., 2012). Subsequent tracing and annotation was also performed in TrakEM2. In this volume, we largely reconstructed arbors via skeletonization rather than full segmentation via membrane tracing. Locations of synaptic inputs and outputs were denoted by placing ball objects as markers on the skeletons.

All tracing, segmentation, and analysis was performed by JP. To reduce errors, the arbors of the motor neurons discussed in **Table 1** and **Figures 1**, **2** were reviewed at least twice. As has been previously reported (Ohyama et al., 2015), false negatives (missing branches) were far more likely errors than false positives (adding the wrong branch).

# Electrophysiology

Adult leeches were anesthetized in ice-cold saline, dissected and chains of four midbody ganglia were removed and pinned in Sylgard-coated dish. The ventral sheath of the second ganglion and dorsal sheath of the third ganglion were removed to expose cell bodies for penetration with 1.0 mm (OD) × 0.75 mm (ID) glass microelectrodes with an omega dot pulled to a resistance of ∼20 M. Microelectrodes were filled with 20 mM KCl and 1 M potassium acetate. To verify that the S cell was impaled, and its intracellular spikes were matched 1:1 with the largest extracellularly-recorded spikes in the connective between the third and fourth ganglia. To verify cell 116's identity, we loaded electrode tips with either Alexa Fluor 488 or Alexa Fluor 594 (Thermo Fisher Scientific) and filled the electrode shanks with 3 M potassium acetate. Dye was then injected with alternating depolarizing and hyperpolarizing


Presynaptic cells are listed in the first column and postsynaptic cells are listed in the first row. All expected connections were found, with the exception of direct connections from DI-1, DI-102, or VI-2 onto the L cells. Some unexpected synapses were also found but were typically low in number compared to expected synapses (e.g., right DI-1 onto left DE-3).

current pulses (2 nA for 300 ms, −2 nA for 50 ms, 10% duty cycle for 30 min) and the shape of the arbor compared to the reconstructed arbor from the juvenile ganglion SBEM dataset.

Intracellular current injection and measurement of membrane potential were mediated by an Axoclamp-2B amplifier (Axon Instruments Inc., San Jose, CA, USA) operated in bridge mode. Extracellular recordings were amplified by a Model 1700 A-M Systems differential amplifier. Electrical signals were digitized, recorded and analyzed with WinWCP (Strathclyde Electrophysiology Software). Further analysis was performed with Microsoft Excel (Microsoft).

#### Experimental Design and Statistical Analysis

This bulk of this work represents analyses of a single volume of electron micrographs containing one juvenile leech ganglion. We therefore do not make any statistical comparisons—we present our work as a set of observations which can then be compared to prior work and suggest future experiments. Similarly for our electrophysiology experiment, we do not make any statistical comparisons.

## RESULTS

#### Neuron and Synapse Identification

The somata of leech neurons are arrayed along an outer rind of each midbody ganglion. To identify a soma in our EM volume, we first compared its size and location with the known map (Ort et al., 1974; Muller et al., 1981). Soma position can vary slightly from ganglion to ganglion, but the basic shape of the neuron's arbor can distinguish it from its neighbors (Fan et al., 2005). Our identifications were based on a combination of soma size, position, and arbor morphology. By convention, neurons are named according to their corresponding letter or number identifier in the accepted map (Ort et al., 1974; Muller et al., 1981). In the case of motor neurons, these cell number identifiers are preceded by two letters indicating which motor group the cell innervates and whether its outputs are excitatory or inhibitory. For example, cell DI-1 is an inhibitor of the dorsal longitudinal muscles while cell VE-4 is an excitor of the ventral longitudinal muscles. Most neurons in the leech ganglion are paired, having both a right and a left homolog. An exception (the ''S-cell'') is considered below.

We identified synapses by the criteria discussed in recent work (Pipkin et al., 2016). Briefly, leech presynaptic varicosities lack densely-staining T-bars characteristic of neuropil in Drosophila and some other invertebrate preparations. Instead, presynaptic sites are labeled by small presynaptic tufts of electron-dense material and faint postsynaptic densities that are indistinguishable at the resolution afforded by SBEM (Purves and McMahan, 1972; Muller and McMahan, 1976; Muller and Carbonetto, 1979). Our requirements for synapse identification were twofold: (1) a concentration of small presynaptic vesicles in the presynaptic neuron, some of which lie near to the membrane apposition of presynaptic and postsynaptic neurons; and (2) that the apposition of presynaptic and postsynaptic membranes persists over three or more consecutive imaging sections. Our criteria are more liberal than those afforded by higher-resolution TEM. As a result, while they capture the majority of real synapses we cannot exclude the possibility that we have mis-identified some nonsynaptic appositions as synapses.

## Testing Physiologically-Characterized Circuits Anatomically

The synaptic connections among neurons that generate behaviors in the leech are made in the neuropil of each ganglion. Within our juvenile ganglion volume, we explored the connections of a subset of motor neurons known to participate in the swimming behavior (Ort et al., 1974). Specifically, we searched the neuropil for synapses among the bilateral pairs of neurons DI-1, VI-2, DI-102, DE-3 and VE-4, which innervate dorsal and ventral longitudinal muscles and are responsible in part for the undulation of the leech's body during swimming. In addition, we also searched for connections made by these cells with the pair of L motor neurons, which are excited during the shortening reflex but are inhibited throughout swimming.

FIGURE 1 | Most, but not all, of the predicted physiological connections were found anatomically after reconstructing the arbors of six pairs of dorsal motor neurons. (A) Predicted circuitry based on dual electrophysiological recordings, adapted from Ort et al. (1974). Lines ending in circles represent inhibitory connections; lines ending in a T-junction indicate excitatory connections; resistors indicate non-rectifying gap junctions; diodes represent rectifying gap junctions. (B) Updated circuitry based on what was directly observed after anatomical reconstruction. Electrical connections are grayed out as these are not directly observable with serial blockface electron microscopy (SBEM). All predicted connections were found except those onto the L cell. A few unexpected synapses were found (e.g., from cell 1 to cell 102); these typically involved far fewer overall synapses (Table 1). The total number of synaptic contacts made by both the right and left pairs of neurons are shown next to each line (see also Table 1). (C) Examples of synapses between the right DI-1 and the right DE-3 (upper panel), the right DI-102 and the right DE-3 (middle panel), and the left VI-2 and the right VE-4 (lower panel). In these examples, the cells are fully segmented to display the relative scale of the participating processes; the remainder of their arbors were traced via skeletonization. Scale bars 300 nm.

The physiologically-determined circuit among these cells is depicted in **Figure 1A** (adapted from Ort et al., 1974). In this diagram, non-rectifying electrical synapses are represented by resistors and rectifying electrical synapses are represented by diodes. As the resolution of SBEM precludes the direct observation of gap junctions, we turned our attention first to chemical synapses (**Figure 1A**). We first sought to locate and quantify the number of known inhibitory synapses made within the neuropil in this circuit. To do so, we manually traced skeleton arbors of all the neurons involved, noting where each neuron made a synapse onto the other neurons (**Figures 1B,C**), using the criteria established in our previous study (Pipkin et al., 2016). The number of synapses formed in this network are summarized in the connectivity matrix shown in **Table 1**. We found numerous synaptic contacts consistent with the previously-described direct inhibition of DE-3 by the ipsilateral DI-1 and DI-102 and the direct inhibition of VE-4 by the ipsilateral VI-2. We did not find any chemical synapses from DI-1, DI-2, or DI-102 onto either L cell (**Figure 1B**), suggesting that the observed physiological inhibition occurs via an indirect pathway, potentially via the electrical connections.

As suspected from electrophysiological recordings (Ort et al., 1974), we observed that each DE-3 received direct inhibitory input from the ipsilateral DI-1. We previously observed that each DI-1 forms presynaptic boutons only in the contralateral portion of their arbors (Pipkin et al., 2016). In **Figure 2A**, the right DI-1 (green) is presynaptic to the right DE-3 via 18 synapses (red dots). Within the contralateral arborization of DE-3, these 18 synapses were widely distributed, contradicting previous predictions that inputs from DI-1 might be concentrated onto a single branch (Lytton and Kristan, 1989). We found a similar pattern among the inputs from the DI-102s onto the DE-3s (data not shown). Notably, the right DE-3 received no input from the left DI-1, despite overlap of the vesicle-containing portion of the left DI-1's arbor with the ipsilateral arborization of the right DE-3 (box, **Figure 2B**). With the exception of a single synaptic contact, this was also true for the right DI-1 and left DE-3 and for both DI-102s and DE-3s (**Table 1**). Similar to the dorsal muscle inhibitory motor neurons (DI-1 and DI-102), the ventral inhibitor (VI-2) neurons form presynaptic boutons in only the contralateral portion of their arbor. Consistent with the fact that the pair of ventral excitatory motor neurons (VE-4) arborize exclusively in the in the ipsilateral half of the neuropil each VE-4 received direct inhibition only from the contralateral VI-2 (**Table 1**), a finding that agrees with the electrophysiological characterization of this connection (Ort et al., 1974).

## Electrical Connections

It is impossible to directly observe the fine structures characteristic of gap junction membrane appositions when constrained by the resolution limits of SBEM (Brightman and Reese, 1969). Nonetheless, several of the cells we traced formed electrical connections with each other on the basis of prior electrophysiological evidence (Ort et al., 1974). We therefore took note when the membranes of two cells known to be electrically-coupled came into extended contact over many sections. On the basis of this criterion, we observed several suggestive contacts. In some cases, the contact is extensive in area and seen at many separate sites. For example, we traced the S cell, a unique excitatory interneuron involved in the shortening reflex (Laverack, 1969; Frank et al., 1975; Magni and Pellegrino, 1978; Crisp and Muller, 2006) and known for its large fast-conducting axon that it extends both anteriorly and posteriorly in Faivre's nerve. Halfway between each ganglion, this axon forms an electrical synapse with the S cell of the adjacent ganglion such that spikes generated in one S cell are propagated throughout the

entire nerve cord (Muller and Carbonetto, 1979). Additionally, the S cell is known for making strong electrical connections with two ''coupling interneurons'' that act in part as relays for sensory inputs (Muller and Scott, 1981). In **Figure 3A**, we show a confluence of processes belonging to the S cell (blue) and one of each coupling interneuron (green and pink). In this particular junction, each cell's membrane is closely apposed to and conforms to each other's and this interaction persists over several sections. We also searched for contacts among other known coupled cells. For example, **Figure 3B** depicts the close apposition of the left DI-102 (red) and left DI-1 (yellow). Both these cells are known to be physiologically coupled (**Figure 1A**). Here two of their secondary branches come into close contact as they travel adjacent to each other; notice again that both cells' membranes are closely apposed and conformed to each other. Not all possible junction sites involved symmetrically sized processes. In one case, a thin process belonging to the left DE-3 (orange) burrows into the primary process of the right DE-3 (purple; **Figure 3C**). Again, both these cells are known to be coupled (almost all pairs of dorsal motor neurons are electrically coupled (Ort et al., 1974; Fan et al., 2005)). In every instance involving known electrically coupled cells, we observed sites of membrane contact that could harbor gap junctions. For instance, we found 24 and 26 contacts between the S cell and each coupling interneuron, 5 between the left DI-1 and left DI-102 and 10 between both DE-3s. Like chemical synapses (**Figures 2A,B**), these contact sites were distributed throughout cells' arbors. Because we traced arbors chiefly via skeletonization, we cannot say whether the cumulative amount of membrane apposition predicts electrical connections. However, we can report that we did not observe similarly prolonged, conformed appositions among cells not known to be coupled.

### Predicting a Physiological Connection From an Anatomical Connection

We next sought to test whether an anatomical synapse predicts a physiological connection. For this experiment, we turned to cell 116. Each cell 116 is inhibitory and resides in the dorsal aspect of the anterolateral packet (E.P. Frady and K. L. Todd, personal communication). In tracing arbors of the pair of cells 116, we noticed that each received synaptic input from the S cell. The S cell (blue skeleton, **Figure 4A**) made six synapses onto the right 116 (orange skeleton, **Figure 4A**) and seven synapses onto the left 116 (green skeleton, **Figure 4A**), distributed throughout the extent of the S cell arbor (pink dots, **Figure 4A**). In one case, both cells 116 were postsynaptic to the same S cell bouton.

We next tested to see if inducing action potentials in the S cell network would reliably lead to excitatory potentials in cell 116. Because the S cell in one ganglion is strongly coupled to the S cells in the next ganglion anterior or posterior to it, we were able to circumvent the practical difficulty of simultaneously recording intracellularly from one cell on the ventral surface and another cell on the dorsal surface. Instead, we impaled the S cell in the ganglion adjacent to the one in which we recorded cell

116 (**Figure 4C**). To confirm that the spike traversed through the network, we recorded the connective nerves posterior to the cell 116 ganglion with an extracellular electrode (**Figure 4C**). We observed that each S cell spike reliably preceded a 1–2 mV EPSP in cell 116. The cell 116 response to 15 S cell spikes (overlaid, gray traces in middle panel) is presented in **Figure 4C** along with their average (black trace in middle panel). The 4–5 ms latency between spike and EPSP is consistent with known conduction velocity of the S cell spike through Faivre's nerve (Frank et al., 1975).

the same preparation (gray) and their average (black).

# DISCUSSION

Our results validate a connectomics approach for circuit discovery in the leech ganglion. We show that reconstruction of selected cells can be used to confirm the existence of previously known connections among motor neurons (**Figure 1**, **Table 1**). Previous work showed that the ipsilateral DI-1 and DI-102 monosynaptically inhibit DE-3, while the contralateral VI-2 inhibits VE-4 (Ort et al., 1974; Granzow et al., 1985). At the resolution of light microscopy, others have observed considerable overlap between the processes of these cells and have noted possible sites of apposition of postsynaptic processes with presynaptic varicosities (Granzow et al., 1985; Fan et al., 2005). At the EM level, Granzow et al. (1985) attempted to demonstrate the connection between DI-1 and DE-3 by differentially staining the two cells with intracellular fills (Imposil in DI-1, horseradish peroxidase in DE-3) and taking thin sections of the contralateral half of the neuropil. However, due to suspected disruption of vesicle structure wrought by Imposil they found presynaptic vesicles near only one of many sites of abutment between the two cells (Granzow et al., 1985). By analyzing a complete SBEM volume of an entire ganglion, our report is the first to provide direct EM anatomical confirmation of these synapses among motor neurons.

For each of these known connections (DI-1 -> DE-3, DI-102 - > DE-3, VI-2 -> VE-4), we found more than one synapse from the presynaptic cell onto the postsynaptic cell. The number of such contacts ranged from 2 (from the right VI-2 onto the left VE-4) to 18 (from the right DI-1 onto the right DE-3; **Table 1**). It is unclear in what ways this variability is physiologically meaningful, as we cannot infer the synaptic strength of a given synapse in a SBEM volume. While it is tempting to speculate whether a connection with more contacts is stronger than one with fewer, the highly electrically-coupled motor circuit we reconstructed is not ideal for addressing this question. Other subcircuits in the leech ganglion are more attractive. For example, the connections among the sensory P cells and local bend interneurons are known to vary in strength physiologically (Lockery and Kristan, 1990a). Unfortunately, the cells involved primarily form their arbors in the ventral aspect of the neuropil, where the deteriorated quality of our dataset precluded accurate reconstruction (see ''Materials and Methods'' section). Ongoing work carrying this project forward in the adult leech ganglion by the Wagenaar and Ellisman groups may be able to more fully explore the relationship between contact number and physiological synapse strength.

The range of contact number we observe falls below that measured by light microscopic analysis of overlap between adult sensory and motor neurons (13–41 in DeRiemer and Macagno, 1981). This difference could be due to the maturity of the tissue, the specific cell pairs studied, or methodological differences (processes may overlap at the light level but do not touch at the EM level). The range of synapse number per connection that we find (1–18) is somewhat comparable to what has been found in other systems in which entire arbors have been reconstructed from serial EM images (C. elegans: 1–19 in the hermaphrodite (White et al., 1986), 1–61 in the male tail (Jarrell et al., 2012); Platynereis dumerilii: 1–45 including neurons and muscles of visuomotor circuitry (Randel et al., 2014); Drosophila melanogaster: 1–in the visual circuitry (Takemura et al., 2013), 1–23 from a selectively reconstructed motor circuit in the larva (Ohyama et al., 2015).

We observed some unexpected sites of potential synaptic contact among the motor neurons we traced (for example, the right DI-1 makes a single synapse onto the left VE-4, **Table 1**). Notably, these cases involve far fewer overall contacts (1–3). Ohyama et al. (2015) also examined circuitry in which multiple types of the same cell in Drosophila larvae (Basins 1–4) made inputs onto various postsynaptic cells. In their data, they report instances where these postsynaptic cells predominantly receive input from one of these Basin cell types while still receiving scattered input from the others (for example, the cell they label A12q a1l receives 15/14 synaptic inputs from the right/left Basin 2 s and 0/1 from the right/left Basin 1). There are a number of possible explanations for our finding of unexpected connections: (1) these synapses may be real but so relatively few in number as to be physiologically undetectable and unimportant; (2) these synapses may be present only in juvenile tissue that is still undergoing synaptic refinement; (3) these synapses could be mistakenly identified or otherwise be the result of a tracing error that we cannot detect after reviewing them; (4) some of these synapses might actually be gap junctions occurring at a location that makes them appear to be chemical synapses (e.g., the connections between left DI-1 and left DI-102, two inhibitory neurons known to be electrically coupled (Fan et al., 2005)).

We found that synapses between two cells widely spanned the region of overlap between the vesicle-containing portion of the presynaptic cell's arbor and the postsynaptic cell's arbor (**Figure 2**). Earlier reports had suggested that the synapses made by DI-1 and DI-102 might be concentrated onto separate single branches of the DE-3 arbor (Lytton and Kristan, 1989). We find no evidence for such selectivity in our juvenile ganglion volume, though we cannot rule out that synapse strength might vary depending on where a synapse occurs or that branchselectivity is a process that is not yet complete in juvenile tissue.

We almost exclusively found synapses from the DI-1 and DI-102 cells onto the ipsilateral DE-3 even though the vesiclecontaining portion of the DI-1 or DI-102 arbor overlaps with postsynaptic regions of both the ipsilateral and contralateral DE-3. This lateral selectivity suggests that there may be some chemical basis by which synapse formation is restricted to the ipsilateral cell pair. This result also underscores the strengths of EM vs. light microscopy: arbor overlap is not predictive of where synapses occur. In the retina, random synapse formation on the basis of process proximity cannot explain the location of synapses found between directionselective cells and starburst amacrine cells (Briggman et al., 2011). Similarly, the proximity of axons to dendritic spines is a poor predictor of connectivity in a densely-reconstructed ssTEM dataset spanning a volume of the mouse neocortex (Kasthuri et al., 2015).

The presence and pattern of synapses we found among DI-1, DI-102, VI-2, DE-3 and DE-4 conformed to our expectations given known physiological evidence (Ort et al., 1974). However, we failed to find any synapses from DI-1, DI-102, or VI-2 onto either L cell as previous physiology predicted (**Table 1**, Ort et al., 1974). The L cell is known to be electrically coupled to other excitatory motor neurons that receive direct monosynaptic inhibition from DI-1, DI-102 and VI-2 (Ort et al., 1974; Fan et al., 2005). Therefore, the synaptic input from these cells onto the L cell may be indirect while physiologically appearing otherwise (this pattern has been observed before in the leech whereby sensory cells influence the S cell via a pair of cells electrically coupled to the S cell (Muller and Scott, 1981)). This finding underscores the utility of anatomical synapse verification at the EM level: physiological connections between cells whose arbors overlap are nonetheless not necessarily monosynaptic.

Detecting electrical connections mediated by gap junctions remains an unsolved challenge in SBEM-based connectomics. In our volume, we knew certain cell pairs to be coupled, and were able to locate several places where their membranes came into prolonged contact (**Figure 3**). Some of these sites are almost certain to contain gap junctions, but we cannot determine how many contacts are functional vs. incidental. New specimen preparation techniques (e.g., Pallotto et al., 2015) that preserve or expand the extracellular space can aid in identifying gap junctions even in SBEM. In future samples of leech neuropil, these approaches, in concert with pre or post hoc physiological verification, could lead to the description of patterns of membrane apposition associated with gap junctions in the leech.

Connectomes produce anatomical predictions of neuronal connectivity which can then be verified physiologically. In C. elegans, the connectome has long served as a roadmap for guiding subsequent cell ablation, imaging, and physiological experiments (Bargmann and Marder, 2013). In the larval fruit fly, connectomics predicted a neuronal circuit responsible for multisensory integration involved in rolling behavior (Ohyama et al., 2015), connections that were then verified using calcium imaging. Similarly, we demonstrated that anatomical connections can be recapitulated in physiological measurements by first discovering synapses from the S cell onto both cells 116 in our EM volume and subsequently demonstrating that spikes in the S cell produce a depolarization in cell 116 (**Figure 4**). This result also highlights the advantages of using an electrophysiologically accessible system in which the same cells can be identified from ganglion to ganglion and animal to animal. In principle, a complete reconstruction of a ganglion could dramatically reduce the number of pairwise recordings needed in other ganglia to confirm the existence of the identified connections, as opposed to testing every possible pair of neurons (∼80,000). Importantly, in the leech ganglion these physiological experiments can involve the direct measurement of membrane potential (via intracellular electrophysiology) rather than indirect measures of activity like calcium imaging that struggle to reveal inhibitory connections.

While the leech is studied in part because of how reproducible physiological recordings are from ganglion to ganglion, anatomical features including soma position, neuronal composition (Lent et al., 1991), and fine branching patterns also vary. It is possible that there will exist some cases in which two cells are synaptically connected in some of the 21 ganglia in the nerve cord and not others, or that there are reproducible connections in different ganglia that nonetheless involve differing numbers and locations of synapses. Unfortunately, the high time and labor commitment required to produce full cell reconstructions and annotations currently limits image acquisition and analysis to a single ganglion. In other systems, measuring sample-to-sample variability from EM reconstructions has thus far been largely confined to two samples. In the earliest connectome, C. elegans was reconstructed from partially overlapping datasets from different animals; the connections found in the region of overlap were largely consistent from sample to sample (White et al., 1986). Similarly, in the region of overlap in two different first instar Drosophila larvae, 96% of connections involving two or more synapses in one animal were also found in homologous cells in the other animal (Ohyama et al., 2015), a pattern of connectivity that remained consistent in third instar larvae (Gerhard et al., 2017), though overall numbers of synapses increased proportional to the growth of the arbors. In a partial connectome of the Platynereis visual system, there was also a high concordance between two animals (Randel et al., 2015). Moving beyond these low N experiments will eventually require even further acceleration of imaging and analysis. In particular, automated and semi-automated reconstruction and annotation techniques currently in development (Helmstaedter, 2013; Berning et al., 2015; Kasthuri et al., 2015; Dorkenwald et al., 2017; Staffler et al., 2017; Januszewski et al., 2018) could considerably decrease time costs, enabling larger sample sizes.

Our results demonstrate the utility of applying serial EM reconstruction to a system in which individual neurons can be identified from preparation to preparation. Known connections can be verified or challenged, and previously unknown connections can be discovered and subsequently tested. This connectomics approach enables the interplay between anatomical thoroughness and physiological precision that will allow future researchers to uncover previously inaccessible details regarding the circuits underpinning behavior in the leech ganglion.

# AUTHOR CONTRIBUTIONS

JP and EB carried out experiments and analysis. WK and ME designed the experiments. All authors contributed in drafting the manuscript.

# FUNDING

This work was supported by an NIH research grant (MH43396) to WK, an NSF grant (IOS 835741) to WK, an NIH grant (P41GM103413) to ME, and a grant from the Kavli Institute for Brain and Mind (to WK and ME). ME and EB are carrying the leech connectomics project forward under R01 NS094403 to D. Wagenaar at Caltech and through P41GM103412.

# ACKNOWLEDGMENTS

We thank Mason Mackey and Tom Deerinck for their assistance and advice for preparing and imaging the samples.

#### REFERENCES


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Pipkin, Bushong, Ellisman and Kristan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Pipeline for Volume Electron Microscopy of the Caenorhabditis elegans Nervous System

Ben Mulcahy<sup>1</sup> \*, Daniel Witvliet1,2, Douglas Holmyard3,4, James Mitchell<sup>5</sup> , Andrew D. Chisholm<sup>6</sup> , Yaron Meirovitch<sup>7</sup> , Aravinthan D. T. Samuel<sup>5</sup> \* and Mei Zhen1,2,8,9 \*

<sup>1</sup> Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON, Canada, <sup>2</sup> Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada, <sup>3</sup> Department of Pathology and Laboratory Medicine, Mount Sinai Hospital, Toronto, ON, Canada, <sup>4</sup> Nanoscale Biomedical Imaging Facility, The Hospital for Sick Children, Peter Gilgan Centre for Research and Learning, Toronto, ON, Canada, <sup>5</sup> Center for Brain Science, Department of Physics, Harvard University, Cambridge, MA, United States, <sup>6</sup> Section of Cell and Developmental Biology, Division of Biological Sciences, University of California, San Diego, La Jolla, CA, United States, <sup>7</sup> Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, United States, <sup>8</sup> Department of Physiology, University of Toronto, Toronto, ON, Canada, <sup>9</sup> Department of Cell and Systems Biology, University of Toronto, Toronto, ON, Canada

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Kerrianne Ryan, Dalhousie University, Canada John Graham White, University of Wisconsin–Madison, United States

#### \*Correspondence:

Ben Mulcahy mulcahy@lunenfeld.ca Aravinthan D. T. Samuel samuel@physics.harvard.edu Mei Zhen zhen@lunenfeld.ca

Received: 16 August 2018 Accepted: 08 October 2018 Published: 21 November 2018

#### Citation:

Mulcahy B, Witvliet D, Holmyard D, Mitchell J, Chisholm AD, Meirovitch Y, Samuel ADT and Zhen M (2018) A Pipeline for Volume Electron Microscopy of the Caenorhabditis elegans Nervous System. Front. Neural Circuits 12:94. doi: 10.3389/fncir.2018.00094 The "connectome," a comprehensive wiring diagram of synaptic connectivity, is achieved through volume electron microscopy (vEM) analysis of an entire nervous system and all associated non-neuronal tissues. White et al. (1986) pioneered the fully manual reconstruction of a connectome using Caenorhabditis elegans. Recent advances in vEM allow mapping new C. elegans connectomes with increased throughput, and reduced subjectivity. Current vEM studies aim to not only fill the remaining gaps in the original connectome, but also address fundamental questions including how the connectome changes during development, the nature of individuality, sexual dimorphism, and how genetic and environmental factors regulate connectivity. Here we describe our current vEM pipeline and projected improvements for the study of the C. elegans nervous system and beyond.

#### Keywords: C. elegans, volume electron microscopy, connectome, nervous system, high-pressure freezing

## A BRIEF BACKGROUND OF Caenorhabditis elegans CONNECTOMICS

In the 1960s, Sydney Brenner and colleagues adopted the nematode Caenorhabditis elegans as a model to better understand the development and function of a complete nervous system. Part of their strategy was to reconstruct the entire synaptic wiring diagram of a nervous system using manual volume electron microscopy (vEM). C. elegans was a wise choice. Its small size, a cylinder of ∼1 mm in length and 70 µm in diameter, provided a reasonable chance of success with the laborious and technically challenging procedures required for vEM. Nichol Thompson developed the essential skill in cutting long series of serial sections without gaps. Initial successes included reconstructions of the anterior sensory anatomy (Ward et al., 1975; Ware et al., 1975), the pharyngeal nervous system (Albertson and Thomson, 1976), and the ventral nerve cord (White et al., 1976). When John White and Eileen Southgate succeeded in tracing the nerve ring, the first near-complete wiring diagram of an animal's nervous system was obtained (White et al., 1986; White, 2013).

The C. elegans connectome provided the first comprehensive physical map through which information flows to select, enact, and modify motor functions. This structural foundation first allowed the formulation and experimental validation of hypotheses for mechanosensory and motor behaviors (Chalfie et al., 1985). The small number of neurons and their connections has since inspired numerous theoretical and experimental studies to model entire sensorimotor circuits (e.g., Varshney et al., 2011; Towlson et al., 2013; Szigeti et al., 2014; others).

With the recent emergence of partial wiring diagrams for neural circuits in other invertebrates and some vertebrates (e.g., Helmstaedter et al., 2013; Takemura et al., 2013; Randel et al., 2014, 2015; Kasthuri et al., 2015; Ryan et al., 2016, 2017; Eichler et al., 2017; Veraszto et al., 2017; Williams et al., 2017; others), the search for conserved features and circuit motifs that might have homologous functions across species becomes possible.

Caenorhabditis elegans connectomics will play a crucial role in uncovering general principles of neural circuit structure and function. The C. elegans nervous system embeds computational properties sufficiently powerful for many complex behaviors: different motor patterns and states, adaptive, and integrative sensory perception, as well as forms of associative learning and memories (Zhang et al., 2005; Ardiel and Rankin, 2010; Sasakura and Mori, 2013; Allen et al., 2015; Zhen and Samuel, 2015). Its small and accessible size – both in terms of neuron number (302) and synapse number (∼7000) – makes it a tractable system to propose and test theoretical models of nervous system function. If the circuit designs that enable sensory coding, decision-making, and plasticity are evolutionarily conserved, understanding mechanisms of the compact C. elegans nervous system will yield useful insights into shared principles.

Progress still needs to be made at multiple fronts in C. elegans connectomics.

First, the original C. elegans connectome was assembled from partially overlapping fragments of a few individuals, not one intact individual (White et al., 1986). The validity of this approach hinges on the stereotypy of the wiring diagram across individuals. The stereotypy observed for most C. elegans cells identified by lineage studies (Sulston and Horvitz, 1977; Sulston et al., 1983) and preliminary comparison of the central nervous system connectivity of two animals (Durbin, 1987) made this plausible. However, an explicit analysis of variability across connectomes of multiple individuals is required.

Second, postembryonic neurogenesis occurs across C. elegans development. Post-embryonically born neurons make up ∼25% of neurons in the adult. The original C. elegans connectome was assembled from parts of several adults and one last stage larva, reflecting one snapshot of a dynamic wiring diagram. How the connectome develops, remodels to incorporate newly born neurons, and modifies the behavioral repertoire at different developmental stages needs to be addressed.

Third, sexual dimorphism is prominent in the C. elegans nervous system. Compared to adult hermaphrodites, adult males have an additional 85 neurons, accounting for ∼20% of the nervous system (Sulston and Horvitz, 1977; Sulston et al., 1980; Sammut et al., 2015; Molina-Garcia et al., 2018). Though progress has been made on the wiring of parts of the male nervous system (Hall and Russell, 1991; Jarrell et al., 2012), a complete and comprehensive side-by-side comparison of high-quality male and hermaphrodite connectomes awaits.

Fourth, natural variants of C. elegans exhibit substantial genetic and behavioral differences from that of the laboratory wild-type strains. The connectomes of these and other nematode species should be obtained and compared.

Addressing questions about individual variability, developmental plasticity, sexual dimorphism, genetic perturbations, and so on requires higher-throughput vEM reconstruction. Recent focus on technology development, such as automation in serial sectioning (Schalek et al., 2012), image acquisition (Inkson et al., 2001; Denk and Horstmann, 2004; Holzer et al., 2004; Heymann et al., 2006; Knott et al., 2008; Hayworth et al., 2014), and segmentation of neurons and connections (Saalfeld et al., 2009; Helmstaedter et al., 2011; Cardona et al., 2012; Boergens et al., 2017), has accelerated vEM throughput. Originally designed to allow acquisition of connectomes of single large samples, these technological advances offer small model systems such as C. elegans an opportunity to employ vEM as a rapidly deployable tool for developmental and comparative connectomics, and other aspects of nematode biology.

Below we describe such a pipeline.

# OUTLINE OF A PIPELINE FOR CURRENT C. elegans EM STUDIES

This pipeline has been successfully used for high-throughput volume reconstruction of intact C. elegans of all developmental stages, and has yielded high-resolution connectomes for multiple animals (**Figure 1**; Witvliet et al., in preparation). We describe technical issues general to vEM studies and highlight key technical considerations for C. elegans.

# Step 1: Preparing Samples for EM

Rapid freezing of living animals facilitates uniform vitrification. Subsequent freeze-substitution and fixation allows preservation of organelles, cells, and tissues in their native states. Due to its small size, intact C. elegans is well suited to highpressure freezing, circumventing the mechanical damage and physiological perturbation caused by dissection. Through standard en bloc and post-sectioning staining with heavy metals, sufficient contrast can be imparted to lipids, proteins, and nucleic acids for visualization with an electron microscope.

#### Step 2: Serial Sectioning

The thickness and number of serial sections are determined by the sectioning method, as well as the size of the object of interest. Reducing section thickness facilitates reconstruction of fine cellular structures (such as neurites), and distinction between intracellular features (such as vesicles, ER, and microtubules). Because of the small diameter of C. elegans neurites, serial sections of 50 nm or thinner are needed for reliable connectome reconstruction.

reconstruction, which yields topographical information of cells and neurons, is facilitated by computational filling followed by manual proofreading using VAST.

# Step 3: Image Acquisition and Processing

Image resolution is set by the size of object of interest. For adult and larval connectome reconstructions, a resolution of 1–2 nm per pixel is optimal for reliable synapse annotation. A montage of images that cover the area of interest are computationally stitched and aligned into a 3D volume. Minimization of artifacts during sample preparation (e.g., mechanical compression during sectioning) and imaging (lens distortion and shrinkage during electron beam exposure), and their correction are critical for acquiring a well-aligned image volume.

#### Step 4: Segmentation

The aligned image stacks are segmented into objects of interest. For connectomes this means tracing neurons and mapping synapses. Volumetric segmentation consists of coloring in each section of neurite throughout the volume, reconstituting the 3D morphology of the cell. Skeleton segmentation consists of placing a point in the center of the neurite on each section. Tracing skeletons is faster than volumetric segmentation, but less rich in morphological detail.

#### Step 5: Synapse Annotation

Synapse identification is based on stereotypic ultrastructural features. A sample with well-preserved neurite morphology and intracellular organelles, such as presynaptic active zones and synaptic vesicles, facilitates high-confidence annotation of chemical synapses. However, synapse annotation is not completely objective. Subjectivity arises in the identification of small synapses, gap junctions, and assigning postsynaptic partners for polyadic synapses. Increased section thickness, section and staining artifacts, and unfortunate synapse orientation relative to the plane of sectioning also increase subjective uncertainty. Parallel annotation of the same dataset by multiple tracers, constructing connectomes from multiple animals, and comparing with existing datasets help to reduce annotation errors.

# Step 6: Neuron Identification

Every somatic C. elegans cell can be assigned a unique name. The location and identity of each nucleus was lineage-mapped by following its migration throughout development (Sulston and Horvitz, 1977; Sulston et al., 1980, 1983; White et al., 1986). Additionally, all processes within the neuropils have characteristic features, allowing identification without necessarily tracing the process back to the cell body. Stereotypic features include entry-point into the neuropil, neurite trajectory and morphology, placement within the neuropil, abundance of clear and dense-core vesicles, multi-synapse clusters, and unique morphological features. Each neuron can be identified by characteristic features at multiple points along its process, increasing the confidence of tracing.

# STEP-BY-STEP DESCRIPTION OF METHODS AND CONSIDERATIONS

#### Preparation of EM Samples

fncir-12-00094 March 19, 2019 Time: 14:22 # 4

#### General Considerations for High-Pressure Freezing and Freeze Substitution

For the original C. elegans wiring diagram reconstruction, animals were submerged in one or more chemical fixatives, either glutaraldehyde followed by osmium tetroxide, or osmium tetroxide alone (White et al., 1986). Some animals were cut by razors to aid the diffusion of fixatives through the tissue. This fixation process is not instantaneous (e.g., tomato hair cells have been estimated to be fixed at a rate of 2 µm/s in a glutaraldehydecacodylate solution; Mersey and McCully, 1978), and distortions to native ultrastructure occur before fixation is complete (Smith and Reese, 1980; Gilkey and Staehelin, 1986; **Figures 2A,C**).

A better strategy for tissue preservation involves rapid freezing of samples in vitreous ice, dehydration at low temperatures to prevent the growth of damaging ice crystals, and simultaneous fixation. In early work in other experimental systems, this was achieved by subjecting samples to extremely low temperature (around −175◦C), either by plunging the sample into cold liquids, propelling the cold liquid at the sample (Feder and Sidman, 1958; Moor et al., 1976), or slam freezing – dropping tissue onto a metal block cooled with liquid nitrogen or

FIGURE 2 | High-pressure freezing improves preservation of ultrastructure. (A) The dorsal cord of an adult prepared using the slow chemical fixation protocol (White et al., 1978). The DD motor neuron is making a neuromuscular junction to dorsal muscle cells. (B) The dorsal cord of an adult fixed using high-pressure freezing and imaged using TEM. The DD motor neuron is making a neuromuscular junction to dorsal muscle cells. (C) The ventral nerve cord of a chemically fixed first stage (L1) C. elegans larva (White et al., 1978). The DD axon makes a NMJ to the ventral muscle cell (M). (D) A TEM micrograph of the ventral nerve cord of a high-pressure frozen first stage larva (L1) at similar region, where DD makes a NMJ to the ventral muscle cell. The advent of high-pressure freezing allows better preserved neurite morphology, synapse structure, and extracellular space, facilitating connectomic and topological analyses of the C. elegans nervous system. Scale bar 1 µm. Panel (A) was reprinted with permission from White et al. (1978). Panel (C) a scan of the micrograph used in White et al. (1978), hosted by the WormImage Consortium (www.wormimage.org).

helium (van Harreveld and Crowell, 1964; Heuser et al., 1979; Heuser and Reese, 1981). Vitreous ice typically forms only within a few micrometers from the surface of the tissue. However, when water is pressurized to 2100 atmospheres, vitreous ice forms more easily and deeply (Kanno et al., 1975; Dahl and Staehelin, 1989; Dubochet, 2007). By applying this level of pressure during rapid freezing, Hans Moore and Udo Riehle obtained good preservation several hundred micrometers from the surface of biological tissues (Riehle, 1968; Moor, 1987).

Frozen samples are then freeze-substituted, a process where the immobilized water is dissolved by an organic solvent (Simpson, 1941). Fixatives such as osmium tetroxide are included in the freeze substitution solvent to fix the sample as it is warmed to room temperature. Once the sample reaches −80◦C, secondary ice crystals may grow and disrupt ultrastructure (Steinbrecht, 1985; but see Dubochet, 2007). Thus, organic solvents that are liquid below −80◦C, such as acetone, are used for freeze substitution.

The recent availability of commercial high-pressure freezers has made this approach more accessible. Successful highpressure freezing and freeze-substitution of C. elegans preserves ultrastructure and extracellular space better than chemical fixation (**Figures 2B,D**).

#### High-Pressure Freezing of C. elegans

Basic protocols for high-pressure freezing of a range of organisms including C. elegans have been described (e.g., Weimer, 2006; McDonald, 2007; Manning and Richmond, 2015). Below is a modified procedure that we have used successfully with both the Leica HPM100 and ICE models of high-pressure freezing machines.


FIGURE 3 | High-pressure freezing of C. elegans. (A) To pack the carrier with worms, our preferred method is to swipe it across a densely packed lawn of worms and bacteria. After swiping, the worm-bacteria mixture is spread across the cavity of the carrier with tweezers or a worm pick (a thin platinum wire mounted to a holder), the lid put in place, and the sample immediately high-pressure frozen. The entire process takes less than 30 s. (B) A carrier when it is packed. It is filled just right, without air bubbles. The smallest cavity for freezing is used, as freezing efficiency decreases with increasing depth. (C) A carrier packed with a mixed-staged larva after high-pressure freezing, freeze substitution, and resin infiltration. This carrier has retained the "cake" of worms, but much of the time the cake floats out. One can see how densely the worms are packed by the swiping method. (D) Worms are separated from the cake and individually embedded and cured in plastic blocks. Well-packed carriers as shown in panel (C) can yield hundreds of intact worm samples.

Tips:


#### Freeze Substitution With C. elegans Samples

For morphological analyses, freeze substitution is performed in a programmable freeze substitution unit, where frozen samples are kept at −90◦C in the presence of tannic acid and glutaraldehyde, before being replaced by 2% OsO4, and brought to room temperature (**Box 1**; Weimer, 2006). This protocol yields consistent results as long as samples are handled properly (see sections "General Considerations for High-Pressure Freezing and Freeze Substitution" and "High-Pressure Freezing of C. elegans"), and the high-pressure freezer is properly assembled and maintained.

This protocol can be further modified to reduce processing time and increase the membrane contrast, with the following considerations. Tannic acid helps target osmium to the membrane (Bridgman and Reese, 1984), but glutaraldehyde, inactive at −90◦C (Bridgman and Reese, 1984; McDonald, 2007), is likely expendable for the first-step fixation. Inclusion of 5% water in the organic solvent may improve membrane staining (Walther and Ziegler, 2002; Buser and Walther, 2008). To increase heavy metal deposition one can use a mordant to perform a double osmium stain, such as tannic acid (Simionescu and Simionescu, 1976; Wagner, 1976; Jiménez et al., 2009), or thiocarbohydrazide (Seligman et al., 1966; Webb and Schieber, 2018), followed by further en bloc uranyl acetate and lead acetate staining (Webb and Schieber, 2018). Lastly, we have confirmed that a fast freeze substitution protocol lasting just a few hours (McDonald and Webb, 2011) also yields well preserved C. elegans.

#### Infiltration and Embedding C. elegans Samples in Resin

After freeze substitution, the sample needs to be infiltrated with resin and cured in a block. We infiltrate in the same cryotube used for freeze substitution, either in graded steps on a rocker, or employing a fast protocol using centrifugation (McDonald, 2014). For morphology studies carried out by standard TEM and ATUM-SEM, we use Spurr-Quetol resin (NSA 27.88g, ERL4221 9.70g, DER 4.50g, Quetol651 6.12g, and BDMA 0.87g; Ellis, 2006) because it has good sectioning and staining properties, and a relatively low viscosity. For serial block face and FIB-SEM imaging, samples are infiltrated and cured with harder resins, such as hard Epon (EMbed 812 22.6g, DDSA 9.05g, NMA 14.75g, and DMP-30 0.8g) or Durcupan (Durcupan ACM resin 11.4g, DDSA 10.0g, dibutyl phthalate 0.35g, and DMP-30 0.15g).

Once infiltrated, contents of the cryotube are poured into a plate ready for embedding. By this stage, the disk-shaped "cakes" of worms and bacteria will often have fallen out of their carriers. If they are still inside the carrier (**Figure 3C**), an intact cake can be pried out of the coated carriers using the fine tip of a broken wood stick while holding the carrier in place with tweezers. Using a wooden stick instead of metal instruments is gentler on both the sample and the carriers. We embed either the whole cake, or individual worms released from the cake by repeatedly tapping the cake with the tip of a broken wooden stick until the bacteria crumble away, and intact worms remain (a delicate procedure, especially for young larvae).

Horizontal molds are used to cure samples, as we find it easier to orient samples for subsequent serial sectioning. To place the worm in the center of the block, which makes trimming and cutting easier, we semi-cure half-filled molds by putting them at 60◦C for a few hours, let cool, then fill to the top with fresh resin.

BOX 1 | Some freeze substitution protocols for C. elegans volume EM. Both (A) and (B) are effective protocols for ultrastructural preservation (Weimer, 2006).

After we transfer and orient the worms as desired inside the mold, they are cured at 60◦C for at least 24 h. The resulting blocks are ready for cutting (**Figure 3D**).

#### Serial Sectioning

Imaging sequential layers of a sample normally requires collecting serial sections for the sample. Although block face imaging techniques avoid this step (Inkson et al., 2001; Denk and Horstmann, 2004; Holzer et al., 2004; Heymann et al., 2006; Knott et al., 2008), samples are destroyed during imaging. There will always be applications for obtaining and preserving long image series. Many effective techniques have been developed (see **Box 2**).

#### Manual Serial Sectioning for TEM


height of the trapezoid should be as close to the top and bottom edges of the worm as possible, and the width should be ∼0.7 mm (**Figure 4A**). Gently dab a thin layer of glue (Elmer's rubber cement, in a mixture of 1 part glue, 3 parts xylene) to the bottom edge of the block to aid the ribbon formation.





BOX 2 | A collection of sectioning strategies for vEM. vEM using non-block face imaging (TEM and SEM) requires collecting large unbroken series of serial sections. There are multiple ways of making the process less error-prone, each with its own merit. One simply has to choose which process works best for them, or devise their own strategy. <sup>1</sup>Gay and Anderson (1954); Westfall and Healy (1962); Fahrenbach Wolf (1984); Galey and Nilsson (1966); Mironov et al. (2008); Anderson and Brenner (1971); Rowley and Moran (1975); Abad (1988); Wells (1974); Mironov et al. (2008); Stevens et al. (1980); Hall (1995); Schalek et al. (2012); Micheva and Smith (2007); Burel et al. (2018); Leica Microsystems, Germany.

(h) Sections are post-stained with 2% aqueous uranyl acetate and 0.1% lead citrate.

(D) Many grids of serial sections, stored in a grid box, are ready for imaging.

Tips:


#### Automation of Serial Sectioning for SEM

Alternative methods have been devised to automatically cut large volumes of serial sections, including the automated tape collecting ultramicrotome (ATUM; Schalek et al., 2012). Here, the sample is cut on an ultramicrotome and picked up by a rolling reel of tape. The tape is cut into strips, glued to a wafer and poststained with uranyl acetate and lead citrate. Electrons cannot pass through the tape, therefore scanning electron microscopy (SEM) must be used to image samples cut using an ATUM. We have used this approach to collect serial sections at 30 nm thickness, and used a SEM capable of high resolution imaging (1 nm/pixel; FEI Magellan XHR 400L) to acquire several high-quality datasets for C. elegans connectomics studies. Modern high-end SEMs are capable of producing TEM-equivalent micrographs and are suitable for identifying both chemical synapses and gap junctions with high confidence (e.g., **Figure 6**).

In contrast to the traditional approach of cutting, staining, then imaging sections in an electron microscope, new methods have been established to mount an uncut sample inside the microscope, image the surface using SEM, cut off the top layer, and image again. This process is repeated until the entire region of interest is processed. The cutting uses either a diamond blade inside the microscope (serial block face EM; Denk and Horstmann, 2004), or of a focused ion beam (FIB-SEM; Inkson et al., 2001; Holzer et al., 2004; Heymann et al., 2006; Knott et al., 2008). Both applications can produce images of large volumes for connectomics studies in an exceptionally short amount of time (Briggman and Bock, 2012). Without post-section staining, however, both SBF-SEM and FIB-SEM rely on en bloc staining for contrast.

## Image Acquisition and Processing

For connectome reconstruction, we acquired images of entire C. elegans cross-sections by either TEM or ATUM-SEM, at 1– 2 nm/pixel resolution. We found such a resolution to be necessary for unambiguous annotation of intracellular structures, tracing through small neurites, and synapse annotation. Acquiring the entire cross-section not only allowed us to fully reconstruct dorsal-ventral commissures and lateral nerve cords, but also provided landmarks that facilitated neuron identification.

After sections are imaged, they are stitched and aligned into a 3D volume. This requires processing of acquired images to compensate for artifacts generated during sectioning (e.g., differential compression of sections), and imaging (e.g., lens distortion, shrinkage of samples due to the energy of the electron beam). There are multiple solutions for alignment of datasets into 3D volumes (reviewed in Borrett and Hughes, 2016). We found TrakEM2 (Saalfeld et al., 2010; Cardona et al., 2012) to be most suitable for our C. elegans datasets, and we outline the process below.



Tips:


montages and compatible with a range of cameras (Mastronarde, 2005).


<sup>1</sup>www.ini.uzh.ch/ acardona/trakem2\_manual.html <sup>2</sup>http://imagej.net/Feature\_Extraction

FIGURE 6 | Chemical synapses and gap junctions in C. elegans. (A) A section of the first larva (L1) ventral ganglion neuropil imaged using SEM at 1 nm/pixel. Multiple chemical synapses are visible (white arrows) as well as a gap junction (white flat-ended line). (B) Enlarged view of the chemical synapse highlighted with a dashed box in panel (A). There is a presynaptic dense projection and a pool of synaptic vesicles, as well as some dense core vesicles further back in the neurite. This synapse is polyadic, releasing onto three neurons. (C) Enlarged view of the gap junction highlighted with a dashed box in panel (A). There is a relatively flat area of close apposition between the membranes.

superimposing a transparent copy of the previous layer to register poorly aligned sections. Using manually placed landmarks to register multiple sections is also an effective strategy.

• Care must be taken not to distort or twist the images whilst proceeding through the image stack.

#### Segmentation

We have used several open-source software packages for manual segmentation of image stacks. For small image stacks, we have used Reconstruct (Fiala, 2005; Yeh et al., 2009; Hung et al., 2013) and TrakEM2 (Cardona et al., 2012; Meng et al., 2015; Lim et al., 2016) for volumetric reconstruction. For connectomics studies, which requires handling of large image datasets, we have used CATMAID (collaborative annotation toolkit for massive amounts of imaging data; Saalfeld et al., 2009) for skeleton tracing, and VAST (Volume Annotation and Segmentation Tool; Kasthuri et al., 2015) for volume reconstruction.

#### Skeleton Tracing With CATMAID

To generate C. elegans connectomes, we apply skeleton tracing to reconstruct all neurons and their connectivity. Skeleton tracing consists of placing dots, or "nodes," in the center of a

neurite throughout the volume, forming a skeleton as the tracing progresses. Compared to volumetric reconstruction, skeleton tracing allows faster manual reconstruction of the nervous system. With a high-quality dataset, a first larval stage nerve ring (the worm central nervous system) can be manually traced to completion by a well-trained and committed tracer in a few days. As neurons are traced, they are identified based on stereotypic structures and connectivity patterns, along with neurite trajectory and placement, and cell body position (see below). Ambiguities may arise due to artifacts such as section folding or stain precipitation, and can be resolved by completing the tracing of the rest of the neurons in the immediate area. Neurons are identifiable by features distributed throughout the nerve ring.

After neurite tracing is complete, connectors can be placed between nodes of different skeletons to signify chemical synapses and gap junctions. Visualization of neuron skeletons in 3D is often sufficient for assessing the coarse position and process trajectory of individual neurons, as well as the overall architecture of neuropils and ganglia (**Figures 5A,B**). However, substantial morphological information is omitted.

#### Volumetric Segmentation With VAST

To accurately obtain morphological information such as neuron size, shape, and the relative contact area between neurons, volumetric segmentation is necessary. Additional segmentation of intracellular ultrastructure can yield information such as the distribution, morphology, number, and size of microtubules, mitochondria, ER, presynaptic densities, synaptic and dense core vesicles and other vesicular structures. This is useful to understand the cell biology of the neuron (**Figure 5C**).

The VAST software package is capable of segmenting in such a way (Kasthuri et al., 2015). In our hands, VAST has the best performance when handling large datasets like the entire C. elegans nerve ring (**Figure 5D**). Manual volumetric segmentation, however, is very low throughput. Fully automated segmentation methods have been reported, but they have yet to perform well with our C. elegans datasets. We took an alternative, semi-automated approach. In this approach, membrane probability maps were generated from small training stacks (Meirovitch et al., 2016), and nodes that were generated from skeleton tracing were expanded to the calculated membrane boundary to fill the neurite (Meirovitch et al., in preparation). This is followed by manual proof-reading in VAST (**Figure 5D**).

#### Synapse Annotation

Different fixation protocols can lead to differences in the morphology of fixed tissues. Therefore, it is important to adjust criteria for synapse annotation for datasets generated using different fixation protocols and imaging conditions. For example, the slow fixation protocol used for generating the original C. elegans adult wiring datasets was optimized for cell membrane contrast. Fine intracellular ultrastructure was less well preserved, and presynaptic dense projections appear as a dark density close to the membrane, with hard to discern morphology. This makes chemical synapse annotation more prone to staining artifacts. The slow fixation protocol caused shrinkage of neurites, which tore apart weak adhesions between adjacent neurites. Such a distortion could complicate the assignment of postsynaptic partners in polyadic synapses, but highlight gap junctions, which remain intact. Synapse annotation and connectome assembly were carried out cautiously and carefully with these caveats in mind (White et al., 1986). Any reconsideration of these micrographs should involve careful study of the entire dataset and apply similarly rigorous criteria to avoid the "false positive" identification of synapses.

Even with a well-preserved sample that has been fixed using high-pressure freezing and aligned well into a 3D volume, synapse annotation requires training, and includes of element of subjectivity (see below; **Figures 6**, **7**). For a compact nervous system such as C. elegans, where neuron and synapse numbers are small, it is even more pertinent to establish stringent criteria for sample preparation and synapse annotation, and to obtain and compare multiple datasets from isogenic individuals, so that errors can be minimized.

Below we describe the criteria used for synapse annotation in our high-pressure frozen and freeze substituted volumes of the C. elegans nervous system.

#### Chemical Synapses

Caenorhabditis elegans presynapses generally consist of a swelling in the neurite, with a visible electron-dense presynaptic density attached to the plasma membrane marking the active zone, with a cloud of vesicles adjacent to the presynaptic density (**Figures 6A,B**, **7A**). Vesicle clouds often consist of many clear core synaptic vesicles close to the active zone, and a small number of large, dense-core vesicles that reside more peripherally. Vesicle clouds can cover large areas with multiple small presynaptic dense projections, especially in the nerve ring. If the synapse is small, cut at an awkward angle, or if there are artifacts covering or interfering with the putative synapse, assigning whether it is a synapse or not can sometimes be a bit subjective (**Figures 7B,C**). Many synapses are polyadic. Since most synapses in the C. elegans nervous system do not have visible postsynaptic densities, postsynaptic partners are assigned based on their proximity to the presynaptic active zones, which can be a source of subjectivity (**Figure 7D**).

To minimize the problem of subjectivity, our datasets are fully annotated by three independent annotators. Using CATMAID one can assign confidence scores to synapses, with a score of 5 indicating a high level of confidence, and a score of 1 indicating very low confidence. The triplicate annotations are then merged, and every inconsistency between annotators is flagged for discussion. If agreement is not reached by the three annotators after debate, an average of the confidence scores is reported to allow subsequent data users to make their own judgments.

#### Gap Junctions

Gap junctions are notoriously difficult to identify in vEM. There are some morphological criteria that can help identify some with reasonable certainty. A classic gap junction profile includes a close, relatively flat area of membrane apposition of

small synapse (from RIS to RIBL and RMDR). The annotation of this synapse is less confident that the one presented in panel (A). (C) Serial sections of a membrane swelling that is confidently annotated as not-a-synapse. A small density in the membrane of RIBL with sparse vesicles is not a presynaptic specialization. (D) Serial sections through a synapse showing the occasional subjectivity involved in defining postsynaptic partners. While all annotators agreed RMGL was a postsynaptic partner of RIGL, whether SAADL should be included as a postsynaptic partner was cause for debate. White arrowheads indicate the membrane of interest. Scale bars are 500 µm.

limited extracellular space (∼2 nm) across multiple sections, a thicker membrane, with a characteristic sharp zippering of the membranes immediately at the boundaries of the putative gap junction (**Figure 6C**). These features can be quite clear if cut at the perfect angle with thin (30–50 nm) sections, but even in well-stained samples not all gap junctions can be marked unambiguously. Tomography, which acquires images of the same section at different tilt angles to generate a high-resolution 3D volume of the section, helps survey a putative gap junction, but it is unrealistic to apply such an approach to the entire series of the nervous system.

We corroborate our gap junction annotation by comparing patterns across our multiple new datasets and to the original datasets (White et al., 1976, 1986). The slow chemical fixation protocol used for the original adult connectome, while distorting neurite morphology and pulling apart weaker contacts between neurites, allowed strong membrane connections such as gap junctions to be particularly well distinguished. Some of the morphologically identified gap junctions have been functionally validated (Chalfie et al., 1985; Liu et al., 2017). Comparing new and old datasets allows us to refine criteria for gap junction annotation in high-pressure frozen datasets. These criteria are validated by uncovering recurrent gap junction-like structures when comparing the same membranes between neuronal classes across datasets. Because in each sample, the junction between each neuron pair was sectioned from a different angle, stereotypic gap junctions can be confirmed in multiple views. Our approach will likely miss small or sparse gap junctions.

Multiple approaches have been attempted to highlight gap junctions in EM volumes. CLEM (correlative light and electron microscopy), where gap junctions are labeled by immunostaining against one of the C. elegans innexin::GFP fusions, showed promise (Markert et al., 2016, 2017). This approach requires a weak fixation that compromises structural preservation, and it would be difficult to expand this approach to all 25 C. elegans innexins. We and others are working to develop EM preservation protocols to improve gap junction annotation.

#### Neuron Identification

In a large, good quality C. elegans volume, every single cell can be assigned its unique cell name. Each neuron class has been described in such superb detail in The Mind of a Worm (White et al., 1986) that by reading the neuron descriptions while going through the complete EM series, one can identify

cables. The cell bodies are denoted with spheres, and also have stereotypic positions along the body of the worm and relative to each other.

neurons one by one throughout the volume. WormAtlas hosts scanned copies of the neuron pages from The Mind of a Worm that are accessible through a drop-down menu in an internet browser (Altun et al., 2002–2018). Several features indicate neuron identity: cell body position, neurite trajectory, stereotypic neurite placement or morphology and stereotypic connectivity patterns. We found that this stereotypy holds across postnatal developmental stages for most neurons, with a few exceptions.

For example, in the adult ventral nerve cord, VC processes are generally most dorsal, followed by VD, DD, VA, then VB toward the ventral side. Synapses to body wall muscles come from VA, VB, VD, and VC class motor neurons. Among them, VD presynaptic swellings are large, face directly toward the muscle, most of the time without any neurons as dyadic postsynaptic partners (Jin et al., 1999; White et al., 1976, 1986; **Figure 8A**). On the other hand, VA and VB, form NMJs that consist of smaller swellings, are often on the dorsal side of the neurite, and almost always dyadic with DD dendrites, which send spine-like structure toward the NMJ (White et al., 1976, 1986; Jin et al., 1999; White, 2013; **Figures 8B,C**).

Neurite trajectory and process placement are used to further identify neurons. For example, VAs project axons anteriorly from the soma, whereas VB axons project posteriorly. VDs also project their axons anteriorly, but they send a dorsal-projecting commissure at the end of the axon regions. Commissure trajectory (whether it exits the ventral nerve cord from the left or right side) and partners in each commissure bundle further assist cell identification (**Figure 8D**). For example, VD2 runs in a left-handed commissure, always bundled with that of DD1, DA1, and DB2.

These, and other observations, allow one to recognize the "fingerprints" of motor neuron identity. Similar observations and strategies apply to the other neuropils in the worm, such as the dorsal nerve cord, the nerve ring, and the other cords and ganglia of the worm, as well as across different stages of development. Some neurons are not born until later in development (Sulston and Horvitz, 1977), but most neurons have stereotypic features and connectivity across larval stages. A notable exception is the DD motor neuron class, which exhibits extensive remodeling of connectivity during development (White et al., 1978).

#### Assembly of a Wiring Matrix

After obtaining a connectome, we further assess pairwise connections to gauge confidence in biologically relevant connections. Connections between two neurons consisting of many synapses are considered high confidence. A connection is considered uncertain if it consists of very few synapses. When few synapses are observed between neurons, we often observe inconsistency in the existence of the connection across animals. From comparing multiple datasets that we have acquired for the C. elegans nerve ring and ventral ganglion, three synapses seem to be a sensible lower bound on a high confidence connection. Even so, to minimize variability introduced by annotators, and assess true biological variability, acquiring connectomes from multiple animals is advisable.

# PERSPECTIVES

The pipeline described above represents only a starting point for modern high throughput C. elegans vEM. We should expect rapid and substantial improvement both in terms of throughput and quality. Future improvements will include automated image segmentation, synapse annotation and neuron and neurite identification. This will be facilitated by the generation of new C. elegans connectomes as training datasets for machine learning approaches. Incorporating of these improvements will allow not only rapid reconstruction of connectomes from multiple animals, but also facilitate targeted reconstruction of specific segments of the nervous system by computer vision.

The C. elegans nervous system is compact, allowing precise correlation of anatomy (connectome) with membrane physiology (activity and excitability of individual neurons), sign of synaptic communication (neurotransmitter and receptor of individual synapses), and behavior. The delineation of the neurotransmitter type and receptor complement of each neuron (Serrano-Saiz et al., 2013; Pereira et al., 2015; Gendrel et al., 2016), combined with the connectivity, allow for more sophisticated modeling of information flow through the nervous system. Whole brain calcium imaging from fixed and behaving animals allows observation of the activity of functioning neural circuitry (Schrödel et al., 2013; Prevedel et al., 2014; Kato et al., 2015; Nguyen et al., 2016; Venkatachalam et al., 2016), allowing correlation of anatomic and functional connectivity. Performing connectomics on animals with genetic mutations that affect diverse properties of neurons – neuronal fate, synaptic transmission, cell adhesion and signaling – holds the promise of identifying genetic and biochemical pathways that determine connectivity. This system holds a promise to reveal insight on principles of how a connectome leads to hardwired and flexible behaviors (Johnson et al., 1995; Harris-Warrick et al., 1998; Marder and Bucher, 2007; Agnati et al., 2010).

The field of C. elegans connectomics is at a new beginning. Modern techniques now allow us to use connectomics to address questions about the dynamic and comparative structures of complete nervous systems. How does a connectome remodel across development? What sexual dimorphisms are held within a connectome? How do mutations in genes that establish the trajectory of neurite growth, the specificity of synapse partners, and the molecular complement of the plasma membrane, change a connectome? Does a connectome drift with age? How much inter-individual variability is there? Is learning and memory physically manifested within the connectome? What about the influence of environment? How are the behavioral differences between morphologically similar but evolutionarily distinct Caenorhabditis species represented by the connectome? How does a connectome evolve?

Finally, volume EM of C. elegans does not only generate information about the nervous system. Packaged within the small volume, our volumes of the nervous system data also capture other tissues – the skin, gut, musculature, excretory cells, and reproductive system – each with their own exquisite intracellular ultrastructure. All datasets will be useful to the much larger community of biologists.

### AUTHOR CONTRIBUTIONS

fncir-12-00094 March 19, 2019 Time: 14:22 # 15

All authors contributed to developing the approaches described above, as well as writing and editing the manuscript.

#### FUNDING

This work was funded by the Canadian Institutes of Health Research (CIHR-MOP93619 and MOP123250 to MZ), Human Frontier (RGP0051/2014 to MZ and AS), the National Institute of Health (R01-NS-082525 to MZ and AS, and R01-GM-054657 to AC).

# REFERENCES


## ACKNOWLEDGMENTS

We thank our colleagues for helpful discussions and contributions to tool development, including Albert Cardona, Matthew Berck, Daniel Berger, Steven Cook, Richard Fetter, Chi-Yip Ho, Seymour Knowles-Barley, Valeriya Laskova, Jeff Lichtman, Yaron Meirovitch, Marianna Neubauer, Richard Schalek, Christian Stigloher, and Richard Webb. We also thank numerous undergraduate students, particularly WanXian Koh and Maggie Chang for their invaluable contribution to image acquisition and processing. We also thank David Hall and the website, www.wormimage.org, for hosting scans from the MRC/LMB image archive, courtesy of Jonathan Hodgkin and John White, where the scan used in **Figure 2C** is available online. WormImage was supported by a grant to Hall (NIH OD010943). We apologize to those who have made important contributions relevant to this manuscript that could not be cited due to space restrictions. We thank Denmark for producing Daniel Witvliet.

Cardona, A., Saalfeld, S., Schindelin, J., Arganda-Carreras, I., Preibisch, S., Longair, M., et al. (2012). TrakEM2 software for neural circuit reconstruction. PLoS One 7:e38011. doi: 10.1371/journal.pone.0038011




Zhen, M., and Samuel, A. D. T. (2015). C. elegans locomotion: small circuits, complex functions. Curr. Opin. Neurobiol. 33, 117–126. doi: 10.1016/j.conb. 2015.03.009

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Mulcahy, Witvliet, Holmyard, Mitchell, Chisholm, Meirovitch, Samuel and Zhen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Corrigendum: A Pipeline for Volume Electron Microscopy of the *Caenorhabditis elegans* Nervous System

#### Ben Mulcahy <sup>1</sup> \*, Daniel Witvliet 1,2, Douglas Holmyard3,4, James Mitchell <sup>5</sup> , Andrew D. Chisholm<sup>6</sup> , Yaron Meirovitch<sup>7</sup> , Aravinthan D. T. Samuel <sup>5</sup> \* and Mei Zhen1,2,8,9 \*

*<sup>1</sup> Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON, Canada, <sup>2</sup> Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada, <sup>3</sup> Department of Pathology and Laboratory Medicine, Mount Sinai Hospital, Toronto, ON, Canada, <sup>4</sup> Nanoscale Biomedical Imaging Facility, The Hospital for Sick Children, Peter Gilgan Centre for Research and Learning, Toronto, ON, Canada, <sup>5</sup> Center for Brain Science, Department of Physics, Harvard University, Cambridge, MA, United States, <sup>6</sup> Section of Cell and Developmental Biology, Division of Biological Sciences, University of California, San Diego, La Jolla, CA, United States, <sup>7</sup> Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, United States, <sup>8</sup> Department of Physiology, University of Toronto, Toronto, ON, Canada, <sup>9</sup> Department of Cell and*

#### *Approved by:*

*Frontiers In Neural Circuits, Frontiers Media SA, Switzerland*

#### *\*Correspondence:*

*Ben Mulcahy mulcahy@lunenfeld.ca Aravinthan D. T. Samuel samuel@physics.harvard.edu Mei Zhen zhen@lunenfeld.ca*

*Received: 24 February 2019 Accepted: 27 February 2019 Published: 20 March 2019*

#### *Citation:*

*Mulcahy B, Witvliet D, Holmyard D, Mitchell J, Chisholm AD, Meirovitch Y, Samuel ADT and Zhen M (2019) Corrigendum: A Pipeline for Volume Electron Microscopy of the Caenorhabditis elegans Nervous System. Front. Neural Circuits 13:16. doi: 10.3389/fncir.2019.00016* Keywords: *C. elegans*, volume electron microscopy, connectome, nervous system, high-pressure freezing

#### **A Corrigendum on**

*Systems Biology, University of Toronto, Toronto, ON, Canada*

**A Pipeline for Volume Electron Microscopy of the Caenorhabditis elegans Nervous System** by Mulcahy, B., Witvliet, D., Holmyard, D., Mitchell, J., Chisholm, A. D., Meirovitch, Y., et al. (2018). Front. Neural Circuits 12:94. doi: 10.3389/fncir.2018.00094

"Yaron Meirovitch" was not included as an author in the published article.

The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way. The original article has been updated.

Copyright © 2019 Mulcahy, Witvliet, Holmyard, Mitchell, Chisholm, Meirovitch, Samuel and Zhen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Methodological Improvements With Conductive Materials for Volume Imaging of Neural Circuits by Electron Microscopy

#### Huy Bang Nguyen1,2,3 , Truc Quynh Thai 1,2 , Yang Sui 2,4 , Morio Azuma<sup>4</sup> , Ken Fujiwara<sup>4</sup> and Nobuhiko Ohno1,4 \*

<sup>1</sup>Division of Neurobiology and Bioinformatics, National Institute for Physiological Sciences (NIPS), Okazaki, Japan, <sup>2</sup>Department of Anatomy and Structural Biology, Interdisciplinary Graduate School of Medicine and Engineering, University of Yamanashi, Chuo, Japan, <sup>3</sup>Department of Anatomy, Faculty of Medicine, University of Medicine and Pharmacy (UMP), Ho Chi Minh City, Vietnam, <sup>4</sup>Department of Anatomy, Division of Histology and Cell Biology, School of Medicine, Jichi Medical University, Shimotsuke, Japan

Recent advancements in electron microscope volume imaging, such as serial imaging using scanning electron microscopy (SEM), have facilitated the acquisition of threedimensional ultrastructural information of biological samples. These advancements help build a comprehensive understanding of the functional structures in entire organelles, cells, organs and organisms, including large-scale wiring maps of neural circuitry in various species. Advanced volume imaging of biological specimens has often been limited by artifacts and insufficient contrast, which are partly caused by problems in staining, serial sectioning and electron beam irradiation. To address these issues, methods of sample preparation have been modified and improved in order to achieve better resolution and higher signal-to-noise ratios (SNRs) in large tissue volumes. These improvements include the development of new embedding media for electron microscope imaging that have desirable physical properties such as less deformation in the electron beam and higher stability for sectioning. The optimization of embedding media involves multiple resins and filler materials including biological tissues, metallic particles and conductive carbon black. These materials alter the physical properties of the embedding media, such as conductivity, which reduces specimen charge, ameliorates damage to sections, reduces image deformation and results in better ultrastructural data. These improvements and further studies to improve electron microscope volume imaging methods provide options for better scale, quality and throughput in the three-dimensional ultrastructural analyses of biological samples. These efforts will enable a deeper understanding of neuronal circuitry and the structural foundation of basic and higher brain functions.

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Kea Joo Lee, Korea Brain Research Institute, South Korea Adrian Andreas Wanner, Princeton University, United States

> \*Correspondence: Nobuhiko Ohno oonon-tky@umin.ac.jp

Received: 04 June 2018 Accepted: 13 November 2018 Published: 23 November 2018

#### Citation:

Nguyen HB, Thai TQ, Sui Y, Azuma M, Fujiwara K and Ohno N (2018) Methodological Improvements With Conductive Materials for Volume Imaging of Neural Circuits by Electron Microscopy. Front. Neural Circuits 12:108. doi: 10.3389/fncir.2018.00108

Keywords: scanning electron microscopy, volume imaging, charging, ketjen black, conductive resin

# INTRODUCTION

The brain is composed of circuits of neurons connected to one another by neurite projections, which enables information processing in the nervous system. Impairment of neural circuitry is associated with psychiatric and neurological disorders, and a complete understanding of the wiring diagram of neuronal connections, termed the ''connectome,'' will provide important clues to understand brain functions and develop treatments for psychiatric and neurological disorders (Filippi et al., 2013; Deco and Kringelbach, 2014; Fornito et al., 2015). To completely understand neural circuitry, multiple imaging approaches are needed to analyze various brain structures (Le Bihan et al., 2001; Fenno et al., 2011; Grienberger and Konnerth, 2012; Lichtman et al., 2014; Ohno et al., 2016). Light microscopic technologies have enabled high-throughput and detailed analyses of neuronal circuits at a very large scale (Wilt et al., 2009; Osten and Margrie, 2013). In addition, the development of cell-specific labeling with genetically encoded tags led to marking of brain cells with different colors and tracking of specific neuronal projections at the whole-brain level (Gong et al., 2003; Livet et al., 2007). Studies on such ''mesoscopic connectome'' achieved big datasets and demonstrated the physical and functional connections among neurons which can span the whole brain, but a deeper understanding on neuronal circuitry has been hampered by several factors (Ohno et al., 2016). Among them, one critical factor of light microscopic approaches is the difficulty to ensure synaptic connections of fine projections, because the resolution of light microscopy is limited. The processes of neurons can be ∼50 nm in diameter, and the neck of the dendritic spines can be even thinner (Briggman and Bock, 2012). These structures are too small to resolve with light microscopes for volume imaging of the brain. To overcome this problem, the standard approach is electron microscopic observation at the level of individual synapses, which unequivocally visualize fine projections and physical connections among neurons through synapses using serial section images at the ultrastructural level (Palay, 1958; Brightman and Reese, 1969). Serial electron microscope images and reconstruction of three-dimensional ultrastructural information are powerful approaches to understand the neuronal connectivity of complex brain architectures.

The three-dimensional reconstruction of biological samples has been made possible using serial ultrathin sections observed by scanning (SEM) or transmission electron microscopy (TEM; Harris et al., 2006; Bock et al., 2011; Briggman and Bock, 2012). The throughput of these microscopy techniques has recently increased significantly (Briggman and Bock, 2012). In the case of SEM, new section collection procedures such as focused ion beam SEM (FIB-SEM; Knott et al., 2008), serial block-face SEM (SBEM or SBF-SEM; Denk and Horstmann, 2004) and automated tape-collecting ultramicrotome (ATUM; Hayworth et al., 2014) are revolutionizing the field of volume electron microscopy. These new TEM- and SEM-based approaches are often complementary and differ in resolution, throughput, sample types and post-acquisition image alignment. In this context, the SEM-based methods have recently advanced our understanding of three-dimensional structures in various organelles, cells, tissues and organisms in life science and clinical medicine, including large scale neural wiring maps of various organisms (Briggman et al., 2011; Kubota et al., 2011; Holcomb et al., 2013; Terasaki et al., 2013; Ohno et al., 2014; Ichimura et al., 2015; Kasthuri et al., 2015; Katoh et al., 2017). In addition, new devices to image large tissue areas, such as multi-beam SEM, have been developed and facilitated data acquisition from very large tissues such as whole brains (Eberle et al., 2015).

At the same time, methods using SEM for serial image acquisition generally require specific sample preparation techniques, in particular for the acquisition of large stacks of serial images with satisfactory contrast for subsequent tissue annotation, segmentation and analysis. For example in SEM imaging, the available parameter range for beam irradiation, e.g., beam current and voltage, is limited by insufficient conductivity of the biological samples. In order to acquire high contrast and high quality images, it is preferable to have sufficient deposition of heavy metals in the sample. To overcome these problems, extensive efforts have been made to improve throughput and image quality from SEM-based imaging in large tissue volumes.

Here, we review recent methodological advances in volume imaging using SEM with particular emphasis on newly developed approaches and conductive materials used in sample preparations and tissue embedding for serial sectioning and imaging, which will contribute to our understanding of the connectome in different organisms.

### BASIC METHODOLOGY OF SAMPLE PREPARATIONS AND DATA ACQUISITION FOR VOLUME IMAGING USING SEM

In SEM, images are produced by focusing electron beams, scanning over the bulk specimens and detecting ultrastructural information of the specimen surface using secondary or backscattered electrons (BSE). But when BSE and/or secondary electrons derived from the flat block/section surface of resin-embedded tissue samples are detected in SEM, images which are similar to those obtained from the embedded samples in TEM can be acquired (Richards and Gwynn, 1995; Wergin et al., 1997). When low electron energies are used for the block/section face imaging with SEM, the BSE contain information only from near the surface of the embedded samples (Hennig and Denk, 2007), which can result in a depth resolution of <30 nm depending on the energy of landing electrons (Denk and Horstmann, 2004; Knott et al., 2008). For these reasons, observation of block/section faces in SEM facilitated serial image acquisition for large volume 3D reconstruction of the fine processes and synaptic connections of the nervous system, but requires specific sample preparation which can be distinct from conventional approaches for TEM or SEM observation.

Biological samples are mostly composed of light elements such as carbon, oxygen, hydrogen and nitrogen, and therefore imaging non-conductive biological specimens with SEM is often hampered by artifacts associated with charging and insufficient contrast (**Figure 1**). Various efforts have been made to achieve

higher contrast and better resolution for volume imaging of biological specimens under SEM. These efforts consist of modifications of different steps including post-fixation, staining, embedding and image acquisition (**Figure 2A**).

Most tissue preparation procedures for serial imaging with SEM include common fixation with chemicals such as aldehydes and en bloc metal staining involving osmium, uranium and lead. Following these post-fixation and staining procedures, the small pieces of tissue blocks are embedded in common resins. Efficient acquisition and analyses of serial electron microscope images are facilitated by higher contrast in cells and organelles, and therefore the procedures are designed to achieve enhanced deposition and en bloc staining of metals, and are now widely used to observe membranous organelles and cellular morphology (**Figure 2**; Deerinck et al., 2010; Tapia et al., 2012; Ohno et al., 2015; Yin et al., 2016). The en bloc preparation is essential for block-face imaging such as SBEM and FIB-SEM, since the block-face is imaged immediately after exposure. The en bloc staining is also used for imaging of the sections in ATUM or TEM because of the benefits of relatively even staining and more metal deposition for increased conductivity, which results in improved contrast. As a consequence, lower beam doses can be used for imaging which reduces radiation damage. The methods to enhance membrane contrast used heavy metal deposition to cellular membranes (Seligman et al., 1966; Karnovsky, 1971; Walton, 1979). These methods have drawbacks, such as areas with limited staining and tissue destruction from the generation of nitrogen gas. Inhibition of nitrogen bubble formation along with staining of much wider areas was achieved in a method termed BROPA using the additional solvent and pyrogallol (Mikula and Denk, 2015). In addition, another method employed sequential modification of common preparation procedures to facilitate homogeneous metal deposition (Hua et al., 2015). These methods addressed the problems of stain penetration depth by modifying sample preparation methods for observation of large areas in brain tissues (Hua et al., 2015; Mikula and Denk, 2015). Collectively, these approaches including alternative reagents and devices which are combined with historical methods became powerful options for efficient acquisition of high quality datasets from various types of specimens including large brain tissues.

The development of improved staining procedures has been accompanied by the development of new in-chamber techniques for charge compensation that modify the acquisition condition inside of the SEM chambers. The next section introduces some of such mechanical improvements, which are termed ''In-Chamber Techniques for Charge Compensation'' in this review.

# IN-CHAMBER TECHNIQUES FOR CHARGE COMPENSATION

Multiple approaches have been proposed which can modify the circumstances or samples in SEM chambers in order to reduce artifacts and acquire data with higher quality. For example, observation with SEM under low vacuum conditions, such as variable-pressure SEM, has often been used to acquire images from samples with problems of charging. However, these observation methods generally involve electron-gas interactions and electron beam scattering and can reduce the signal-tonoise ratio (SNR) and worsen image quality (Mathieu, 1999). To overcome the observation problems in low vacuum conditions, focal gas injection onto the block-face was used for SBEM imaging, which was termed focal charge compensation (FCC) system (Deerinck et al., 2017). This approach substantially improved charging and enabled image acquisition from samples prepared without dense heavy-metal staining. In FCC, a retractable application nozzle, mechanically coupled to the reciprocating action of the built-in ultramicrotome, was paired with a gas injection valve. The system enables the application of nitrogen gas precisely over the block-face during imaging while the high vacuum of the specimen chamber is maintained. The locally applied nitrogen gas molecules are ionized, approach the sample surface, and neutralize electrons, which charges the sample surface (Thiel et al., 1997). As a result, the FCC system does not interfere with the operation of the SBEM, but greatly reduces image artifacts in the stacks of charge-prone specimens. The addition of FCC does not affect the total time of data acquisition, but can reduce the time by allowing shorter dwelling times due to the improved SNR. Quantitatively, when

for TEM (B), the procedure for volume imaging clearly visualized histological features (C), such as myelinated nerve fibers (D, arrows). Mo, molecular layer; Gr, granular layer; WM, white matter. For SEM imaging, cellular structures, such as myelin membranes (E, arrowhead) and mitochondria (E, arrows), were clearly observed in samples with dense heavy metal staining. N, nucleus. Bars: 50 µm (B,C), 12.5 µm (D), 5 µm (E) or 500 nm (E, inset). Images were adapted from Ohno et al. (2015) with permission.

increasing the accelerating voltage from 2.5 keV to 4.0 keV (60%) and increasing the pixel dwell time from 1 µs to 4 µs (4×), SNR was 28% lower using variable pressure-SEM than FCC, and the resolution obtainable by FCC was nearly the same as measured using high vacuum (Deerinck et al., 2017). Taken together, FCC is a promising approach to observe chargingprone samples by modifying SBEM system but not samples themselves.

In addition to alterations of the sample atmosphere, beam deceleration can significantly improve the contrast and resolution of images in block-face imaging of biological samples in SEM under low landing energy levels and a low beam current (Ohta et al., 2012; Titze and Denk, 2013). In the beam deceleration approach, the specimens are held at a negative bias voltage, and the electrons leaving the column are decelerated before reaching the specimens. The beam deceleration system has multiple advantages including improved detection of signals from negatively biased specimens and better resolution by very low landing energy of the incident electrons. Although the sample conductivity is critical for the beneficial effects of beam deceleration, imaging of such conductive samples at high spatial resolution could be significantly facilitated by applying beam deceleration upon imaging in SEM.

Treatments to increase the surface conductivity of samples have been widely used in observation of biological specimens in SEM. Attempts to apply this concept to the SBEM imaging have been made in SEM chambers by automated block-face metal coating, and charging could be significantly improved during SBEM imaging (Titze and Denk, 2013). In this study, the surface of the imaged blocks was covered with thin (1–2 nm) metallic films composed of chromium or palladium using an electron beam evaporator that is integrated into the microscope chamber. In this system, the conductivity of the surface was increased by the thin metallic films prior to each cycle of imaging. The reduction in SNR caused by the metallic film is smaller than that caused by the widely used low-vacuum method. So the film coating results in better signal than the low-vacuum method, but still fully compensates any charging artifacts. In addition, one big advantage of this in-chamber coating method is that it allows detection of secondary electrons, which in turn enables much higher acquisition speeds than BSE-based imaging. The sample whose surface was 12 mm across could be coated and imaged without charging effects at beam currents of 25 nA, and more than 1,000 serial images could be acquired under the automated cut/coat/image cycles. However, one critical drawback of this approach is the requirements for the specific devices which enable in-chamber coating of the samples with the metallic films.

Another method using plasma etching prior to imaging has been used to remove contaminants and enhance contrast in serial image acquisition using ATUM (Morgan et al., 2016). Plasma cleaning has been used to remove contaminants, and this would be helpful in ATUM since it is possible that various contaminants which perturb image acquisition can be attached on the surface of sections during sectioning, mounting on the tape and subsequent preparation for imaging. In addition, plasma etching can be used to enhance contrast in SEM imaging using secondary electrons, presumably due to the removal of specimen components near the specimen surfaces and generation of surface unflatness (Hukui, 1996). The plasma etching could be beneficial in serial image acquisition in SEM when the secondary electrons are used for imaging of the block/section faces.

The modifications of physical properties, such as sample conductivity, and improvements in observation methods have improved image quality. Dense deposition of heavy metals on specimens is beneficial for SEM imaging because it increases conductivity and improves the SNR of samples. Increasing the conductivity of the embedding media in addition to specimen conductivity could be beneficial for the observation of non-conductive biological materials. Different materials and methods for specimen embedding have improved in the life sciences and clinical medicine, and in the next section we discuss several recent studies that modified embedding procedures and media in order to facilitate serial image acquisition using SEM.

## IMPROVEMENT OF EMBEDDING METHODS FOR CHARGING COMPENSATION

Developing new embedding media for electron microscope imaging requires consideration of the physical properties associated with the imaging procedures, such as stability in sectioning and the degree of deformation under electron beam irradiation. Sectioning with a diamond knife requires careful consideration of the physical properties of the target materials, which significantly affect knife lifetimes (Hashimoto et al., 2016). Imaging and cutting conditions, such as sample temperature, cutting speed, cutting thickness and size, knife shape and knife temperature, also affects knife lifetimes, but material hardness is the most important factor for image quality. In addition, electron beam irradiation causes thermal damage to the resin, and artifacts occur from resin shrinkage and deformation, which can be ameliorated by cooling the samples to cryotemperatures (Luther, 2006). These artifacts also depend on electron beam properties, such as acceleration voltage strength and electron current, which can be evaluated with sections mounted on conductive tapes (Kubota et al., 2018). However, damage and structural deformation of the resin-embedded samples from electron beam irradiation may also be affected by the properties of the stained and embedded tissues.

Historically, various resins have been used for electron microscope observation of biological specimens. Early resins, such as methacrylates, developed for ultrathin sectioning and epoxy resins developed later resulted in less structural changes (e.g., shrinkage) upon curing and high stability during ultrathin sectioning and electron beam irradiation (Glauert and Glauert, 1958; Luft, 1961). Different types of resins, including water soluble and hydrophilic resins, have been developed and used for electron microscope observation, and these resins have unique properties which are suitable for different target samples and staining and observation methods (Staeubli, 1963; Leduc and Bernhard, 1967). The artifacts from shrinkage and deformation typically include depth-direction and planar shrinkage during electron beam irradiation. This type of shrinkage is obvious during SBEM imaging, when there is local failure of physical slicing in areas with intensive irradiation of the electron beam for focusing. TEM-based evaluation revealed that the stability against electron beam artifacts varies among different resins (Kizilyaprak et al., 2015). Interestingly, maximal resistance against electron beam damage is achieved by a mixture of different resins, but the exact mechanisms of improved resistance remain unclear. These studies provide options for the optimization of embedding media, which enables better stability for the imaging of biological specimens with intensive beam irradiation.

Generally, resins used for electron microscope observation have distinct physical properties compared with adjacent embedded biological specimens. Most resins are composed of light elements, which have lower conductivity than that of the embedded specimens, particularly when the specimens are densely stained with heavy metals. In addition, the hardness of the resin is altered in regions with biological specimens. These problems could be potentially solved by modifying the undesired physical properties of the resins around the samples. ''Fillers'' have long been used to modify the physical properties of base materials (e.g., plastics, concrete), such as electrical conductivity and hardness. It is therefore possible that those conventional or new filler materials have beneficial effects on physical properties of the resins and facilitate serial image acquisition in SEM by reducing artifacts. Recent studies have started exploring this possibility and found some promising results using different types of ''fillers'' beneficial for the serial image acquisition in SEM (**Figure 3**).

To facilitate SEM imaging, biological specimens that are not related to the experiment are embedded with the target samples. These biological specimens are used as a kind of ''filler material,'' which modifies the physical properties of

the surrounding resin. Biological filler materials are stained and prepared similarly to the target tissues, and therefore the physical properties of the filler and target tissues are similar. One example of the biological filler materials is tissue from the mouse brain, which was embedded with larval zebrafish for serial sectioning by ATUM (**Figure 3A**; Hildebrand et al., 2017). Homogenous hardness and stability of sample blocks facilitate repeated serial sectioning by ATUM and prevent heterogeneous shrinkage, deformation and folding of sections. Small larval zebrafish samples were post-fixed, stained en bloc, embedded into resin blocks, and finally surrounded by mouse brain tissues for stabilization during sectioning. In this study, 17,963 sections at 60 nm thickness were acquired in ATUM for serial image acquisition. In total, 244 (1.34%) sections were lost and 283 (1.55%) were partially lost, while no two adjacent sections were lost. Although more quantitative analyses on physical properties of the brain ''filler materials'' are required, it is possible that filler biological samples treated and embedded similarly to the target specimens substantially improve production of and imaging from serial ultrathin sections.

Aggregated unicellular organisms can also be used as biological support materials. C. elegans was embedded with E. coli or yeast cells during cryofixation to facilitate handling and localization of the samples (**Figure 3A**; Möller-Reichert et al., 2003; Hall et al., 2012). The samples were still surrounded by biological material during the subsequent tissue preparation procedures, including freeze-substitution, and the resins surrounding the C. elegans sample at the time of observation were filled with biological material that was stained and embedded at the same time (Hall et al., 2012). Because the sample has biological components enriched with metal deposition, the regions occupied by the organism had different physical properties from bare resin. These types of approaches using biological filler materials are promising options to facilitate serial image acquisition using SEM.

Besides biological tissues, other filler materials have been used to modulate the physical properties of resins. For example, the addition of metal particles alters properties such as the electrical and thermal conductivity of plastic (Bhattacharya and Chaklader, 2006). Metallic particles are embedded to image brain tissues by SBEM, where samples with low metal deposition or areas of non-conductive embedding media outside of tissues are susceptible to charging artifacts (**Figure 3B**). Epo-Tek and Epon embedding (EE embedding) uses commercially available epoxy glue containing silver particles, and this technique enables embedding conductive resins with metal particles in the vicinity of target brain samples (Wanner et al., 2016). Although areas with less heavy metal deposition have charging artifacts, this approach facilitated serial imaging of brain samples under high-vacuum conditions. Serial images of 4,750 sections at 25 nm thickness could be acquired in this study, and only one section was lost, proving that EE-embedding is a promising approach and considered to be suitable for ultra-thin sectioning. Recent studies used the same approach, and one of them acquired 11,416 slices of tiled images at 10 nm × 10 nm × 25 nm resolution in SBEM (Genoud et al., 2018; Titze et al., 2018). This approach is further evidence that using conductive metal particles around target samples facilitates serial SEM image acquisition, especially SBEM, which is readily affected by charging artifacts.

Increasing conductivity without influencing the contrast of the embedding medium can be achieved by using conductive materials composed of light elements. Carbonbased materials have relatively high conductivity, and for example, conductive tape covered by carbon nanotubes was used for imaging with ATUM and SEM (Kubota et al., 2018). Carbon black fillers have been used to modify the physical properties of plastics and polymers, such as electrical conductivity and material toughness, and therefore the addition of carbon black to embedding media may improve conductivity without affecting contrast (Yacubowicz et al., 1990; Chekanov et al., 1999; Novák et al., 2005; Domun et al., 2015).

One type of commercially available carbon black, called Ketjen black, reduces the resistance of base resins without altering mechanical stability (Kim et al., 2008). The reduction in resistance depends on the amount of the carbon added to the resin, but Ketjen black increases conductivity at relatively low concentrations (Connor et al., 1998; Chekanov et al., 1999; Balberg, 2002). A more structured carbon black, such as Ketjen black, forms larger agglomerates, which results in networks of conductive fillers with small gaps and improves the conductivity of non-conductive base materials even at lower concentrations (Balberg, 2002). Together, these studies suggest that Ketjen black is the most suitable carbon black for electron microscopy because it efficiently reduces resistance while maintaining mechanical stability.

Indeed, conductive resin produced by Ketjen black is useful for imaging with SBEM under several different sample

FIGURE 4 | Higher concentrations of Ketjen black increased conductivity and viscosity of the conductive resin. A light microscope image of the section obtained from a mouse brain tissue embedded in the conductive resin shows dark granular aggregates of carbon (A, asterisks) in the vicinity of the tissue (A, arrowheads) but little penetration into the tissues (A, arrow). BV, blood vessel. Scanning electron microscope block-face images show abnormal contrast was prominent in resin without carbon black (B, asterisks) but eliminated in resin with Ketjen black (C, asterisks). The schematic graph shows resistance of the block decreased (D, blue arrow) and viscosity of uncured resin increased (D, red arrow) when conductive resin contained increasing concentrations of Ketjen black. Bars: 20 µm (A) or 10 µm (B,C). Images (A–C) were adapted from Nguyen et al. (2016) with permission.

preparations (**Figure 3C**; Thai et al., 2016). Ketjen black particles are too large to enter cells and tissues, and therefore cannot penetrate deep inside tissues even when well dispersed in base resins and incubated with samples for a long time (**Figure 4A**). However, the addition of conductive materials in the resin substantially diminishes charging of the samples and resins for SBEM imaging (**Figures 4B,C**; Nguyen et al., 2016). In addition, embedding Ketjen black into resin ameliorates image deformation caused by insufficient sample conductivity, improves slicing quality and facilitates acquisition of serial images at higher resolution (Nguyen et al., 2016). Conductive resins based on carbon black fillers substantially reduce charging artifacts, result in better ultrastructural data and are applicable to various types of tissues in SBEM imaging (Nguyen et al., 2016, 2018; Thai et al., 2016, in press; Yoshimura et al., 2017; Saitoh et al., 2018; Sawada et al., 2018; Takeda et al., 2018).

# FUTURE PERSPECTIVES OF THE EMBEDDING MEDIA FOR VOLUME IMAGING

Although the currently available conductive resins have beneficial effects in volume imaging with SEM, there are several drawbacks in their usage. For example, the amount of the carbon black that can be added is limited partly by the increased viscosity of uncured resin (Lee, 1992; Nguyen et al., 2016). Addition of more Ketjen black into the resin results in further reduction of resistance, and also further increase in viscosity (**Figure 4D**), which impairs sample embedding. Therefore, the amount of Ketjen black that can be added to the resin is limited by the maximum viscosity acceptable for embedding. It is important to choose the concentration of Ketjen black where the resistance of the cured block and viscosity of the uncured resin are at acceptable levels. This issue might be partly addressed by selection of the base resins with lower viscosity. At the same time, selection of appropriate base resins and embedding media which will reduce deformations from electron irradiation facilitates better serial image acquisition (Kizilyaprak et al., 2015). Future studies might elucidate the optimal selection of the embedding media with acceptable viscosity and deformations, which would significantly facilitate production of conductive resins and acquisition of high quality data from biological specimens.

In addition, carbon-based resins in general require careful dispersion of the carbon powder during mixing with the base resin. Suboptimal dispersion impairs conductivity of the resins produced with the conductive fillers. Metallic filler materials would also have similar requirement of dispersion, and usage of premixed products which are commercially available reduced the burden of manual dispersion of the fillers (Wanner et al., 2016). Development and distribution of such premixed products would be preferred for the future conductive embedding media with conductive filler particles used for electron microscopic imaging.

Lastly, the reduced transparency or complete opacity of the samples applies not only to carbon-filled resins, but also to the other filler materials. These issues are attributable to the non-transparent properties of the filler materials added to the base resins. Although improvement in the conductivity of the base resin could not be achieved so far by addition of transparent and conductive ionic liquid (Nguyen et al., 2016), exploration and application of transparent conductive materials might lead to development of conductive embedding media which is preferred for the identification and orientation of the embedded samples without exposure.

#### REFERENCES


#### CONCLUDING REMARKS

During the past several years, there have been rapid methodological advancements for volume imaging of large biological specimens with SEM including increased options for staining, embedding and observation. Conductive materials are a unique option for better quality of images by reducing the charging of sample blocks in serial image acquisition with SEM, which is prone to charging artifacts. The available methods still have many limitations, and future studies involving the development and application of novel materials and a combination of available modifications may lead to better scale, quality, and throughput for the three-dimensional ultrastructural analyses of biological samples. These efforts will enable a deeper understanding of neural circuitry and provide the structural foundation for basic and higher brain functions.

#### AUTHOR CONTRIBUTIONS

All authors contributed to the writing and approved the final version of the manuscript.

#### FUNDING

This work is partly supported by Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Number 16K12345 (to NO), Research Grant from National Center of Neurology and Psychiatry (No. 30-5 to NO), Cooperative Research Program of ''Network Joint Research Center for Materials and Devices'' and Cooperative Study Programs of National Institute for Physiological Sciences (to NO).

#### ACKNOWLEDGMENTS

We thank Dr. Toshiyuki Oda in University of Yamanashi for providing some images. We would like to thank Setsuro Fujii Memorial, Osaka Foundation for Promotion of Fundamental Medical Research, for providing the support.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor declared a shared affiliation, though no other collaboration, with several of the authors HN, TT and NO at time of review.

Copyright © 2018 Nguyen, Thai, Sui, Azuma, Fujiwara and Ohno. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Multi-Beam Scanning Electron Microscopy for High-Throughput Imaging in Connectomics Research

#### Anna Lena Eberle\* and Dirk Zeidler

*Carl Zeiss Microscopy GmbH, Oberkochen, Germany*

Major progress has been achieved in recent years in three-dimensional microscopy techniques. This applies to the life sciences in general, but specifically the neuroscientific field has been a main driver for developments regarding volume imaging. In particular, scanning electron microscopy offers new insights into the organization of cells and tissues by volume imaging methods, such as serial section array tomography, serial block-face imaging or focused ion beam tomography. However, most of these techniques are restricted to relatively small tissue volumes due to the limited acquisition throughput of most standard imaging techniques. Recently, a novel multi-beam scanning electron microscope technology optimized to the imaging of large sample areas has been developed. Complemented by the commercialization of automated sample preparation robots, the mapping of larger, cubic millimeter range tissue volumes at high-resolution is now within reach. This Mini Review will provide a brief overview of the various approaches to electron microscopic volume imaging, with an emphasis on serial section array tomography and multi-beam scanning electron microscopic imaging.

Keywords: 3D volume EM, scanning electron microscopy, high-throughput imaging, high-content imaging, multibeam

# INTRODUCTION—THE DECADE OF BRAIN IMAGING

Just as "genomics" deciphers complete genomes of live beings since two decades ago, a new field aiming at fully deciphering the circuitry of the nervous system on a large scale is emerging now. This field has accordingly been coined "connectomics" (Sporns et al., 2005), and has similarly been able to attract growing research interest over the past years (BRAIN, 2013).

First descriptions of neuronal morphology and the idea that individual neurons are anatomically connected is already a century old (Cajal, 1899). In more recent times, information about the intrinsic connectivity of the nervous system has been obtained also by in vivo approaches such as electrophysiology (Mandonnet et al., 2010), diffusion tractography [DTI (Mori and Zhang, 2006; Hagmann et al., 2007; Guye et al., 2008)], functional magnetic resonance imaging [fMRI (Mamedov et al., 2012; Lowe et al., 2016)] or optical imaging (Petroll et al., 1994; Kleinfeld et al., 2011; Chen et al., 2015). None of these methods, however, reveals information about the neuronal connections

#### Edited by:

*Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan*

#### Reviewed by:

*Lester Melie-Garcia, Lausanne University Hospital (CHUV), Switzerland Nuno Miguel M Amorim Da Costa, Allen Institute for Brain Science, United States Naomi Kamasawa, Max Planck Florida Institute for Neuroscience (MPFI), United States*

#### \*Correspondence: *Anna Lena Eberle*

*anna-lena.eberle@zeiss.com*

Received: *18 May 2018* Accepted: *23 November 2018* Published: *11 December 2018*

#### Citation:

*Eberle AL and Zeidler D (2018) Multi-Beam Scanning Electron Microscopy for High-Throughput Imaging in Connectomics Research. Front. Neuroanat. 12:112. doi: 10.3389/fnana.2018.00112* at their ultrastructural level, which is expected to reveal deeper insights into how a nervous system functions (Morgan and Lichtman, 2013). The ambitious approach of connectomics is to gain an understanding of the circuitry of the brain by mapping every single component and trace every connection of a certain volume of brain tissue (Lichtman and Denk, 2011).

The actual dimension of the volume of interest depends on a number of factors, for example on the question to be answered, the model organism or the neuronal system to be investigated. Originally describing the connections of a complete nervous system, different methodologies with different spatial resolutions lead to the differentiation of the connectome into different scales: the microscale, the mesoscale, and the macroscale connectome. The macroscale connectome parcellates the brain with millimeter resolution into anatomically or functionally distinct brain regions (Fellemann and Van Essen, 1991) and usually is assessed by non-invasive measures, such as DTI (Beckmann et al., 2009) or fMRI (Nelson et al., 2010). On the mesoscale, neuronal populations with distinct anatomical (Mountcastle, 1997) or functional (Callaway and Katz, 1990) features are described at a spatial resolution of hundreds of micrometers (Zhao et al., 2005). Mapping the finest details on the cellular level corresponds to the microscale connectome (Bargmann and Marder, 2013, Schröter et al., 2017). Bridging the gaps between these different scaling levels might enable a more unified, multiscale description of the connectome.

With specific labeling (Young and Feng, 2004; Lakadamyali et al., 2012), light microscopical methods characterize individual neurons very well, while still offering relatively large sample volumes. The surrounding, unlabeled context, however, is usually missing. Even though the resolution limit of light microscopes has been improved over decades to well below the light wavelength (Klar et al., 2000; Betzig et al., 2006; Hell, 2007), the nervous system contains structures that are not easily resolvable with them. Hence, electron microscopy (EM) has become a commonly used technique to resolve ultrastructural details on a cellular scale.

If acquisition of a whole volume is required, the volume of interest needs to be sectioned physically (Ware, 1975) or optically (Minsky, 1988; Denk et al., 1990; Neil et al., 1997; Huisken et al., 2004; Santi, 2011) before it can be imaged in 2D. The 2D data sets are stacked and aligned in the third dimension, and all individual compartments are usually labeled on the 2D images first and then tracked throughout the volume. This method in the end yields a "dense reconstruction" (Kasthuri et al., 2015) and will answer important questions about the general principles how neurons connect: Does a minimal repetitive circuitry unit or motif exist? How do different brain regions compare, and how does this relate to differences between individuals and species (Womelsdorf et al., 2014, Borst and Helmstaedter , 2015, Lee et al., 2016)? Based on such data for healthy brain tissue, the next step is to learn about deviations in pathological conditions. Are there structural changes in brains with neurodegenerative diseases, and how does this knowledge help to develop novel treatments? However, before such information can be derived, imaging of the volume of interest needs to be accomplished.

# OVERVIEW OF DIFFERENT APPROACHES TO VOLUME ELECTRON MICROSCOPY

Over the last decades, several methods for volume electron microscopy have been established (Briggman and Bock, 2012; Kremer et al., 2015; Mikula and Denk, 2015; Titze and Genoud, 2016). The nature of the experiment determines which method is optimally suited.

One main differentiator is which part of the tissue block is imaged—the cut-off and collected ultra-thin section or the freshly exposed block surface after a cut. The main advantage of collecting serial sections is that the sample is preserved and can be imaged repeatedly. Reconstruction of the volume after imaging is challenging, as the data needs to be corrected for distortion and translation occurring during the cutting process (Saalfeld et al., 2010). The section thickness is limited down to ∼30 nm, leading to non-isotropic voxels when images are acquired with a smaller lateral pixel size. With block-face imaging, the reconstruction of the final data set needs less distortion and translation corrections, because the acquired area is always the same in shape and position. As the sample is lost in the sectioning process, advanced control of the imaging step has to ensure each section is acquired with sufficient quality before moving on to the next cutting step (Binding et al., 2013).

Classically, serial ultrathin sections have been prepared using an ultramicrotome, followed by manually placing them onto copper grids for imaging in a transmission electron microscope [TEM (Harris et al., 2006)]. Recent developments regarding automation on the sample handling as well as the imaging part enabled relatively large-scale sample volumes to be imaged and reconstructed (Zheng et al., 2018). However, the standard TEM grid ultimately limits the size of the accessible volume to a maximum of 1 × 2 mm (Briggman and Bock, 2012). For samples exceeding this size, placement of the sections on a solid substrate is necessary, which in turn requires imaging in a scanning electron microscope (SEM). In principle, classical sample preparation schemes suitable for SEM imaging can be used (Echlin, 2009), however, such protocols might need to get slightly adapted to accommodate larger tissue volumes (Hua et al., 2015; Mikula and Denk, 2015).

For preparation of large series of consecutive sections, the automated ultramicrotome [ATUMtome (Hayworth et al., 2006; Schalek et al., 2011)] uses a conveyer belt type mechanism to automatically pick up sections on a tape right after they have been cut. ATUMtome has been reported to reliably collect thousands of consecutive sections (Hayworth et al., 2014), enabling sectioning of large sample volumes. Several tape materials with different physical properties have been evaluated so far (Kubota et al., 2018). For manual preparation of a small to medium number of sections, a micromanipulator setup with an advanced substrate holder [ASH (Spomer et al., 2015)] is sufficient. This is especially useful with small samples and for the preparation of ribbons of sections (Wacker et al., 2015). Sections can be placed directly onto a silicon wafer as substrate, which is advantageous for imaging in a SEM, or onto indium tin oxide coated coverslips, such that imaging with light microscopes is possible as well. Further developments aim at more efficient handling and placement of serial sections. For example, it has been demonstrated that sections can be controlled magnetically while still floating in the water bath after cutting (Templier, 2017). By tracing each individual slice to its position on the final substrate, this technique allows dense packing of sections onto the wafer, increasing the degree of automation by reducing on the total number of sample carriers.

For block-face imaging, the in-situ microtome (Denk and Horstmann, 2004) and focused ion beam ablation (Knott et al., 2008; Xu et al., 2017) are the two most established methods. The in-situ microtome allows for rather quick ablation of the surface of medium sized sample volumes of up to (0.5 × 0.5 × 0.5) mm<sup>3</sup> . It has been reported to run autonomously for up to several weeks (Wanner et al., 2015). Yet, the section thickness is limited by the radius of the knife edge. About 20 nm are achievable because the sections do not need to be collected, but the thinner a section thickness is chosen, the less reproducible the results will be. If isotropic voxel data with <15 nm side length is required, FIB-SEM is currently the only available technique. The slow ablation speed results in only rather small accessible volumes, currently few tens of micrometers side length at maximum. A recent development aims to increase the throughput of this approach by dissecting the sample with a hot knife without loss into smaller cubicles and parallelizing the ablation/imaging process with several FIB-SEM systems (Hayworth et al., 2015). The acquired datasets are subsequently recombined into the original sample volume. Even tracing of fine neuronal processes seems to be possible across the borders of these cubicles.

#### SAMPLE VOLUMES AND IMAGING TIME

Comparison tables providing an overview of the various electron microscopical approaches to volume imaging are available in the review articles from Briggman and Bock (2012), Titze and Genoud (2016), and Kornfeld and Denk (2018). However, the limiting factor for all approaches is the sample volume that can be assessed in a reasonable time and at the required resolution. Thus, all volume electron microscopy applications will benefit from an increase in image acquisition throughput. As neuronal circuits can span hundreds of micrometers or more, the need for imaging larger sample volumes is particularly important within the field of connectomics.

For example, some types of neurons, such as Claustrum neurons, reportedly can wrap around the whole brain (Reardon, 2017). In the case of an entire circuit, the cortical column (Mountcastle, 1957; Fox, 2018) describes the concept of a modular building block of circuitry. It spans all six cortical layers and part of the underlying white matter, therefore measuring to a depth of up to 2 mm and a lateral extent of up to 500µm. Such volumes can easily exceed the capabilities of a standard single beam scanning electron microscope—or rather the time that can be allocated for such a project. One cubic millimeter of brain tissue, cut into 30 nm thin slices, results in more than 33,000 sections of 1 mm² each. Imaging this total area with a pixel size of 4 nm, which is sufficient to grasp all required details, will take approximately 12 years with a state-of-the-art single-beam SEM (Titze and Genoud, 2016).

However, accelerating the imaging with a single-beam SEM will have an impact on the image quality (Reimer, 1998): Increasing the scan speed of the illuminating electron beam will result in reduced contrast to noise ratio due to the shorter dwell time per pixel, i.e., less electrons per pixel. This can be compensated by increasing the beam current at the cost of decreasing the resolution of the illuminating electron beam due to electron-electron interactions. The solution to this dilemma is parallelizing the imaging process. In principle, one could use several SEMs in parallel; a more economical way is to parallelize imaging within a single instrument. Multi-beam scanning electron microscopes (Ren et al., 2014; Eberle et al., 2015a) will enable data acquisition times of less than half a year in the example above and might therefore help bridging the gap between microscale- and mesoscale-connectome.

### MULTI-BEAM SCANNING ELECTRON MICROSCOPY

Using multiple electron beams in parallel has been of interest in electron beam lithography for decades: reducing the writing time of semiconductor structures with multiple electron beam lithography is of great economic interest (Pease, 1979; Chang et al., 2001; Platzgummer et al., 2013). If, next to multiple-beam illumination, multi-beam imaging is also required, a detection path needs to be added. Up to date, there are a number of different concepts for multi-beam electron microscopes, such as multi-column or multi-beam systems (Mukhtar, 2018). The multi-column approach proposes multiple miniaturized electron optical columns in parallel (e.g., Meisburger et al., 2015). The number of micro-columns that have been proposed is for example 69 in Luo and Khursheed (2014). The multi-beam approach utilizes a bundle of electron beams generated from a single electron source and a single column (Mohammadi-Gheidari and Kruit, 2011; Keller et al., 2014).

What speed advance does a multi-beam SEM provide? Theoretically, the imaging throughput of a multi-beam SEM equals that of a comparable single-beam SEM multiplied by the number of beams. For the single beam SEM, the area throughput is basically given by pixel dwell time and total number of pixels to be acquired. Overhead times, such as stage movements, are usually of minor impact, especially for singlebeam SEMs with large frame stores that allow to tesselate an area with fewer individual, but very large image tiles that take quite long to acquire. For multi-beam SEMs, the pure imaging time for a similarly large image tile consisting of many subimages is reduced by the above mentioned theoretical factor. If the overhead remains unaltered, its relative impact on total acquisition time will increase.

Under experimental conditions, an image acquisition rate of up to one terapixel per hour (Haehn et al., 2017) is achievable with a 61-beam SEM (**Figure 1**). While the illuminating beam array scans over the sample surface, secondary electrons are generated at each position of the primary electron beams. These

are collected into separate channels, and the signal intensity is detected as function of the sample position of the primary electron beams. Pixel by pixel, the image for each individual beam is generated, and these images are merged to form the hexagonal full multi-beam field of view (mFOV, figure 1, right). If the region of interest (ROI) to be imaged is larger than one mFOV, the stage is moved to an adjacent sample position and the next mFOV is acquired with a little overlapping seam. The image information present in the overlap areas is used to correctly stitch together all mFOVs of an ROI. More details of the operating principle have already been described elsewhere (Eberle et al., 2015b; Kemen et al., 2015).

It shall be noted, though, that the highest image acquisition speed is only useful if continuous operation can be guaranteed. In case of a multi-beam SEM, this requires high automation effort, as all beams of the multi-beam array need to perform equally in order to generate homogeneous image data across the full mFOV. **Figure 2** shows an example of a seamlessly imaged mouse brain section of ∼3 mm² at 4 nm pixel size.

#### LARGE DATA CHALLENGES

The overwhelming data rates high throughput EMs produce call for adequate strategies to handle this amount of data.

The simplest approach is to store the data on a local storage system as they are produced. This has the drawbacks of any local storage system, such as limited extendibility and data accessibility, limited simultaneous read/write operations, and backup effort. For small imaging volumes, nevertheless, the simplicity of this solution can still outweigh the drawbacks. Alternatively, the data can be stored in a distributed or even public storage system, with better scalability and accessibility. However, in that case, data transfer may become a significant cost factor.

Storage needs can be reduced when data compression may be applied. There is a tradeoff between data acquisition rate and data quality when imaging at highest data rates. Here, the task isb not to produce the best image with good contrast-to-noise ratio (CNR), but an image that can still be processed reliably. Depending on the application, highest data acquisition rates may be achieved at a point where images have a CNR inadequate for lossless data compression. If the application allows data compression with loss of information, larger compression factors are achievable. Next to general image compression methods, this might also be vectorization of data, e.g., by contour finding. With a typical single beam image size of 5–12 Mbyte, and a typical size of a vectorized data set of few kbytes to several 10 kbytes, the achievable data reduction rate is then about 10<sup>2</sup> -10<sup>3</sup> . In the case of contour finding, for example, this value depends on the number of features in the image and the required contour accuracy. The more a priori knowledge about the images is available, the better the image data can be condensed into a corresponding model.

Ultimately, real time data processing will allow for maximum data rate reduction. In the extreme case, each image could be reduced to e.g., one bit of information, depending on whether it matches a predefined criterion or not, and just storing this information. In general, a number of key parameters, corresponding to a few bytes, will be extracted from each image, and only these parameters need to be stored. The data reduction rate will then be on the order of 10<sup>6</sup> .

This extreme case is often not possible. Even worse, data amounts may increase during processing before they can be reduced. In a connectomics data set, for example, potentially billions of neurons need to be represented unambiguously, so 64 bit encoding is required initially. As the image data usually

an enlarged cutout. Sample with courtesy from Jeff Lichtman and Richard Schalek, Harvard University; figure adapted from Eberle et al. (2017).

is encoded in 8 bit format, the segmentation data size exceeds the actual image size by a factor of 8. Only after segmentation, data will be compressed by a factor of about 700 (Haehn et al., 2017). This example shows that connectomics research today is forced to also focus onto the development of suitable processing solutions for such huge data sets.

# OUTLOOK

The seamless integration of storage and computing solutions to the imaging system will be one of the future requirements for high throughput EM experiments in connectomics. Once this has been accomplished, further improving the imaging throughput of single-beam and multi-beam SEMs will enable investigating even larger volumes of neuronal circuitry. This becomes even more relevant if new sample preparation technologies with faster ablation and higher resolution, such as high-speed ion milling, become available (Nowakowski et al., 2017; Kornfeld and Denk, 2018).

For example, an ultrahigh-precision stage could reduce downstream computational efforts for seamless image stitching between adjacent mFOVs. Improving stage move times will reduce the imaging overhead. With improved contrasting of the sample, images could be taken at less electrons per pixel. This means, at constant or improved current per beam, faster scans would be possible. For several types of connectomics

#### REFERENCES

Arganda-Carreras, I., Turaga, S. C., Berger, D. R., Cire¸san, D., Giusti, A., Gambardella, L. M., et al. (2015). Crowdsourcing the creation of image segmentation algorithms for connectomics. Front. Neuroanat. 9:142. doi: 10.3389/fnana.2015.00142

investigations, there will be a need for improved resolution. Multi-beam SEM technology has just recently become available and has the potential to fulfill the throughput and resolution requirements of future connectomics experiments needs.

Although both improved throughput, causing higher data rates, as well as better resolution, enabling smaller and therefore more voxels per volume, will pose an even larger challenge on the already limiting computational effort, the development of computation technology is expected to match the demands of connectomics research in the years to come. Manual tracing and segmentation (White et al., 1986) has been replaced by machine learning and neural networks trained with ground truth from manual segmentation (Turaga et al., 2010; Arganda-Carreras et al., 2015; Berning et al., 2015; Januszewski et al., 2016).The main task for the near future will be to implement the existing tools into scalable architectures.

## AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

# FUNDING

Carl Zeiss Microscopy GmbH.

Bargmann, C. I., and Marder, E. (2013). From the connectome to brain function. Nat. Methods 10, 483–490. doi: 10.1038/nmeth.2451

Beckmann, M., Johansen-Berg, H., and Rushworth, M. F. (2009). Connectivity-based parcellation of human cingulate cortex and its relation to functional specialization. J. Neurosci. 29, 1175–1190. doi: 10.1523/JNEUROSCI.3328-08.2009


manufacturing. J. Vac. Sci. Technol. B Nanotechnol. Microelectron. Mater. Process. Meas. Phenom. 33:06FN01. doi: 10.1116/1.4931589


**Conflict of Interest Statement:** Both authors are employed by the company Carl Zeiss Microscopy GmbH, Oberkochen.

Copyright © 2018 Eberle and Zeidler. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# DVID: Distributed Versioned Image-Oriented Dataservice

William T. Katz\* and Stephen M. Plaza

*Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, United States*

Open-source software development has skyrocketed in part due to community tools like github.com, which allows publication of code as well as the ability to create branches and push accepted modifications back to the original repository. As the number and size of EM-based datasets increases, the connectomics community faces similar issues when we publish snapshot data corresponding to a publication. Ideally, there would be a mechanism where remote collaborators could modify branches of the data and then flexibly reintegrate results via moderated acceptance of changes. The DVID system provides a web-based connectomics API and the first steps toward such a distributed versioning approach to EM-based connectomics datasets. Through its use as the central data resource for Janelia's FlyEM team, we have integrated the concepts of distributed versioning into reconstruction workflows, allowing support for proofreader training and segmentation experiments through branched, versioned data. DVID also supports persistence to a variety of storage systems from high-speed local SSDs to cloud-based object stores, which allows its deployment on laptops as well as large servers. The tailoring of the backend storage to each type of connectomics data leads to efficient storage and fast queries. DVID is freely available as open-source software with an increasing number of supported storage options.

#### *Edited by:*

*Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan*

#### *Reviewed by:*

*Daniel Haehn, Harvard University, United States William Silversmith, Princeton University, United States Matthias Georg Haberl, University of California, San Diego, United States Jingpeng Wu, Princeton University, United States*

#### *\*Correspondence:*

*William T. Katz katzw@janelia.hhmi.org*

*Received: 25 July 2018 Accepted: 14 January 2019 Published: 05 February 2019*

#### *Citation:*

*Katz WT and Plaza SM (2019) DVID: Distributed Versioned Image-Oriented Dataservice. Front. Neural Circuits 13:5. doi: 10.3389/fncir.2019.00005* Keywords: versioning, connectomics, EM reconstruction, dataservice, big data, datastore, collaboration, distributed version control

# 1. INTRODUCTION

Generation of a connectome from high-resolution imagery is a complex process currently ratelimited by the quality of automated segmentation and time-consuming manual "proofreading," which entails examination of labeled image volumes and correction of errors (Zhao et al., 2018). Advances in the acquisition and segmentation of high-throughput volume electron microscopy (VEM) create larger data sets (Kornfeld and Denk, 2018) that stress data management tools due to the volume of data, the need to support proofreading as well as automated, high-throughput batch operations, and the sharing and integration of results from different research groups. While many data distribution systems focus on large numbers of relatively small datasets or file-based distribution (Dutka et al., 2015; Viljoen et al., 2016), VEM reconstructions are not easily distributed and usable to researchers through file distribution. For teravoxel to petavoxel datasets, centralized data services can provide low latency access to areas of interest without requiring the download of much larger volumes of data (Saalfeld et al., 2009; Burns et al., 2013; Haehn et al., 2017; Kleissas et al., 2017).

As reconstructions increase in both number and size, more data will be published after automated segmentation and a decreasing portion of the reconstructions will be manually proofread due to the flood of new data. Research groups around the world should be able to download regions of interest and edit them locally to further improve reconstructions to higher levels of accuracy. However, no connectomics data system exists that allow remote or postpublication editing on data copies with the option to easily integrate these changes with other copies, including the original, centralized data repository. Distributed version control systems for software, like git and the collaborative website github.com (Blischak et al., 2016), provide workflow examples of how scientific data could be shared, forked, and collaboratively edited, even if git is not a viable system for handling large VEM reconstructions.

Connectomics data is also quite heterogeneous. In addition to the large volumes of grayscale and segmentation images, there can be agglomeration information in the form of supervoxels and merge/split trees as well as synapse and workflow data useful to managing the reconstruction process. Low latency retrieval of neuron data will probably require denormalizations of data such that segmentation is not only held across multiple resolutions but also separated into neuron-specific sparse volumes (i.e., compressed binary representations that can span large volumes). The various forms of data can be mapped onto different storage systems based on requirements for data size, latency, and cost per terabyte. Data services should be available in isolated, small compute environments like laptops as well as institutional clusters and multi-region clouds.

Over time, connectomics researchers will create a variety of tools that need access to the data despite possible changes in how the data is stored. A high-level Science API, focused on connectomics operations, can shield clients from infrastructure changes and allow easier support of multiple tools.

DVID<sup>1</sup> has made several contributions to the state of the art. First, it provides a simple mechanism that efficiently adds branched versioning to storage systems that provide key-value store interfaces (**Figure 1**). Our branched versioning system permits instantaneous viewing of older versions, novel workflows for training proofreaders, and git-like methods of distributing data and updating remote stores. It allows one to treat committed nodes as immutable data and leaf nodes as mutable. Under this model, most of the connectome data will be immutable. The use of storage via a key-value interface allows us to exploit a spectrum of caching and storage systems including in-memory stores, embedded databases, distributed databases, and cloud data services. By partitioning data into key-value pairs, we efficiently handle versioning by only storing new key-value pairs covering modifications and not copying all data for each version.

DVID introduces the idea of typed data instances that provide a high-level Science API, translate data requirements to keyvalue representations, and allow mapping types of data to different storage and caching systems. The Science API provides a reliable connectomics interface for clients and frees them from requiring specific database technology or reimplementing domain-specific processing. The mapping system allows DVID to assign some data to very low-latency storage devices like Non-Volatile Memory Express (NVMe) SSDs while exploiting

cheap, petabyte-scale cloud stores and efficient caching systems for immutable grayscale data.

DVID provides a publish/subscribe mechanism for messaging between data types so changes in one data instance can trigger modifications in another. For example, if a segmentation changes, associated synapses will be automatically modified so that requests for all synapses in a particular label will be correct.

DVID was introduced in 2013 as an open-source project and became the principal data system for the FlyEM team at Janelia Research Campus for several of the largest, dense VEM reconstructions done to date. Over the course of its use, we added a number of features driven by reconstruction demands including multi-scale segmentation, regions of interest, automatic ranking of labels by synapse count, supervoxel and label map support that provides quick merge/split operations, and a variety of neuron representations with mechanisms for updating those denormalizations when associated volumes change. This paper discusses some of the issues and interesting benefits that we discovered in using a branched versioning system for our research.

#### 2. SYSTEM DESIGN

The DVID system is a highly customizable, open-source dataservice that directly addresses the issues encountered by image-driven connectomics research. DVID provides versioning and distribution inspired by software version control systems, customizable domain-specific data types (e.g., grayscale and label volumes, synapse annotations) accessible via a HTTP API, and

<sup>1</sup>http://github.com/janelia-flyem/dvid

#### TABLE 1 | Sample of science HTTP API.


*Each datatype implements its own HTTP endpoints although similar datatypes (e.g., ones dealing with image volumes) can reuse interfaces like the first "raw" endpoint.*

flexibility in choosing underlying storage engines, allowing its use on laptops, institutional clusters, and the cloud.

DVID persists data through an abstract key-value interface that is satisfied by a number of swappable storage engines. We started with a key-value interface because (1) there are a large number of high-performance, open-source caching and storage systems that match or can be reduced to a key-value API, (2) the surface area of the API is very small, even after adding important cases like bulk loads or sequential key read/write, and (3) versioning can be easily added by modifying keys to incorporate a version identifier.

From a user's perspective, the DVID system can be described through its two major interfaces. The first is a client-facing Science API that provides a rich set of connectomics operations through a REST (Level 2 of Richardson Maturity Model<sup>2</sup> ) HTTP API (**Table 1**). The second is a Storage API that provides a limited set of key-value operations (**Figure 2**).

<sup>2</sup>https://martinfowler.com/articles/richardsonMaturityModel.html

From a developer's perspective, pluggable data type code packages (e.g., a **uint8blk** type that supports uint8 image volumes like VEM grayscale data) expose a data type-specific HTTP/RPC API on the user-facing side and processes data in the form of key-value pairs that get persisted through the storage interface. The storage interface is handled by pluggable storage engine code packages that can obtain version and data instance IDs from the key and store the key-value pair in a reasonable way for the particular storage system. Data types also can be constructed that simply proxy requests to a backend service like bossDB (Kleissas et al., 2017) at the cost of versioning.

#### 2.1. Example Usage

Before detailing how DVID implements the Science API and versioned data, we will describe how DVID is used in an example reconstruction and connectome analysis workflow.

DVID administration can be performed through a DVID command in a terminal, sending a HTTP request through

tools like curl or httpie<sup>3</sup> , or using the DVID Console web application<sup>4</sup> . Using one of those three methods, we first create a new repository, and then add a **uint8blk** data instance called grayscale and a **labelmap** data instance called segmentation.

Due to the large scale of our image volumes, FlyEM employs python scripts that load strips of grayscale data using HTTP POST requests to the grayscale instance. These HTTP requests are typically handled by libdvid, a C++ library with python bindings<sup>5</sup> . Similarly, scripts load the automatically segmented label data (64-bit unsigned integer per voxel) (Januszewski et al., 2018) into the segmentation instance using strips of highlycompressed DVID blocks (see section 2.5). Both grayscale and segmentation voxel data can be loaded into DVID, letting DVID do the image pyramid generation as well as the label indexing (i.e., determining the blocks spanned by each label). However, for very large volumes, it is more efficient to offload the image downsampling and label indexing to a cluster and then directly ingest the results. We have published Spark (Zaharia et al., 2010) tools that can be used for large-scale processing<sup>6</sup> .

After ingestion of the image volume and segmentation, we commit (lock) the root version and create a new version for our manual proofreading. Additional data instances are typically created, such as a synapses instance of **annotation** to hold synapse point annotations and various **keyvalue** instances to hold proofreader assignments, generated skeletons, and other data useful to the various clients and scripts used for reconstruction and connectome analysis. In each case, python, C++, or Javascript clients connect with DVID through the languages' built-in HTTP library or intermediate libraries like libdvid.

Proofreaders use tools like NeuTu (Zhao et al., 2018), Neuroglancer<sup>7</sup> , and various scripts to edit the segmentation, view 2D image sections, 3D sparse volumes, meshes, and skeletons, and manage data necessary for our proofreader workflows. HTTP traffic to DVID can easily exceed 100,000 requests per minute and include server metadata queries that return within microseconds as well as sparse volume requests for massive bodies that take tens of seconds.

At some point in time, we may decide to create a snapshot of all the data so we commit the current version and create a new one. The state of the data at the time of that commit will always be available for instantaneous viewing.

#### 2.2. Versioned Data

Versioning can be modeled in at least two ways: branched versioning using a directed acyclic graph (DAG) as in the git software version control system, or a linear timeline that can be thought of as one path through the DAG. Current connectomics data services use no versioning or linear versioning where there is one head, the current state. The underlying storage can be optimized for access of the current state while any changes are recorded into a log, which will be likely accessed less frequently than the head (Al-Awami et al., 2015).

A DAG-based approach is more powerful, allowing branching and merging, and has been shown to be very effective for collaborative efforts like distributed software development. While branched versioning is already showing utility for proofreader training as described below, we believe its utility will be more obvious as published reconstructions increase in both number and size and the portion of manually proofread data decreases. As discussed in Future Work, specialists in various neural circuits will be able to improve reconstruction accuracy of published regions, maintain their own private branch until publishing results, and then optionally merge edits back into the central repository.

The DAG, in one way or another, will be dictated by postpublication manual proofreading as well as any collaborative editing involving decentralized data storage. Each edited clone is essentially a branch, even if described as a control layer

<sup>3</sup>https://httpie.org/

<sup>4</sup>https://github.com/janelia-flyem/dvid-console

<sup>5</sup>https://github.com/janelia-flyem/libdvid-cpp

<sup>6</sup>https://github.com/janelia-flyem/DVIDSparkServices

<sup>7</sup>https://github.com/google/neuroglancer

over linear versioning, and attempts to merge results require a DAG for provenance tracking. Aside from edits due to continued reconstruction improvements, full-fledged branching and merging is also important for collaborative data analysis (Huang et al., 2017). The drawback of a DAG is its complexity, but even if a versioning system uses a DAG internally, its clients could show a single selected branch unless handling operations that need to expose that complexity.

Distributed software version control systems like git use the nodes in a DAG to represent each committed version of data. A commit is a snapshot of data at that time, and as such, would include an accumulation of changes since the last commit. Provenance is kept at the commit level, not the change level, so if a line in a file were changed three times since the last commit, only its final state would be recorded and not the individual changes between the commits. While it would be nice to have a complete record of all changes to data, there could be a significant storage overhead for storing every change regardless of its importance. For this reason, fine-grain provenance, if desired, is delegated to the data type implementation while commit-level versioning is standard for all versioned data instances. Many data types publish each mutation to an Apache Kafka system, a distributed logging system that can be used to inform other services of changes to data (Wang et al., 2015). In addition to Kafka logging, the data type **labelmap** always provides finegrained provenance by logging all mutation operations like merges and splits to an append-only file. This log is used on server restart to populate an in-memory label map, which provides supervoxel id to agglomerated label mappings.

At its core, DVID adopts the DAG view of versioning used by software version control systems like git. Unlike git, DVID partitions data not in files but in data instances of a data type, for example, there could be segmentation-param1 and segmentation-param2 instances of data type **labelmap**, which supports label volumes and label-specific sparse volume retrieval and operations. DVID also allows access to any version of the data at any time.

A dataset in DVID is described as a "repository" and corresponds to a single DAG. Each node is a version identified by a RFC4122 version 4 UUID<sup>8</sup> , a 32 character hexadecimal string that can be generated locally and is unique globally. Datasets are typically identified by the UUID of the root. At each node of the DAG, users can store a description similar to git's commit message as well as append to a node-specific log.

DVID requires each branch of the DAG to have a unique string name. By default, the root node is part of the "master" branch that uses the empty string for a name. For each committed node, we can create one child that extends the parent branch or any number of children with new branch names.

The ability to easily branch and handle distributed editing is a significant advantage of a DAG approach. Proofreaders can branch their own versions to allow training and testing (**Figure 3**). As described below, branching requires little additional storage cost since only modifications need to be stored. Also, no modification to tools are required since DVID clients

can simply specify the UUID of a training version and leave the current "master" data unaffected.

Over the past 3 years, the FlyEM team has used DVID during reconstructions of seven columns of medulla (Takemura et al., 2015) and the mushroom body (Takemura et al., 2017) of Drosophila. The reconstruction process produced large DAGs with regions of heavy branching due to proofreader training or experimental edits (**Figure 4**). DVID provides an extensive HTTP API for clients to download server state and dataset metadata, including the DAG. The DVID Console web application provides a simple view of the master branch, and an alternative version allows viewing of the full DAG as well as the ability to click any node to view the log and data instances associated with that node.

The DAG is useful for quality control and being able to look back to previous states of our dataset as well as the comments attached to it. If mistakes were made, we can determine where they were introduced. While viewing historical data only requires versioning, not necessarily branched versioning, its ease of use requires a data service that can display all versions without a significant time delay.

#### 2.3. Branched Versioning of Key-Value Data

DVID implements branched versioning over different types of key-value data by (1) keeping metadata, including the version

<sup>8</sup>https://tools.ietf.org/html/rfc4122

DAG, in memory and (2) extending keys to include data and version identifiers. When data is modified in an uncommitted, open node, the data type implementation retrieves and stores key-value pairs as needed while the core DVID system modifies the keys to include identifiers for the particular data instance and the version. We want to emphasize that modifications in a version usually only affect a subset of key-value pairs. For example, the **labelmap** data type partitions 3D space into small blocks (typically 64 × 64 × 64 voxels) such that each block is a single key-value pair. Modifications of the label volume in a new version requires only storing the affected blocks, and keyvalue pairs corresponding to untouched areas will be inherited by ancestors as described below.

The core DVID storage package uses a key composed of a data instance identifier, the datatype-specific key, a version identifier, and a tombstone marker, in that order by default as shown in **Figure 5**. A data instance can insert multiple classes of key-value pairs, each with different formats of datatype-specific keys and corresponding values. For example, the **labelmap** and **labelarray** data types (described in more detail in section 2.5) use two classes of key-value pairs: (1) blocks or cuboids of label data where the datatype-specific key has a scale integer prepended to the ZYX block coordinate, and (2) label indices where the datatypespecific key is simply a 64-bit unsigned integer label and the value describes the blocks containing the label in question.

DVID maintains a mapping of globally unique 128-bit data instance and version UUIDs to unsigned 32-bit integers solely to decrease key sizes. The 32-bit identifiers are server-specific since these identifiers could collide with identifiers in remote DVID servers as new data instances and versions are added locally and remotely. When key-value pairs are exchanged with remote DVID servers, the source server identifiers are converted to remote server identifiers by comparing the globally unique data instance and version UUIDs.

In **Figure 5**, two data instances are shown: a **labelmap** instance (data id 1) and an **annotation** instance (data id 2). The **labelmap** instance has key-value pairs for label 198's index and two label blocks in version 0, and the **annotation** instance holds a single block of annotations. A tombstone flag can mark a keyvalue as deleted in a version without actually deleting earlier versions, as shown for the last key, which marks the deletion of annotations in block coordinate (23, 23, 10) in version 1. The annotations for that block still exist in version 0 since a non-tombstoned key exists.

#### 2.3.1. Overhead of Versioning

**Figure 6** shows how key-value pairs from different data instances can be distributed across a DAG. In this example, segmentation data for a 6,400<sup>3</sup> voxel volume with 1,000 labels is stored in a **labelmap** instance (shown in blue) at the root version 8fc4. The segmentation requires one million key-value pairs for label block data and another 1,000 key-value pairs for the label indices. Additionally, synapse 3D point data is stored in an **annotation** instance (shown in red). The annotation data requires key-values for only the blocks containing synapses.

The majority of key-value pairs are ingested at the root of the DAG and only modified key-value pairs need to be stored for later versions. In **Figure 6**, three additional versions have been created. In version e14d the synapse annotations for block (1, 2, 3) was deleted by storing a tombstone key. Clients that access version e14d can access all the data stored in the root version with the exception of synapses in that one block. In


FIGURE 5 | Each data type persists data using datatype-specific key-value pairs. Key-value pairs for two data instances are shown: a labelmap instance (data id 1) in blue and an annotation instance (data id 2) in red. The datatype-specific component of a key (TKey) could be a block coordinate for a block of voxels. DVID then wraps this TKey, prepending a short data instance identifier and appending a version identifier. A tombstone flag (T) can mark a key-value as deleted in a version without actually deleting earlier versions, as shown for the last key, which marks the deletion of annotations in block coordinate (23, 23, 10) in version 1.

version ec80 we splitted a small fragment from label 23, which required modification of the label block (37, 51, 53) containing that fragment as well as the key-value pairs for the label 23 index and a new label 1001 index for the split voxels. In version d353 we splitted from label 23 another small fragment that spanned two blocks, and we added a new synaptic annotation to that region. These operations required the addition of a few more key-value pairs that take precedence over earlier key-value pairs.

Teravoxel datasets can require more than a hundred million key-value pairs, depending on the chosen size of a labelmap instance's block, and mutations to individual neurons will alter a very small percentage of the key-value pairs. So rather than duplicating unmodified data for each commit or snapshot, the decomposition of data into more granular key-value pairs allows efficient versioning.

One issue is determing the relevant key-value pairs for any version of a data instance. As we saw in **Figure 5**, the version identifier is typically appended to the type-specific key, which leads to different versions of a type-specific key to be grouped together in systems that order keys. Key-ordering occurs in many popular key-value databases, particularly those that employ log-structured merge trees (O'Neil et al., 1996) like leveldb<sup>9</sup> , RocksDB10, and newer systems based on WiscKey (Lu et al., 2016). These databases provide range queries (see Ordered Key-Value API in **Figure 1**). Because the versions of a key-value pair are grouped together, we can use range queries to sequentially load the keys or key-value pairs into memory, and then use our in-memory DAG to select the most recently stored ancestor of our desired version. Sequential access provides significant speed advantages across rotational disks, solid-state drives, and even in-memory storage (Jacobs, 2009).

Returning to the example of 3d label data types like **labelmap**, if a block was modified in N versions of that data instance, it will require N key-value pairs. Support of versioning for that particular block will incur the overhead of now reading N keyvalue pairs instead of just one, as well as the time to calculate the closest stored version using our in-memory DAG. However, the cost of handling unnecessary versions is countered by the significant speeds of both disk-based and in-memory sequential access. The number of modified versions for a block should be small because most regions are not constantly modified due to manual proofreading. As seen in section 2.5, the **labelmap** data type uses relatively static voxel labels (i.e., supervoxel identifiers) and maintains supervoxel-to-neuron mappings as neurons are split and fragments are merged.

#### 2.3.2. Support for Non-ordered Key-Value Stores

We have also built support for non-ordered key-value stores where range queries are either costly or not available. Google Cloud Storage<sup>11</sup> can be viewed as a distributed, petabyte-scale key-value store that supports conditional writes as well as key prefix searches, which could be used to implement range queries. Unfortunately, these key prefix searches introduce significant latency for each data request. We observed that key spaces can be divided into two categories: a computable key space where valid keys can be computed (e.g., the **uint8blk** data type stores blocks of grayscale with block coordinate keys, easily calculated for requested 3D subvolumes) or a non-computable key space where arbitrary keys are used (e.g., the **keyvalue** data type that allows user-specified keys).

In practice, we only use Google Cloud Storage for data types with computable key spaces. Even with this restriction, versioning requires range queries or speculative queries on all possible key versions when retrieving a particular key-value pair (kv).

To solve this problem, we introduced a novel strategy to eliminate costly key searches or the need to separately maintain an index of stored keys. For each type-specific key, we maintain a single, versionless kv that stores the keys for all versions and the highest priority kv, which comes from the most recent kv in the master branch or, if no version of this key exists in master, the most recent kv of any branch. Writes of a versioned kv start with a conditional write to the versionless kv. If the conditional write succeeds, it is the first write of any version to this key and we are done. If the conditional write fails, we read the versionless kv and compare the new kv to its stored kv. If the new key has higher priority, the new value evicts the stored kv to its own versioned kv. If the new key has lower priority, we write the new versioned kv and append its version to the list of all versions stored in the versionless kv. With this approach, we achieve the following properties:


As shown by the Google Cloud Storage example above, a DVID storage engine can tailor the implementation of versioned storage to the capabilities of a storage system. DVID storage engines can also override the default key and form a version-first representation (Bhardwaj et al., 2014) if it is more advantageous to group all kv by version instead of by type-specific key. This approach can be particularly useful for optimizing transmission of kv corresponding to a subset of versions, as would happen when synchronizing with a remote DVID server. A proposed DVID store, described in Future Work, takes this approach since we can create a compact, in-memory index of all stored keys in committed, immutable versions.

#### 2.4. Data Types

For each type of data, researchers can tailor a HTTP API and trade-off access speed, storage space, versioning support, and ease of programming.

DVID provides a well-defined interface to data type code that can be easily added by users. A DVID server provides HTTP and RPC APIs, versioning, provenance, and storage engines. It delegates data type-specific commands and processing to data type code. As long as a DVID type can return data for its implemented commands, we don't care how it is implemented.

By modifying or adding DVID data type implementations and writing layers over existing storage systems, DVID allows customizable actions on data via a HTTP API. We can tune key-value representations for acceptable performance among storage space, access speed, and ease of programming trade-offs.

<sup>9</sup>http://leveldb.org

<sup>10</sup>https://rocksdb.org

<sup>11</sup>https://cloud.google.com/storage/

Different types of checksum and compression can be used for each data type at the key-value level. And we can choose among the different key-value stores and assign the best match for each data instance. For example, for highly compressed label data, we can choose fast but relatively small NVMe SSDs to maximize access speeds.

DVID supports a variety of data types including the following: **uint8blk**: 3d grayscale volumes.

**labelmap**: 64-bit label 3d volumes, including multi-scale support and sparse volume operations.

**imagetile**: multiscale 2d images in XY, XZ, and YZ orientation, similar to quadtrees.

**annotation**: 3d points that can be accessed by associated label, tags, or spatial coordinate.

**roi**: regions of interest represented via a coarse subdivision of space using block indices.

**keyvalue**: a simple key-value pair store that can be used as a versioned file system.

Each of the data types above use key-value pairs in different ways. While **uint8blk** and **labelmap** both partition 3D space into smaller blocks, the **labelmap** data type persists highly compressed 64-bit supervoxel identifiers in the blocks and also maintains other key-value pairs for label (i.e., neuron identifier) indexing that describe the blocks and supervoxels within any given label. The **annotation** data type can employ up to three different classes of key-value pairs holding JSON-encoded points (synapses, bookmarks, etc.) sorted by ZYX block coordinate, underlying label, and arbitrary string tag. The **keyvalue** data type is a simple pass-through to the underlying key-value store. It is typically used for new types of data until researchers understand the kinds of requests that will be required and whether a new data type should be built to optimize the handling of those requests.

DVID provides a publish/subscribe mechanism for syncing changes in one data instance with associated data instances. For example, we can declare a segmentation instance of data type **labelmap** should be synced with a synapses instance of data type **annotation**. If a label in segmentation is split or merged with other labels, the mutation will be passed to synapses, which then updates its internal indexing used for quickly returning all synapses in a given label.

Users can access a detailed description of each data type's Science API by pointing a web browser to a running DVID's HTTP interface. For example, the interface to the **uint8blk** data type can be examined by browsing http://emdata.janelia. org/api/help/uint8blk for a DVID server running on port 80 of **emdata.janelia.org**. Any supported data type can be reviewed by replacing the last word in the help URL with the data type name. Since a detailed exploration of each data type is beyond the scope of this paper, we provide a sampling of the Science API in **Table 1** and refer readers to the embedded data type documentation in the DVID github repository.

#### 2.5. Versioning 3D Label Data

Each DVID data type provides its own portion of Science API and method of translating the necessary data into key-value pairs. In this section, we describe how data types can evolve by describing the history of four 3D label data types: **labelblk**/**labelvol**, **labelarray**, and finally **labelmap**. The implementation of each data type impacts the speed of neuron editing, the storage efficiency of versioning, and the functions available through its Science API.

The first 3D label data types were **labelblk** and **labelvol**, which handle 64-bit label arrays and each label's sparse volume representation, respectively. The **labelblk** data type allows many ways to read and write the 64-bit unsigned integer label at each voxel. These include reading 2D slices in XY, XZ, and YZ orientation in a variety of formats (e.g., PNG or JPG), reading 3D subvolumes as label arrays in any supported compression scheme (uncompressed, lz4, gzip and/or Neuroglancer's compression format), querying single or multiple voxel coordinates using JSON, and even returning 2D PNG color images where each label has been hashed to a color. For maximum throughput, we also allow reading by blocks so that little processing is necessary and data is streamed from the underlying key-value store to the HTTP connection. The **labelvol** data type allows reading and editing sparse volumes for labels. Its Science API allows reading arbitrarily clipped sparse volumes using run-length encoding (RLE) with optional lz4 or gzip compression. Sparse volumes can also be split or merged.

These first data types only support two scales: the original ingested voxels or "coarse" volumes where each block of voxels was downsampled to a single voxel. Internally, the **labelblk** data type persists key-value pairs where each type-specific key is a ZYX block coordinate that corresponds to the label array for that block. The **labelvol** uses a type-specific key composed of the 64 bit unsigned integer label prefixed to the ZYX block coordinate, and the associated value is the sparse volume RLEs within that block. By doing a range query on a label, the data type code can easily retrieve all RLEs for a given label as well as clip sparse volumes by Z coordinate.

As described in the section above, instances of these two data types can be synchronized using DVID's internal publish/subscribe mechanism. Let us assume that a DVID server is operating on port 8000 of the server mydvid.net with a single version at UUID ee78982c87b14d008bb3f93e9e546c10. A two-way sync can be established between a segmentation instance of **labelblk** and a sparsevol instance of **labelvol** by sending a JSON string {"sync":"segmentation"} to http://mydvid.net/api/node/ee789/sparsevol/sync and a reciprocal string to http://mydvid.net/api/node/ee789/segmentation/sync. Note that HTTP requests only need a recognizable substring of a UUID rather than the full 32 character hexadecimal string.

If a user merges two labels via the **labelvol** merge request, a synced **labelblk** instance will automatically modify all voxels affected by the merge. Similarly, if the labels of voxels are modified through **labelblk** instance requests, the change will be sent to the synced **labelvol** instance and the sparse volumes of any affected label will be modified.

We could also sync a synapses instance of **annotation** data type with the underlying label volume by sending a JSON string {"sync": "segmentation"} to http://mydvid.net/api/node/ee789/synapses/sync. This oneway sync means changes in the label volume will automatically modify the list of synapses corresponding to the affected labels.

While the first iteration of 3D label data types was successful and allowed very fast retrieval of sparse volumes due to its separate storage, we found that maintaining the sparse volumes using our admitedly simple format could dominate the underlying key-value store. So we created the **labelarray** data type that consolidated both **labelblk** and **labelvol** Science APIs under one implementation without the need for syncs.

The **labelarray** data type supports multi-scale representation and primarily uses two classes of key-value pairs as described earlier in section 2.3: label data organized into blocks and label indices describing the blocks intersected by each label. The storage requirement is significantly smaller than synced **labelblk**/**labelvol** instances because potentially large sparse volumes are replaced by an index of blocks. The new data type also exhibits faster write and much slower sparse volume read times since precomputed RLEs are not stored but must be computed on-the-fly.

The most recent **labelmap** data type adds in-memory label maps to the **labelarray** architecture and uses supervoxel identifiers as the block label data. Many segmentation techniques generate an initial base segmentation that tends to be conservative followed by more aggressive agglomeration passes (Nunez-Iglesias et al., 2014; Parag et al., 2015; Januszewski et al., 2018). The **labelmap** data type supports this approach. By using an in-memory label map, label merges are extremely fast and do not alter the underlying supervoxel blocks. We also allow "cleave" operations that split label bodies along supervoxel boundaries, thereby preserving underlying supervoxel blocks as well and only modifying the label map. Supervoxel splits require modification of the block key-value pairs but are relatively rare compared to merges and cleaves, particulary as both the underlying grayscale imaging and automatic segmentation processes improve.

For each node in the DAG, the **labelmap** data type stores label edits (merges, cleaves, and supervoxel splits) in an append-only log. Requests can cause lazy loading of all edits from the root to the given version and population of the in-memory label map.

The newer **labelmap** and **labelarray** data types store label data in a highly compressed format inspired by the Neuroglancer compression scheme12, which partitions each block of data into smaller sub-blocks. The DVID label compression format makes the following changes: (1) adds a block-level label list with subblock indices into the list (minimal required bits vs 64 bits per index in the original Neuroglancer scheme), and (2) the number of bits for encoding values is not required to be a power of two. A block-level label list allows easy sharing of labels between sub-blocks, and sub-block storage can be more efficient due to the fewer bits per index (at the cost of an indirection) and better encoded value packing (at the cost of byte alignment). We gain space, up to an additional 2x compression for a given block, and simpler label updating at the cost of increased computation and Neuroglancer's explicit GPU support. Although label blocks are stored in this highly compressed format, data can be transcoded to Neuroglancer's compressed segmentation format during requests.

FIGURE 7 | Scalability of uncompressed grayscale image reads from Google Cloud Store backend. As the number of DVID servers increase, simultaneously requesting non-overlapping image subvolumes from a 16 TeraVoxel dataset, the throughput plateaus just below 1.2 Gigavoxels or 9.6 Gigabits per second. Servers were at the Janelia cluster with 16 real request threads per server, connecting to a Northern Virginia Google Cloud Store through a 10 Gigabits per second connection. The grayscale instance had only one version corresponding to the ingested image (8-bit/voxel) volume.

Despite how differently the four data types implement 3D label support, the HTTP APIs are mostly identical save for optional features that were added in later data types.

#### 2.6. Storage Backends

The use of key-value storage (KVS) systems as the underlying store brings a number of benefits. Open source KVS systems span from simple, embedded leveldb implementations to strongly-consistent, globe-scale distributed systems. Once data is immutable, there are number of distributed KVS systems for efficiently caching the data (e.g., groupcache). This allows us to build branched versioning systems that use different kinds of KVS systems for different classes of data. For example, relatively immutable, very large data like original grayscale imaging can be assigned to extremely scalable cloud-based systems and cached locally using off-the-shelf software due to its immutable nature, while much more compressible and mutable data like segmentation can be stored on fast NVMe SSD drives.

**Figure 7** shows the scalability of the Google Cloud Store as a backend for immutable, uncompressed grayscale (unsigned 8-bit intensity per voxel) volumes. Because the data is immutable, any number of DVID servers can be spun up and directed toward the cloud. The maximum throughput using test servers at Janelia requesting data from Google Cloud Storage is slightly less than 1.2 Gigavoxels (9.6 Gigabits) per second, which corresponds to the 10 Gigabit per second connection from Janelia to the internet. If we were looking through a sequence of grayscale images, this would amount to approximately 4,400 (512 × 512 pixel) images per second, or 73 proofreaders scrolling through those images at 60 fps.

Currently, the bulk of the FlyEM Team's reconstruction and segmentation data is held in leveldb databases on NVMe solid

<sup>12</sup>https://goo.gl/LNMLJo

state drives and cheaper RAID-10 systems with hard disk drives. Newer grayscale volumes are kept in Google Cloud Store and we are experimenting with a simple key-value interface to the file system. Contributors have recently added a storage engine for OpenStack Swift.

#### 2.7. Availability

DVID is freely available on github (http://github.com/janeliaflyem/dvid) under the Janelia Open-Source Software license. The wiki section of that github repository provides user guides as well as installation instructions for pre-built binaries, conda builds, and docker containers.

#### 3. RELATED WORK

Typically, researchers have dealt with image-oriented data by either storing it in files or writing software systems that use a relational database to store image chunks or file pointers. Connectomics data servers include bossDB (Kleissas et al., 2017), OpenConnectome (Burns et al., 2013), CATMAID (Saalfeld et al., 2009), and more visualization-focused systems like BUTTERFLY (Haehn et al., 2017). DVID is distinguished from these other systems by its support of branched versioning, an extensible Science API through data type packages, and extremely flexible storage support through a variety of key-value store drivers.

The first to support branched versioning at large scale was SciDB (Stonebraker et al., 2011). An approach to branched versioning in relational databases culminated in OrpheusDB (Huang et al., 2017). Both SciDB and OrpheusDB could be used as storage backends for DVID data types that match their strengths. For example, SciDB is particularly adept at handling multi-dimensional arrays and could be used for the voxel data component of DVID label data types, while OrpheusDB could be used for heavily indexed synapse point annotations.

The DataHub effort (Bhardwaj et al., 2014) has very similar aims to bring a distributed versioning approach to scientific datasets, offering an analog to github.com with a centralized server that builds on a Dataset Version Control System (DVCS). DataHub and DVID developed in parallel and focused on different types of data. DVCS was designed to handle datasets in the sub-Terabyte range without an emphasis on 3d image data, and it's API is a versioning query language based on SQL so the significant connectomics-focused data layer would still be needed. Much as OrpheusDB is a possible storage engine for smaller data types like annotations, DVCS could be considered a possible storage interface to DVID.

Ideally, connectomics tools would be able to use a variety of data services. This would require the community to develop common interfaces to standard operations. Currently, simple operations like retrieving 2D or 3D imagery are sufficiently similar across services so that tools like CATMAID, Neuroglancer, and BigDataViewer (Pietzsch et al., 2015) can use different image volume services including DVID.

#### 4. FUTURE WORK

Distribution of versioned data can help to efficiently synchronize remote servers, a significant problem given the scale of VEM data (Lichtman et al., 2014). For example, when establishing remote copies of massive image volumes, we envision shipping one or more disks and then synchronizing servers by sending only data associated with new nodes in the version DAG. The speed of such operations depend on the ability to easily extract and transmit data from a subset of versions as well as fast mechanisms for moving data between servers. Our current version-last approach to key encoding makes versionbased transmission costly, since it requires scanning all keys.

As the FlyEM Team increases our sharing of reconstructions to researchers around the world, we expect to spend some energy to improve data transfer rates and how version data is organized.

One such effort is a petabyte-scale DVID-tuned datastore now in the planning stages. Mutations are relatively expensive since they generally require transactions and impose difficult coordination issues when scaling operations to multiple servers. Immutable data storage is simpler and can be accelerated through a variety of techniques. For VEM reconstructions at Janelia, the majority of data exists near the top of the DAG since most of our workflows involve ingesting very large image volumes and pre-generating segmentation for every voxel (**Figure 8**). This suggests a multi-stage store where on initial ingestion and subsequent version commits, the committed, immutable data is transformed to optimize reads, storage size, and ease of versionbased distribution.

The bulk of our data can persist in immutable stores that combine compact, in-memory key indexing with version-first, append-only file storage, suitable for easy access and transmission of version deltas. This allows us to use smaller, faster storage solutions for the mutable portion of the DAG, namely the leaf nodes where manual editing tends to dominate. Retrieval of data from any version then requires concurrent retrieval from both the immutable and mutable stores.

We want to enable researchers to work on their own branches, optionally limit download to regions of interest, and share changes via pull requests (**Figure 9**). This is particularly appealing when considering the publication of massive datasets where specialists may improve regions and submit changes.

Currently, DVID provides branched versioning that meets the needs of most of our current reconstruction workflow. Only some work has been done on the remote distribution and syncing aspects corresponding to the push and pull operations of distributed versioning systems like git. DVID can push data to remote repositories and merge nodes using simple conflict resolution like node A always wins against node B if there is a conflict. In order to allow more sophisticated merges, we need to add data type-specific merge tools to the DVID ecosystem. For example, when merging two nodes of segmentation, we would want a merge tool to provide visualizations of conflicts and allow a user to choose a proper merge result. DVID should be agnostic to the form of the merge tool yet provide a conflict resolution API that could be used to select conflicts and post results.

Availability of a merge tool also allows the possibility of scaling proofreading by using entirely separate DVID servers instead of scaling up a single DVID server.

Versioning should allow downloading portions of massive datasets since it can reference the originating UUID. While full datasets may require large servers with many terabytes of highspeed storage, we plan to facilitate proofreading of regions of interest on laptops even in an offline setting. This would be similar to standard git workflows where programmers modify code locally and then submit pull requests of their changes to the central server.

In the near future, we plan on adding Badger<sup>13</sup> and RocksDB as drop-in replacements for the current leveldb storage backend.

Although DVID has initially focused on key-value stores, we are evaluating OrpheusDB (Huang et al., 2017) and may eventually support fundamentally different types of stores (polyglot persistence) like graph, relational, and scientific array databases. We are currently investigating OrpheusDB as a backend for the DVID synapse annotation data type, which indexes synapse point annotations across space, assigned labels, and arbitrary tags. Unfortunately, polyglot persistence comes at the cost of increased code to extend operations like remote distribution beyond simple key-value pairs to these new types of stores.

# 5. CONCLUSIONS

The DVID system is a powerful tool that allows us to immediately view our dataset at any commit time, and also enables training of proofreaders so that they can handle large connectomes. It has allowed us to flexibly store very large immutable datasets in the cloud in conjunction with fast, smaller storage for mutable data. This has allowed us to scale our operation and provide regional data services to collaborators. More importantly, we feel that distributed versioning in connectomics could be an extremely powerful tool for collaborating with researchers

<sup>13</sup>https://github.com/dgraph-io/badger

around the world. As the amount of published data increases dramatically due to advances in imaging, segmentation, and reconstruction workflows, there will be an increasing need to provide provenance and mechanisms for collaborative data editing and analysis. Just as distributed versioning with its notion of pull requests has greatly impacted the open source software movement, we believe that it can alter the way we think of sharing and editing connectomics data.

#### AUTHOR CONTRIBUTIONS

WK designed and implemented the core DVID system. SP helped design DVID and implemented a labelgraph data type and the Google Cloud Store engine.

#### REFERENCES


#### FUNDING

This research was funded by the Howard Hughes Medical Institute.

#### ACKNOWLEDGMENTS

DVID has received contributions from Stuart Berg, Oliver Kuederle, and Ignacio Tartavull. The DVID ecosystem includes web consoles that received contributions from Jody Clements, Alex Weston, Jenny Xing, and Rob Svirskas.

We would like to thank the extended FlyEM Team at Janelia for working with DVID and our collaborators at Janelia, Dalhousie, and Harvard.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Katz and Plaza. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Large-Area Fluorescence and Electron Microscopic Correlative Imaging With Multibeam Scanning Electron Microscopy

Shinsuke Shibata1,2 \* † , Taro Iseda1,2† , Takayuki Mitsuhashi<sup>3</sup> , Atsushi Oka<sup>2</sup> , Tomoko Shindo1,2, Nobuko Moritoki<sup>1</sup> , Toshihiro Nagai<sup>1</sup> , Shinya Otsubo<sup>2</sup> , Takashi Inoue<sup>4</sup> , Erika Sasaki<sup>4</sup> , Chihiro Akazawa<sup>5</sup> , Takao Takahashi<sup>3</sup> , Richard Schalek<sup>6</sup> , Jeff W. Lichtman<sup>6</sup> and Hideyuki Okano1,2,7 \*

#### Edited by:

Yoshiyuki Kubota, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Yunfeng Hua, Shanghai Jiao Tong University, China Naomi Kamasawa, Max Planck Florida Institute for Neuroscience (MPFI), United States

#### \*Correspondence:

Shinsuke Shibata shibata@keio.jp; shibata@2001.jukuin.keio.ac.jp Hideyuki Okano hidokano@a2.keio.jp

†These authors have contributed equally to this work

Received: 16 November 2018 Accepted: 08 April 2019 Published: 08 May 2019

#### Citation:

Shibata S, Iseda T, Mitsuhashi T, Oka A, Shindo T, Moritoki N, Nagai T, Otsubo S, Inoue T, Sasaki E, Akazawa C, Takahashi T, Schalek R, Lichtman JW and Okano H (2019) Large-Area Fluorescence and Electron Microscopic Correlative Imaging With Multibeam Scanning Electron Microscopy. Front. Neural Circuits 13:29. doi: 10.3389/fncir.2019.00029 <sup>1</sup> Electron Microscope Laboratory, Keio University School of Medicine, Tokyo, Japan, <sup>2</sup> Department of Physiology, Keio University School of Medicine, Tokyo, Japan, <sup>3</sup> Department of Pediatrics, Keio University School of Medicine, Tokyo, Japan, <sup>4</sup> Central Institute for Experimental Animals, Kawasaki, Japan, <sup>5</sup> Department of Biochemistry and Biophysics, Graduate School of Health Care Sciences, Tokyo Medical and Dental University, Tokyo, Japan, <sup>6</sup> Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, United States, <sup>7</sup> Laboratory for Marmoset Neural Architecture, RIKEN Center for Brain Science, Wako, Japan ¯

Recent improvements in correlative light and electron microscopy (CLEM) technology have led to dramatic improvements in the ability to observe tissues and cells. Fluorescence labeling has been used to visualize the localization of molecules of interest through immunostaining or genetic modification strategies for the identification of the molecular signatures of biological specimens. Newer technologies such as tissue clearing have expanded the field of observation available for fluorescence labeling; however, the area of correlative observation available for electron microscopy (EM) remains restricted. In this study, we developed a large-area CLEM imaging procedure to show specific molecular localization in large-scale EM sections of mouse and marmoset brain. Target molecules were labeled with antibodies and sequentially visualized in cryostat sections using fluorescence and gold particles. Fluorescence images were obtained by light microscopy immediately after antibody staining. Immunostained sections were postfixed for EM, and silver-enhanced sections were dehydrated in a graded ethanol series and embedded in resin. Ultrathin sections for EM were prepared from fully polymerized resin blocks, collected on silicon wafers, and observed by multibeam scanning electron microscopy (SEM). Multibeam SEM has made rapid, largearea observation at high resolution possible, paving the way for the analysis of detailed structures using the CLEM approach. Here, we describe detailed methods for large-area CLEM in various tissues of both rodents and primates.

#### Keywords: correlative imaging, immuno-EM, CLEM, connectomics, multibeam SEM

**Abbreviations:** CLEM, correlative light and electron microscopy; CNS, central nervous system; EM, electron microscopy; GFP, green fluorescent protein; iEM, immuno-electron microscopy; ISH, in situ hybridization; LA-CLEM, large-area CLEM; LM, light microscope; PB, phosphate buffer; PBS, phosphate buffered saline; r.t., at room temperature (24–25◦C); SEM, scanning electron microscopy; TEM, transmission electron microscopy; UA, uranyl acetate.

# INTRODUCTION

fncir-13-00029 May 6, 2019 Time: 16:10 # 2

Comprehensive investigation of neural circuits in relatively large and complex brains such as those of humans and marmosets requires simultaneous low- and high-magnification observations within each layer of the cerebral cortex. To elucidate the structural interconnection between neurons in neocortices at both of these levels, connectomics analysis based on the knowledge of neocortical layer development is critical. Neocortical development involves three key processes: neurogenesis, migration and differentiation/maturation. The mature mammalian neocortex has a six-layered structure; the neurons in each layer of the neocortex are generated by division of neural stem/progenitor cells that surround the lateral ventricles of the embryonic forebrain (Sidman et al., 1959; Takahashi et al., 1995). These neurons migrate radially toward the pial surface in an inside-out manner (Rakic, 1972) and express a specific pattern of "marker" proteins (Hevner, 2007). Thus, by detecting these layer-specific markers using immunohistochemical staining or in situ hybridization (ISH), it is theoretically possible to specify the layer position of specific neurons of interest in which the synapses have been analyzed by EM. However, such analyses have been constrained by technical challenges due to the fact that procedures that combine results from light microscopy (LM) and EM require the use of different instruments and sample preparation methods and by the fact that both LM and EM demand high levels of expertise. CLEM has begun to enable the elucidation of subcellular architectures and morphologies (Begemann and Galic, 2016). Traditionally, CLEM is performed by correlating results obtained from LM and TEM. Fluorescence microscopy has the advantage of visualizing immunolabels that recognize specific molecules using antibodies or fluorescent proteins such as GFP (Giepmans, 2008; Watanabe et al., 2011). Fluorescent dyes can be distributed to a target area or to molecules in a relatively wider field with optimal efficiency and can be detected by LM. However, the spatial resolution of conventional LM is restricted to a few hundred nanometers at best due to the diffraction of light. Super-resolution light microscopy was developed to overcome this diffraction barrier, and its developers were recognized with the Nobel Prize in Chemistry in 2014 (Chereau et al., 2015). Because fluorescence imaging inherently focuses on labeled objects, peripheral cellular structures often remain poorly visualized. EM yields much higher-resolution images than LM but is difficult to use to observe large tissue areas or to make precise observations of highly dynamic processes such as those that occur in the human brain or in living cells (Giepmans, 2008; Watanabe et al., 2011; Chereau et al., 2015). Although CLEM has been used for decades, until recently it has only been applied to small-volume samples. The development of improved CLEM techniques has enabled scientists to achieve nanometer resolution analyses in samples that are more than several mm<sup>2</sup> in area, including samples of the gyrencephalic brain (Eberle et al., 2015a). Using multibeam SEM, we have developed a novel implementation, LA-CLEM, that offers additional advantages for the detection of molecular localization in large areas of the CNS at EM resolution and faster speeds. Visual information provided by layer-specific markers in EM images proved helpful in understanding the precise location of observed samples, particularly in the cerebrum of the common marmoset, which is much larger than that of mouse.

Transmission electron microscopy of ultrathin sections obtained from human biopsy or autopsy samples or rodent brain and collected on an EM grid has traditionally been used to observe synaptic connections between neurons (**Figure 1A**). In this process, brain tissues are dissected into small pieces of <1 mm and fixed with glutaraldehyde and osmium. The brain tissue block embedded in the plastic is sectioned at a thickness of approximately 50–80 nm using a diamond knife, and the sections are collected on an EM grid. This procedure remains in common use for the observation of synaptic structure. Recent improvements in the resolution of SEM images now enable the observation of synaptic structure by back-scattered electron imaging and by secondary electron imaging. For largearea EM observations, section SEM is now frequently used (**Figure 1B**). In this procedure, sample preparation is similar to that for TEM except for the collection of the ultrathin sections on flat conductive substances including silicon wafers, conductive coated glass, or conductive tape rather than on an EM grid. Observation of neural circuitry by EM, when combined with visualization of specific layer components in the cerebrum by fluorescence and EM, yields an unprecedented depth of information on the complex features of the gyrencephalic brain. Below, we introduce a new approach, LA-CLEM, that makes it possible to observe samples several millimeters square in area at resolutions that make it possible to detect individual synapses (**Figure 1C**). To identify the cerebral layer in which these synapses reside, the most common approach is the use of antibody staining or ISH to label layer-specific markers. The localization of specific target proteins and nucleotides (RNA/DNA) was demonstrated not only by fluorescence at the LM level but also by gold with iEM (immuno-EM) at the EM level (**Figure 1D**). Antibodies against layer-specific marker molecules, including antibodies against calbindin, calretinin, RORβ, Cux1, and FoxP2, are often used to evaluate normal layer formation. By combining immuno-EM and large-area SEM imaging, LA-CLEM can be used to visualize the localization of specific molecules in large areas at super-high resolution.

In this report, we present the detailed procedure of LA-CLEM, a combination approach involving pre-embedded immuno-EM and multibeam SEM technology that has been adapted for use in the marmoset cerebral cortex.

#### SAMPLE PREPARATION

#### Animals

Adult common marmoset monkeys (Callithrix jacchus, CLEA, Tokyo, Japan, n = 3), adult mice (Mus musculus, C57BL6/j from Japan SLC, Shizuoka, Japan, n = 10), and Sox10-Venus BAC transgenic mice (n = 4) (Shibata et al., 2010) were used in this study. Housing of animals and all animal experiments were conducted in compliance with the Guidelines for the Care and Use of Laboratory Animals of Keio University School of

(A,D). It requires several days to image the entire 1 mm<sup>2</sup> area on the grid. In contrast, a 2–3 mm<sup>2</sup>

restricted by the width of the diamond knife, and it requires several hours to image the entire 2–3 mm<sup>2</sup> area on the silicon wafer.

Medicine (approval numbers 11006-2 and 09091-12) and the Central Institute for Experimental Animals (approval numbers 16023 and 17031). All efforts were made to reduce the number of animals used and to minimize animal suffering.

# PRIMARY SAMPLE PREPARATION FOR LA-CLEM

The basic procedure used in immunohistochemical analysis was performed as described previously (Shibata et al., 2010). Briefly, animals were deeply anesthetized by intramuscular injection of ketamine (50 mg/kg, Fujita Pharmaceutical, Tokyo, Japan) and xylazine (4 mg/kg, Bayer, Leverkusen, Germany) for marmosets and by an overdose of isoflurane (Pfizer) inhalation for mice. Vascular perfusion was performed using a saline (0.9% NaCl, Sigma, St. Louis, MO, United States) rinse followed by 4% paraformaldehyde (PFA, 16%, Electron Microscopy Sciences, PA, United States), pH 7.4, in phosphate buffered saline (PBS from 10×, Nacalai tesque, Kyoto, Japan) that had been chilled on ice (Step #1 in **Table 1**). Perfusion with the fixative (300 ml and 30 ml) was conducted at approximately 20 ml/min and 2 ml/min for marmosets and mice, respectively. The target area in the brain tissue was dissected into coronal slices 3–6 mm thick using a 76 µm-thick cutting blade (Nisshin EM Co., Ltd., Tokyo, Japan) and a marmoset brain matrix. The sectioning matrix specific for marmoset brain was designed from three-dimensional (3D) data obtained from magnetic resonance imaging (MRI) (**Figures 2A,B**). The pieces of the perfused brain were postfixed with 4% PFA in 0.1 M PB, pH 7.4 for 10–12 h at 4◦C. Tissue blocks were cryoprotected by incubation in 15% and then 30% sucrose solutions (Nacalai tesque, Kyoto, Japan) for 12 h each and embedded into cryomolds (Tissue-Tek, Sakura Finetek, Tokyo, Japan) with cryocompound (Leica Biosystems, Wetzlar, Germany) for subsequent cryostat sectioning. Frozen sections (20 µm thick in this case) were prepared using a cryostat (Leica CM3050s, Leica Biosystems, Wetzlar, Germany), placed on

is a section approximately 1 mm<sup>2</sup>

section is the average size

#### TABLE 1 | Detailed procedure for LA-CLEM imaging with mSEM.


(Continued)

#### TABLE 1 | Continued

fncir-13-00029 May 6, 2019 Time: 16:10 # 5


mSEM, multibeam SEM; MW, microwave; Ms, mouse; Rab, rabbit; Ab, antibody; o/n, overnight; r.t., room temperature (24–25◦C); 2 h, 2 hours.

1 cm.

thawed and redried, followed by the application of blocking solution on the day of antibody staining. Scale bars: (A) 1 cm, (B) 2 mm, (C) 1 cm, (D) 2 mm, (E) – (F)

adhesive microscope slides and dried on a hot plate for 2–3 h at 37◦C until the sections were tightly attached to the slides (Step #2 in **Table 1** and **Figures 2C,D**). Folding and wrinkling of the tissue sections inhibit the preparation of ultrathin sections after resin embedding, making it important to ensure that the sections are as flat as possible so as to obtain larger flat samples for EM observation. The dried sections on the slides were stored in either a −30 or a −80◦C freezer in a cryosection box until antibody staining on experimental day 1 (Step #3 in **Table 1** and **Figure 2E**). The type of coating on the slides and the materials from which the slides were constructed are critical for LA-CLEM. For general immunostaining, microscope slides have multiple adhesive coatings and are positively charged to keep the section tightly fixed to the slides even after exposure to solution for several days. In some cases and depending on the coating conditions, it may be difficult to remove tissue sections from the slides, as described in Step #36 in **Table 1**. Prior to experiments involving samples of limited availability, trial removal of the resin should be performed to confirm error-free processing. Glass slides, which are commonly used in immunostaining due to their flatness, hardness, and limited autofluorescence, can also be used in the LA-CLEM procedure. However, it is easier to remove specimens from plastic slides, and these are also sometimes used in immunostaining, as described in Step #36 in **Table 1**.

## IMMUNOHISTOCHEMICAL PREPARATION WITH ANTIBODY APPLICATION

The LA-CLEM procedure was based on pre-embedding iEM, as previously described (Shibata et al., 2015b); the samples were subsequently analyzed by multibeam SEM. On day 1, frozen sections were thawed and dried under a cool dryer for 10–30 min at room temperature (r.t.; Step #4 in **Table 1**). Before applying blocking solution, we often use liquid blocker or a pap pen to create a barrier that keeps the blocking solution and antibody solution on the section and to prevent contamination of the slides by other solutions or leakage from the slide top. Dried sections were washed with 0.1 M PB 3 times for 3 min at r.t. and then placed in a moist chamber (Step #5 and **Figure 2F**).

Specific pretreatments for immunostaining, including antigen retrieval using an autoclave and microwaving, can be applied to the sections before the blocking solution is applied (Step #6 in **Table 1**). Depending on the requirements for each antibody, pretreatment for antigen retrieval was conducted in special solutions, such as pH 6.0 citrate buffer, by heating the slides in a heat-resistant staining pot in an autoclave or microwave. Other commercially available solutions, including pH 6.0 target retrieval solution (TRS from DAKO), should be evaluated on a sampleby-sample basis. It is necessary to allow approximately 1 h for the sections and solution to cool completely to room temperature before performing the PBS wash for 3 min (Step #7 in **Table 1**).

Blocking solution with or without detergent (saponin, Merck, Darmstadt, Germany) was applied to the sections to block nonspecific antibody binding (Step #8 in **Table 1** and **Figure 2F**). We usually used 5% BlockAce (DS Pharma Biomedical, Osaka, Japan) with 0.01% saponin (Merck, Darmstadt, Germany) in 0.1 M PB (Muto Pure Chemicals, Tokyo, Japan) for 0.5–1 h. Commercially available blocking solutions other than BlockAce, such as Blocking Reagent (PerkinElmer, MA, United States) and Blocking Buffer (ab126587, Abcam, Cambridge, United Kingdom), can be used for blocking. The use of donkey, goat, and horse serum is also acceptable if the species from which the primary antibody is derived differs from the species from which the serum used in blocking was derived. It is critical to use a detergent such as saponin, Triton X-100 (FUJIFILM Wako Pure Chemical Corporation, Osaka, Japan), Tween (Sigma, St. Louis, MO, United States), or SDS (Nacalai tesque, Kyoto, Japan) at the minimum required concentration for smooth antibody infiltration. The stronger the detergent we used, the lower was the signal from cell membranes detected with EM due to the breakdown of the lipid bilayer by active permeabilization with detergent.

If detergent is not used in antibody staining, tissue preservation should be much better for EM observation. However, it is difficult for antibodies to enter brain tissue. We attempted to evaluate the penetration of the antibodies and nanobodies in the absence of detergent and after minimal detergent treatment (Fang et al., 2018). The cerebral cortices of Sox10-Venus transgenic mice (Shibata et al., 2010) were dissected, and coronal brain slices 800 µm thick were prepared promptly after perfusion of the animals with 4% PFA pH 7.4 in PBS on ice using a vibrating blade microtome (Leica VT1000 S, Leica Biosystems, Wetzlar, Germany). After blocking with blocking solution containing no detergent, the slices were stained with antibodies and nanobodies, followed by fluorescence labeling with secondary antibodies and Hoechst dye (Hoechst 33258, Sigma, St. Louis, MO, United States), respectively (**Figures 3A,B**). Stained brain sections were vertically sliced in the sagittal dimension at 100 µm thickness, and the penetration of the fluorescence was evaluated. Antibodies did not infiltrate sections several micrometers in thickness without detergent, but fluorescence-conjugated nanobodies penetrated brain tissue to a depth of several hundred micrometers (**Figure 3B**) (Fang et al., 2018). Nanobodies are a promising immuno labeling reagent due to their small size; however, the available selection of nanobodies and gold labeling systems is currently very limited compared to the selection of antibodies, which number more than a million. As described in Steps #9–11 in **Table 1** and **Figure 3C**, the selection of the procedure mainly depended on the purpose of the experiment and the available reagents (antibodies and nanobodies). The main focus of this study is to identify the cerebral cortical layer positions of neurons in large marmoset brain sections using well-known and widely used antibodies.

Sections incubated with blocking solution (5% BlockAce, DS Pharma Biomedical, Osaka, Japan) with 0.01% saponin for permeabilization were incubated with primary antibodies in blocking solution for 3–4 days at 4◦C (Step #9 in **Table 1**, same as **Figure 2F**). Dilution of the antibodies should be evaluated by light microscopy using a solution similar to that used in LA-CLEM. In this study, the following layerspecific primary antibodies were used: rabbit anti-calbindin

(Category 2). For EM observation, gold labeling or DAB (3,3<sup>0</sup> -diaminobenzidine tetrahydrochloride) reaction with HRP (horseradish peroxidase) was required to visualize the antibody localization (Category 3). In this study, fluorescence- and gold-conjugated secondary antibodies were mainly used (Category 4). When an appropriate dual-labeled secondary antibody was not available, the use of biotin-conjugated secondary antibodies and fluorescence- and gold-conjugated streptavidin helped complete the procedure (Category 5).

(1:500, Chemicon, Merck, Darmstadt, Germany), mouse anticalretinin (1:200, Swant Inc., CH-1723, Switzerland), chick anti-Tbr1 (1:100, Chemicon, Merck, Darmstadt, Germany), mouse anti-neurofilament H (1:250, clone N52, Sigma, St. Louis, MO, United States), anti-RORβ (1:200, Perseus Proteomics Inc., Tokyo, Japan), rabbit anti-Cux1 (1:500, Santa Cruz Biotechnology, Inc., Dallas, TX, United States), and goat anti-FoxP2 (1:200, Santa Cruz Biotechnology, Inc., Dallas, TX, United States). LA-CLEM can be adapted for use in any host animal (mouse, rabbit, chick, rat, guinea pig, sheep, goat, human, etc.) by using gold- or biotin-conjugated secondary antibodies (**Table 2**).

On day 2, we washed the samples 10 times for 10 min each time with 0.1 M PB and 0.005% saponin for a total of approximately 2 h at r.t. (Step #10 in **Table 1**). While washing, secondary antibodies were prepared for Step #11 by dilution in blocking solution containing BlockAce with 0.01% saponin in PB, as described above. In our laboratory, we used a FluoroNanogold-conjugated secondary antibody (Alexa Fluor 488- and Nanogold-conjugated goat anti-mouse or anti-rabbit antibody, 1:100, Thermo Fisher Scientific, MA, United States) for mouse and rabbit primary antibodies (Step #9 in **Tables 1**, **2**). For detecting antibodies prepared in other species, such as chick, rat, guinea pig, sheep goat, and human antibodies, fluorescence and gold dual-labeled secondary antibodies are commercially available, and biotin-conjugated secondary antibodies (1:500, Jackson Immuno Research, West Grove, PA, United States, or Vector Laboratories, CA, United States) with fluorescence

and gold dual-labeled streptavidin can be used (Step #11 in **Table 1** and **Figure 3C**). Alternatively, a 1- or 1.4-nm colloidal gold-conjugated secondary antibody (Nanoprobes Inc., NY, United States) can also be used for Step #11 in **Table 1** in conjunction with the fluorescence-conjugated secondary antibodies (**Table 2**).

In the case of the samples in Step #11 in category 5 in **Figure 3C**, high-affinity binding of biotin to streptavidin was applied on experimental day 3. After 10 washes with 0.1 M PB and 0.005% saponin, the sections were incubated with FluoroNanogold-conjugated streptavidin (Alexa Fluor 488 and Nanogold-conjugated streptavidin, 1:100, Thermo Fisher Scientific, MA, United States) for 24 h at 4◦C (Steps #12 and #13 in **Table 1**) along with Hoechst 33258 (10 µg/ml, Sigma, St. Louis, MO, United States) for nuclear staining. The inclusion of these additional steps (#12 and #13) on day 3 meant that the total experimental schedule had to be adapted to the host species from which the primary antibodies were obtained and to the lineup of secondary antibodies. In our case, the busiest experimental day, day 4, was usually fixed first, and the schedule for applying primary and secondary antibodies was adjusted later depending on the host species from which the primary antibody was obtained.

## FLUORESCENCE IMAGING WITH LIGHT MICROSCOPY

After washing several times with 0.1 M PB and 0.005% saponin for approximately 2 h in Step 14, the immunostained samples were observed using a confocal laser scanning microscope (LSM880, Carl Zeiss, Oberkochen, Germany) or a fluorescence microscope (BZ-9000, Keyence, Osaka, Japan) on experimental day 4 (Steps #14 and #15 in **Table 1** and **Figures 4A**, **7A– C**). The sections were soaked in buffer (0.1 M PB) rather than in mounting medium. To avoid damage to the sections due to direct contact with the cover glass, we maintained a small space between the slide glass and the cover glass by attaching adhesive tape or positioning an additional cover glass at the edge of the slide glass to create an artificial space. To identify the nuclear localization and tissue structure during fluorescence imaging, nuclear staining dyes (Hoechst or DAPI) were usually included in the secondary antibody solution. Moreover, multicolor imaging with differently colored dyes can be conducted simultaneously by identifying the other epitopes using additional sets of primary and secondary antibodies in Steps #9, #11, and #13.

As an example, four-color fluorescence images were obtained using Hoechst (blue), anti-FoxP2 (green), anti-RORβ (red), and anti-Cux1 (far red), and EM images were obtained using FoxP2 (gold) labeling. Goat anti-FoxP2, mouse anti-RORβ and rabbit anti-Cux1 antibodies were applied as a set of primary antibodies on day 1. Biotin-conjugated donkey antigoat secondary antibody (1:500, Jackson Immuno Research, West Grove, PA, United States) was applied on day 2. The Hoechst dye, FluoroNanogold-conjugated streptavidin (Alexa Fluor 488 and Nanogold), Alexa Fluor 555-conjugated goat anti-mouse IgG and Alexa Fluor 647-conjugated goat anti-rabbit secondary antibodies were applied on day 3 (**Figure 4A**).

As demonstrated in **Figure 4B**, which shows a lateral side view of a section with Z-stack imaging after the Cux1 antibody reaction, 20-µm-thick cryostat sections were fully infiltrated by the primary and secondary antibodies without any gaps (**Figure 4B**). The thickness of cryostat sections varies from laboratory to laboratory. In our laboratory, sections were usually prepared at a thickness of 50–100 µm for free-floating vibratome sections, 10–20 µm for frozen cryostat sections, 5–10 µm for paraffin sections, 50–80 nm for resin-embedded EM sections, and 30–90 nm for resin-embedded SEM observation (**Figures 4C,D**). Cryostat sections 20 µm in thickness were the maximum thickness that allowed complete infiltration of the antibody described in Steps #9 and #11 in **Table 1**; this thickness is also ideal for ultrathin sectioning with an ultra-microtome for TEM and SEM observation as described in Step #48 in **Table 1**.

## SECTION PROCESSING FOR EM BLOCK PREPARATION

Soon after completing the fluorescence imaging, the sections were washed with 0.1 M PB for 5 min at r.t. and fixed with 2.5% glutaraldehyde in PB for 10 min at r.t. for EM-grade fixation (Steps #16 and #17 in **Table 1**). To minimize exposure to the vapor produced by the toxic reagents and reduce the amount of solution required for each process, a plastic slide case that holds five slides (MR-500, Matsunami glass, Osaka, Japan) is convenient for processing, especially for glutaraldehyde fixation at Steps #16– #18 and for osmium staining, dehydration and Epon infiltration at Steps #22–#34. The use of dummy empty slides to fill the empty wells helps reduce the amount of solution required.

The sections were washed again with 0.1 M PB for 5 min at r.t. and buffered with 50 mM HEPES (FUJIFILM Wako Pure Chemical Corporation, Osaka, Japan) (pH 5.8) for half an hour (10 min × 3 min) at r.t. (Steps #18–#19). Adjustment of the pH for this buffer used 1 N NaOH since Cl<sup>−</sup> ions from hydrochloride (HCl) generate a white precipitate with Ag<sup>+</sup> ions that increases the background. Silver enhancement was required to enlarge the Nanogold or 1-nm colloidal gold signal due to the small size of these reagents. Silver enhancement was conducted using an R-gent Se-EM kit (Aurion, PD Wageningen, Netherlands) and developed for approximately 30–40 min at r.t. in a bright room or with the HQ-silver kit (Nanoprobes Inc., Yaphank, NY, United States) for approximately 10–12 min at r.t in a dark room (Step #20 or #20<sup>0</sup> in **Table 1**). When the R-gent Se-EM kit was used, 10 or 20 droplets of activator and one droplet of the initiator (10:1 or 20:1) were mixed well by vortexing to prepare the developer, and 50 droplets of the enhancer and 10 or 15 droplets of developer (10:2 or 10:3) were mixed with vortexing to prepare sufficient reaction solution for processing five slides (**Figures 5A–D**). When the HQ-silver kit was used in the dark room, 20 droplets of solution A and 20 droplets of solution B were mixed well; 20 droplets of solution C were then added to the tube and mixed well with vortexing to prepare sufficient solution (1:1:1) for processing 5 slides. Stopping of the enhancement

reaction was determined by the timing of the color change of the sections to brown or gray. To stop the silver enhancement reaction, the sections in a slide basket were actively washed five times in distilled water (DW) at r.t. for 1 min followed by a wash in 0.1 M PB for 1 min at 4◦C in the slide glass plastic case (Step #21 in **Table 1**). The sections were postfixed with osmium tetroxide (OsO4, Nisshin EM Co., Ltd., Tokyo, Japan) for 90–120 min at 4◦C, and 15 ml solution was used for 5 slides in a plastic case (Step #22 in **Table 1**). After removal of the osmium solution, the slide case was washed with DW once for 5 min followed by two incubations with 50% EtOH for 5 min each time (Steps #23 and #24 in **Table 1**).

To enhance the membrane contrast of the EM images for the whole block, en bloc staining with 2% UA solution in 50% EtOH was performed for 20 min at 4◦C (Step #25 in **Table 1**). As found in various trials to improve EM image quality and summarized in **Table 3**, en bloc staining with UA was one of the most effective factors in our trial. Dehydration in graded concentrations of ethanol (70, 80, 90, and 100% EtOH) was performed twice at each concentration for 5 min (Steps #26–29). At the step in which the sample is exposed to absolute ethanol (100% EtOH), the lines from the liquid blocker should be removed from the tops of the slides using a razor blade while the slides are immersed in a 10- or 15-cm plastic dish filled with absolute ethanol to facilitate smooth removal of the section after polymerization (Step #29 in **Table 1** and **Figure 5E**).

To replace the solution with 100% Epon for polymerization, acetone was applied for 5 min at r.t. followed by the application of QY1 twice for 5 min at r.t. (Steps #30 and #31 in **Table 1**). The sections were exposed to the resin-containing solution QY1:Epon (1:1) for 1 h at r.t. and then to 100% pure resin several times at r.t. (Steps #32 and #33) and overnight at 4◦C (Step #34 in

**Table 1**). Steps #30 to #32 were usually used only with glass slides, glass chamber slides, and vials made of glass with sufficient solvent resistance. Because plastic slides have limited resistance

were fully polymerized by incubating at 60◦C for 72 h. Scale bars: (A,C,E)–(G) 1 cm; (B,D,H) 5 mm.

to solvents, plastic slides should not be exposed to acetone or to QY1. Sections on plastic slides were exposed to gradually increasing concentrations of the resin in absolute ethanol [100%

embedded in a silicone mold for slides. Air bubbles under the slides should be removed before beginning polymerization. (G,H) Brain slices stained with antibody

EtOH:100% Epon = 25% (3:1), 50% (1:1), 75% (1:3) for 10 min each at r.t. (Steps #300–#32<sup>0</sup> in **Table 1**]. The sections were then exposed to 100% Epon for 1 h at r.t. and incubated overnight at 4◦C (Step #33). At the final step before beginning the polymerization (Step #34), the slides on which the sections were mounted were transferred to a new plastic slide case containing 100% pure resin to minimize carryover from the previous solvent.

At least one overnight infiltration of 100% pure Epon was used both for slides made of plastic and for slides made of glass; the slides were then embedded in new resin in a slide glass embedding mold made of silicone (SNP2, microstar, Tokyo, Japan) for 72 h (approximately three overnights) at 60◦C for polymerization (Step #35 in **Table 1** and **Figure 5F**). To achieve complete curing of the resin (**Figures 5G,H**), it is important to maintain the optimal temperature for the resin (60◦C in our case, from datasheet) for a sufficient period. We monitored the actual temperature on a minute-by-minute basis during the entire period using a temperature probe. When the temperature reached 60◦C, approximately 1 h after the slides were placed in the oven for polymerization, the air bubbles under the slides were removed using small wooden toothpicks to maintain the thickness of the resin on the section.

#### EM BLOCK SECTIONING AND WAFER PREPARATION

After curing was completed, the sections in the polymerized resin were manually removed from the silicone mold (**Figure 6A**). Using sectioning blades, the sections were removed from the slides on the hot plate by an experimenter wearing anti-injury gloves that were resistant to cutting (Step #36 in **Table 1** and **Figure 6B**). The temperature of the sections and the coating of the slides were crucial for smooth removal (**Figure 6C**). The temperature setting of the hot plate should be adjusted depending on the type of resin and the hot plate. With our resin composition, an iron top hot plate (Taitec Corporation heat block, Saitama, Japan) was set to approximately 100◦C. This method was well suited to removal of the specimen at 95–105◦C as measured by an infrared thermometer. If the resin was not sufficiently hot (85–95◦C), it was difficult to remove the specimen from the slide. If it was too hot (>110◦C), the resin was easily fragmented into small pieces, presenting the worst condition for block preparation. It is simple to remove resin-embedded specimens from slides made of plastic on a hot plate with limited use of blades (**Figure 6D**).

Soon after completing the smooth removal and with the sample on the same hot plate, the sections were dissected into blocks several millimeters square (Step #37 in **Table 1** and **Figure 6E**). Using small forceps, the blocks were placed on the empty resin block (sectioning stage) and glued to the block using old resin with high viscosity (stored in the freezer in a syringe, Nipro, Osaka, Japan). The tissue sections should be placed on top of a uniquely numbered sectioning stage (**Figures 6E–G**; the white arrow in **Figure 6E** indicates the target section in the experiment at this time point). The block should never be placed upside down. The glued blocks were incubated for 24 h at 60◦C. After fixing on the block, the sectioning stage with a piece of the section on the resin block was stored in a desiccator for at least several hours in a paper sample storage box to prevent loss (Step #38).

Ordinarily, one block would be sufficient for large-area imaging; however, we always fix all small blocks on the sectioning stages for two important reasons: to permit numbering of all the small blocks, and to preserve the adjacent blocks in case these are needed. There is unique numbering on the lateral side of the sectioning stage made up of resin blocks (**Figure 6F**). Both the original position of the brain section and the block number information are always clearly recorded in a notebook. This makes it easier for us to determine the original location of the immunostained section. If we do not place small pieces of blocks on the stage and save them separately in a paper storage box, it is difficult to identify the original position of the antibody-stained section on the resin block. In some cases, it is necessary to prepare additional ultrathin sections from adjacent blocks due to the occurrence of cracks, breaks, or bumpy surfaces that interfere with ultrathin sectioning. For this reason, most of the fluorescence imaged area, at least, is usually prepared as blocks for EM sectioning. The blocks were trimmed to a size of several millimeters square using a sectioning blade, a glass knife or a diamond trimming knife in an ultramicrotome (Leica UC7, Leica Biosystems, Wetzlar, Germany or RMC ATUMtome, Boeckeler Instruments, Inc., Tucson, AZ, United States) for preparing ultrathin sections (Step #40 in **Table 1**). The size of the sectioning surface was determined by the object, the width of the diamond knife and the flatness of the surface. To obtain full-layer sections from marmoset cerebral cortex, a surface with an area of approximately 2 mm × 3 mm was sufficient to cover all layers (from layers I to VI). To distinguish the pial surface from the ventricular side of the sample, it is useful to prepare the block surface in a trapezoidal shape or in a home base shape that makes it easy to identify the top of the brain.

Ultrathin sections were prepared at a thickness of approximately 30–90 nm using a diamond knife in an ultramicrotome at r.t. (approximately 24◦C) (Step #40). Ultrathin sections prepared using the ATUMtome (Boeckeler Instruments, Inc., Tucson, AZ, United States) were collected on conductive tape, manually transferred to silicon wafers or collected on copper grids (Step #41 in **Table 1** and **Figure 6I**). The thickness of the sections was set using the ultramicrotome and was limited by the type and hardness of the resin. For large-area imaging with SEM, ultrathin sections with a thickness of 50–80 nm were prepared, and the sections were transferred to the silicon wafer from the diamond knife boat using a ring transfer (microstar, Tokyo, Japan) (**Figure 6J**). For section collecting, a silicon wafer or tape for ATUMtome that had been treated with plasma to obtain a clean, hydrophilic surface was used. The larger samples were transferred from the knife boat using a water-filled beaker (**Figure 6K**). The diamond knife boat containing several sections was dipped into a water-filled beaker. The sections floating on the water were collected directly onto the silicon wafer or conductive tape. For TEM observation, sections approximately 50–80 nm in thickness were manually collected on a copper grid (#100 or #150 Veco, Nisshin EM

the resin were removed from the slide glass on a hot plate using a sectioning blade. (C) The temperature of the section on the hot plate was critical for the smooth removal of the section from the glass slide. Heating the resin to 90–100◦C yielded the best results. (D) A plastic slide (plastic chamber slide) softened on the hot plate and was smoothly detachable from the resin. (E) Removed sections were dissected on a hot plate into several-millimeter-square blocks using a fine sectioning blade for ultrathin sectioning with a diamond knife. The white arrow shows the block imaged in Figure 7. (F) Used resin, which has a high viscosity, was placed on the empty resin block in the capsule stand to serve as a kind of "glue." (G) Tissue sections on the sectioning block were placed on the top of the sectioning stage with small forceps. The glued blocks with the sections on top were incubated for 24 h at 60◦C until the sections were strongly attached. (H) Blocks were trimmed using a sectioning blade, glass knife, or diamond trimming knife, and ultrathin sections were prepared using an ultra-microtome or an ATUMtome. (I) Ultrathin sections were manually collected on copper grids or silicon wafers. (J,K) For SEM observation, sections were transferred to a silicon wafer from the diamond knife boat using a ring transfer or by manual collection in a water-filled beaker. Scale bars: (A–G) 1 cm, (H) 2 mm, (I) 1 cm, (J) 3 mm, (K) 1 cm.

Co., Ltd., Tokyo, Japan). All sections were dried completely in a desiccator for several hours at r.t. (Step #42).

Ultrathin sections were prepared for EM using UA for 10 min at r.t. and lead citrate (Pb) for 10 min. Sections on a tape of approximately 5–10 cm in length (depending on the staining tube), sections on silicon wafers attached to a silicon wafer holder (microstar, Tokyo, Japan), and sections on copper grids attached to a grid staining stick (microstar, Tokyo, Japan) were dipped into UA solution for an optimal period of time at r.t. (Step #43 in **Table 1**). The sections were then washed three times with DW in a beaker at r.t. (Step #44). The sections were dipped into Pb solution at r.t., followed by three washes with DW (Steps #45 and #46). The ultrathin sections on the tape and on the silicon wafer were gently blown with an air brush, the remaining water droplets were removed from the top, and the sections were dried completely on clean filter paper for several hours at r.t. (Step #47 in **Table 1**).

### ELECTRON MICROSCOPIC IMAGING WITH MULTIBEAM SEM

Electron microscopic observation was conducted with multibeam SEM (multiSEM 505, Carl Zeiss, Oberkochen, Germany), single-beam SEM (SU6600 from Hitachi High-tech, Tokyo, Japan, Sigma from Carl Zeiss, Oberkochen, Germany), and TEM (JEM1400plus JEOL, Tokyo, Japan) according to the manufacturer's instructions (Step #48 in **Table 1**). For LA-CLEM imaging, multibeam SEM is one of the most powerful microscopic techniques available for use in high-speed imaging of large sections of brain tissue. The sections on the collecting tapes were first placed on the silicon wafer with conductive double-sided adhesive tape. The silicon wafers containing the sections were attached to the specimen holder with silver DAG and imaged using an optical microscope, Imager Vario (Carl Zeiss, Oberkochen, Germany), to confirm the precise position at low magnification as a reference position from which the fluorescence images were obtained (Step #15 in **Table 1** and **Figures 7A–C**). After establishment of the workflow with multibeam SEM so as to observe the brain sections with the best focus and with the best acquisition parameters, the experiment was begun with automatic focus acquisition of a sufficient number of tiled images to cover the target section (**Figure 7D**). The mosaic image files were automatically generated soon after completing the imaging (**Figure 7E**). On zooming into the specific region indicated by the white box in **Figures 7A–C,E**, the localization of the fluorescently stained nucleus was clearly demonstrated (**Figure 7F**). Low-magnification EM images can be used to identify the specific nuclear localization (yellow) of the RORβ (**Figure 7G**). In **Figure 7F**, the red circles in (G<sup>0</sup> ) originate from the yellow-colored nucleus in **Figure 7G** and are superimposed on the fluorescence images shown in (A<sup>0</sup> ) and (B<sup>0</sup> ). The pattern of the red circles in **Figure 7F** does not completely match the fluorescence and Hoechst images in **Figures 7A,B** due to the difference in thickness of the immunostained cryostat sections and ultrathin sections. The higher-magnification imaging demonstrated that the gold signals were mostly localized

in the nucleus and enabled the observation of myelin and synapses (**Figure 7H**). To confirm the detailed structure of the tissue using another microscope at different magnification, single-beam TEM observation of adjacent ultrathin sections placed on the grids was conducted (**Figure 7I**).

To enhance the accuracy of the overlay between the fluorescence and EM images of large brain sections, we performed multicolor immunostaining using landmark markers,

nucleus (Hoechst) provided useful clues for finding the precise location of the EM block because these were visible in EM observation. Inset: actual surface image of the resin block corresponding to the white square windows in panels (A–D); the white arrows indicate landmark blood vessels. (E–H) As shown in Table 3, various trials were conducted to find methods for improving the quality of the LM and EM images. The EM images shown in (E,F) were produced using method (a), and the images shown in (G,H) were produced using method (f) in Table 3. The images were evaluated by multibeam SEM (E,F) and by TEM (F,H). n, nucleus; arrowheads, synapse. Scale bars: (A–D) 1 mm, (E–H) 1 µm.

including markers for blood vessels (VE-cadherin, Santa Cruz Biotechnology, Dallas, TX, United States), the nucleus (Hoechst) and the pia mater (physical edge) (**Figures 8A–D**). As shown in the inset in **Figure 8D**, the EM block surface also contained a superjacent section of the pia mater and many blood vessels (white arrows) of various diameters. These positional clues are effective not only for our manual correlative observations but also for the computer-based correlative analysis by AI in future.

We also sought to evaluate various approaches that could be used to enhance the image quality of our LA-CLEM observations. A limited number of sample preparation conditions are summarized in **Table 3**. The overall LM/EM image quality obtained using different conditions, including the use of glass or plastic slides, pretreatment for antigen retrieval, detergent use, postfixation glutaraldehyde and OsO4, and UA en bloc staining, was compared. For example, whereas autoclaving with citrate buffer pH 6.0 is one of the best procedures for antigen retrieval, autoclaving in target retrieval solution (Dako) was powerful and effective for LM imaging but harmful for EM imaging, offering limited microstructure preservation. One of the best procedures tested was method (a) in **Table 3**. The images shown in **Figures 8E,F** were obtained with multibeam SEM and TEM, respectively. The application of reduced OsO<sup>4</sup> in the postfixation step was also effective for drastic enhancement of the membrane contrast (**Figures 8G,H**); however, the resin-embedded sections were stuck to the slide glass very rigidly and were very difficult to remove. Identifying better conditions for improving the quality of the images for large sample observation using the LA-CLEM procedure remains an important challenge.

## DISCUSSION

In this study, we report a newly developed procedure, LA-CLEM, that can be used to visualize specific molecular localizations in large areas of the CNS at EM resolution and at high speed through the use of multibeam SEM. Information on layer markers in the EM images was helpful for identifying cortical layers in a given region, especially in the cerebrum of the common marmoset. This method may make it possible to rapidly observe large biological specimens, including specimens of human tissue, at EM-level resolution while obtaining information about molecular localization.

Correlative light and electron microscopy has often been used to obtain a correlation between images from LM and EM in very limited areas. In previous reports, various practical approaches using cultured cells or transgenic animal models such as Drosophila melanogaster, Caenorhabditis elegans, zebrafish, and mouse have been described (Karreman et al., 2016). To identify molecular position at higher resolution, CLEM technology combined with immuno-EM could overcome the limitations of LM by compensation with EM imaging (Cortese et al., 2009), although it is not easy to identify the same area or the same cell using both LM and EM at different magnifications in the same specimen. This is one of the reasons why general CLEM imaging has remained focused on limited areas.

There are two major approaches to obtaining images by EM, SEM, and TEM; these two methods detect signals using scattered electrons and transmitted electrons, respectively. Typically, SEM reveals the surface micromorphology of the specimen, while TEM can be used to visualize the internal composition of thin sections. Due to the basic strategy of the SEM/TEM image acquisition procedures, SEM is more suitable than TEM for the observation of larger areas. TEM can maximally observe an area of up to several mm in diameter within the EM grid at one time, while SEM can scan areas of approximately several cm<sup>2</sup> . Because CLEM imaging has usually been conducted with TEM, the observable area has remained limited (Chen et al., 2012; Kubota et al., 2015).

In contrast, TEM delivers much higher resolution than SEM; however, SEM technology is gradually improving and is now approaching the resolution of TEM. Recent advances have made it possible to obtain images of the internal composition of thin sections on a flat surface that are quite similar to those obtained using TEM (Marx, 2013). In this study, we sought to observe large areas of ultrathin sections of marmoset and mouse brain with SEM by detecting the secondary electrons

#### TABLE 2 | List of the antibodies and nanobodies used in this study.


from the surface of the sample. To increase throughput for numbers of large sections from the brain, we used multibeam SEM, increasing the number of primary beams and detectors to enlarge the imaging area compared to single-beam SEM (Marx, 2013; Eberle et al., 2015b). Multibeam SEM has opened a new era of EM observation, enabling nanoscale resolution imaging of areas on the order of mm<sup>2</sup> or cm<sup>2</sup> in size (Marx, 2013). In addition to parallel imaging with a multidetector in a single image, multiple scanning with precise tiling can be used to image the entire surface of large samples (Eberle et al., 2015a). The multibeam SEM that was used in this study achieved extraordinarily high-speed imaging with parallel electron beams. The specifications sheet of the multibeam SEM stated that a 1 cm<sup>2</sup> area can be imaged within an hour at 4 nm/pixel resolution.

The LA-CLEM procedure introduced in this paper is a novel approach in which CLEM is combined with multibeam SEM. Primary observation with LM was conducted to visualize the fluorescence localization, followed by observation of the same specimen with multibeam SEM at different magnification and resolution. Due to the use of antibody-specific fluorescent labeling, the molecular identity of each labeled cell in the monkey brain can be clearly categorized. Previously, 3D molecular localization in the primate brain was visualized by immunostaining; however, that study was conducted only at the LM level (Mikula et al., 2009). Gold-labeled signals can be detected simultaneously with the detailed intracellular structure revealed by EM, directly confirming the subcellular localization of targets in the nucleus, cytoplasm, cell membrane and synapse at EM resolution. By combining CLEM imaging with multibeam


SEM technology in the brain it is possible to identify the localization of specific neuronal subtype markers at the EM level in a large cerebral section, and such localization is important for efficiently determining the function and connection of specific neuron types. It would be highly beneficial to identify layerspecific markers that can provide information at the EM level for evaluating layer-specific connections in the brain.

Classically, SEM provides 3D images, while TEM provides two-dimensional (2D) images. Recent technological advances have made it possible to visualize the 3D structure of a specimen at EM resolution using SEM; the results not only show the irregularity and roughness of the surface but also provide multiple serial imaging of the 2D flat surface (Shibata et al., 2015a; Karreman et al., 2016). Some serial section EM (ssEM) approaches, including focused ion beam (FIB)-SEM, serial blockface electron microscopy (SBEM), and automated tape-collecting ultramicrotome (ATUM)-SEM, are available. The advantages of ssEM with FIB-SEM are that the highest Z resolution available by ultrathin slicing with the FIB is several nm and that the technique is applicable to hard tissues (teeth and bone) that are not suitable for cutting with a diamond knife. Instead of using a FIB, SBEM serial sectioning is performed by using a diamond knife to slice the top surface of the tissue, and the newly created surface is imaged with SEM. In contrast to the destructive techniques of FIB-SEM and SBEM, which destroy the sample as it is being imaged, in ATUM-SEM serial sections are produced by a standard ultramicrotome, collected automatically on tape, and imaged by SEM, offering the possibility of reimaging the same section multiple times if necessary (Kasthuri et al., 2015; Morgan et al., 2016; Hildebrand et al., 2017). In addition, it is possible to observe a larger area with sufficient conductivity by ATUM-SEM using the on-tape conductivity escape from the chargeup. By reconstructing the 3D structure of the sample from the images obtained with multibeam SEM, the LA-CLEM approach will enhance throughput and may become an important tool in the near future.

Whole brain-wide connectomics reconstructed at EM resolution requires novel procedures complemented by precisely evaluated fixation and staining procedures for preserving the cellular ultrastructure throughout the brain and sophisticated data processing protocols for the management of petabyte-scale data (Lichtman and Denk, 2011; Mikula and Denk, 2015; Mikula, 2016; Hildebrand et al., 2017). For the reliable reconstruction of neural circuits, the identification of synapses and the detection of cell bodies are critical. X-ray microcomputed tomography (X-ray microCT) and X-ray microscopy rely on the detection of X-rays transmitted through samples to visualize the internal

#### REFERENCES


morphological composition of the block (Bushong et al., 2015; Mikula and Denk, 2015). Improvements in X-ray 3D imaging will help enhance microscale imaging of the whole brain by supplying the information necessary for assessing brain integrity, including the formation of blood vessels and large bundles of nerve tracts (Mikula, 2016).

#### ETHICS STATEMENT

Housing of animals and all animal experiments were conducted following the Guidelines for the Care and Use of Laboratory Animals of Keio University School of Medicine (approved number 11006-2 and 09091-12), and of the Central Institute for Experimental Animals (approved number 16023 and 17031). All efforts were made to reduce the number of animals used and animal suffering.

#### AUTHOR CONTRIBUTIONS

SS, ES, TT, RS, JL, and HO conceived and designed the analysis. SS, TarI, TM, TS, NM, TN, and TakI conducted the experiments. SS, TarI, TM, TS, NM, TN, TakI, ES, and CA contributed to animal sample preparation. SS, TS, NM, TN, RS, and JL planned and conducted the electron microscopic imaging. SS, TarI, TM, AO, TT, and HO contributed to the interpretation of the results. SS, TarI, TM, AO, SO, and HO wrote the manuscript. All authors provided critical feedback and helped shape the research, analysis and manuscript. All authors approved the final submission of this manuscript.

#### FUNDING

This work was supported by a Grant-in-Aid for Young Scientists from MEXT, Japan (Grant Number 18K14840), a grant from Keio Gijuku Academic Development Funds, a grant of National Center for Child Health and Development (29-7), and a grant from Brain Mapping by Integrated Neurotechnologies for Disease Studies (Brain/MINDS) by AMED under the grant numbers JP18dm0207002 (TakI, ES, and SS) and JP18dm0207001 (HO).

#### ACKNOWLEDGMENTS

We thank the members of the electron microscope and Okano laboratories for their invaluable comments.

microscopy of biological specimens. Microsc. Microanal. 21, 231–238. doi: 10. 1017/S1431927614013579



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Shibata, Iseda, Mitsuhashi, Oka, Shindo, Moritoki, Nagai, Otsubo, Inoue, Sasaki, Akazawa, Takahashi, Schalek, Lichtman and Okano. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

digital media

of impactful research

article's readership