Abstract
“Brainless” cells, the living constituents inhabiting all biological materials, exhibit remarkably smart, i.e., stimuli-responsive and adaptive, behavior. The emergent spatial and temporal patterns of adaptation, observed as changes in cellular connectivity and tissue remodeling by cells, underpin neuroplasticity, muscle memory, immunological imprinting, and sentience itself, in diverse physiological systems from brain to bone. Connectomics addresses the direct connectivity of cells and cells’ adaptation to dynamic environments through manufacture of extracellular matrix, forming tissues and architectures comprising interacting organs and systems of organisms. There is imperative to understand the physical renderings of cellular experience throughout life, from the time of emergence, to growth, adaptation and aging-associated degeneration of tissues. Here we address this need through development of technological approaches that incorporate cross length scale (nm to m) structural data, acquired via multibeam scanning electron microscopy, with machine learning and information transfer using network modeling approaches. This pilot case study uses cutting edge imaging methods for nano- to meso-scale study of cellular inhabitants within human hip tissue resected during the normal course of hip replacement surgery. We discuss the technical approach and workflow and identify the resulting opportunities as well as pitfalls to avoid, delineating a path for cellular connectomics studies in diverse tissue/organ environments and their interactions within organisms and across species. Finally, we discuss the implications of the outlined approach for neuromechanics and the control of physical behavior and neuromuscular training.
Introduction
Cells of the human body populate their habitat through division, starting with two cells at conception and expanding to over 70 trillion cells over the course of a lifetime (). Throughout the lifespan of the organism they inhabit, cells memorialize the biophysical and chemical stimuli they experience via gene expression of structural proteins created from molecular building blocks, e.g., amino acids. In this way, cells encode an organism’s and their own experiences in the physical world, by creating and adapting tissues, throughout life. Just as punch cards encode the recursive logic of textile weaves created with weaving looms (where card holes allow passage of hooks and the fibers they shuttle), genes encode and translate the arrangement of amino acids comprising elastin, collagen and other structural proteins making up tissue weaves (, ; ,). A major barrier to understanding the emergent behavior that underpins this tissue genesis and adaptation is the lack of methods to image and analyze cellular connectivity across length and time scales.
The manuscript proposes a paradigm shifting approach to understand the cellular underpinnings of diseases as different as osteoarthritis and early onset dementia in bone and brain. We know as biologists that cells manufacture, remodel and adapt tissues throughout life (; ). The tissues render physically the collective cellular experience, reflected in architectures (bones) and memories (brain) which themselves exhibit emergent properties (). These emergent properties cannot simply be deduced from the individual parts, which themselves do not exhibit such properties; rather, these emergent properties arise from spatial and temporal arrangements among multiple parts, e.g., memories that are physically encoded in neurons are not observable in single neurons but rather emerge from the spatial arrangement and temporal behavior of interacting neurons in the brain. A pathological example of emergence would be disease emergence, e.g., of osteoarthritis in the musculoskeletal system or early onset dementia in the brain, which cannot be predicted based on the occurrence of a single sick cell but rather at the stage of loss in function or loss in return to homeostasis due to emergence of disease amongst groups of cells that interact.
The elucidation of such disease emergence represents a currently untenable yet compelling research problem. On the one hand, the lack of methods to probe and understand emergent behavior of inhabitant cells within their complex ecosystems presents a hurdle to understanding and fundamental discoveries. On the other hand, the role of cell populations and the loss of their connectivity in disease progression has been stymied by the tradeoff between achieving sufficient resolution across vastly different length and time scales, e.g., single field of view and single time point images (nano- to microscale for electron to optical microscopy), and other imaging modalities that enable high temporal albeit less spatial resolution (MRI). Rapid advances in the field seek to overcome this current hurdle. To address each of these points, workflows are needed to render and analyze vast amounts of imaging data from nano- to meso- length scales. The manuscript describes that process and sets a path forward.
The neuroscience community refers to the totality of cellular connections and their three-dimensional (3D) networks, e.g., in the brain, as the connectome and the process of rendering, analyzing and understanding the connectome as connectomics (; ; ). A recently integrated biosystems engineering, imaging and analysis platform enables a connectomics approach to map cellular connectivity across organs as diverse as brain and bone (; , ; ). Tested in mouse brains (; ; ; ; ; ; ) and in our own pilot studies of the human hip (; ; ), as well as validated through the delineation of standardized protocols and workflows (), these biosystems engineering approaches may find future applications relevant for every organ of the body.
Here, key enabling steps are described for quantifying relationships and connectivity between cells in different disease states. Specifically, we test machine learning algorithms with cellular network maps of the human hip to elucidate the role of cell networks in organ and organism (patho)physiology throughout life (Figure 1). This approach may pave the way for next generation theranostics, i.e., enabling prediction of emergent cell scale pathology, including disease detection as well as treatment, well before permanent damage occurs at tissue and organ length scales. Based on the results of this pilot study, we assess opportunities and identify potential pitfalls of the integrated imaging, modeling and machine learning approaches.
FIGURE 1
Datasets Rendered as Cellular Networks in Maps of Human Tissue
Human tissue samples from the femoral neck and head of patients undergoing total hip replacement were obtained with Institutional Review Board Approval (Cleveland Clinic IRB12-335). Samples were prepared for electron microscopy (EM) using a published protocol (
Samples were first etched (to expose cells just below the block face surface) and imaged using multi-beam Scanning Electron Microscopy, to achieve nano- to meso-scale renderings of cellular inhabitants (mainly osteocytes with some red blood cells visible in resulting images). Based on this protocol, carbon coating provided sufficient contrast to visualize osteocytes exposed by chemical etching on the surface of the sample block. Although not used for the current study, sequential layers of cellular networks could be revealed by reiterating the etching and imaging steps, resulting in a volume of tissue with fully rendered three dimensional (3D) cellular network.
Three datasets were acquired using three generations of multibeam Scanning Electron Microscopes (mSEM) to image three different samples, starting with a 61 beam prototype at 12 nm pixel size and ending with a state-of-the-art commercial system (Zeiss mSEM 505) (Table 1).
TABLE 1
| Data metrics | 1st generation (gen)# | 2nd gen+ | 3rd gen |
| Total area imaged (mm2) | 5.69 | 13.1 | 1,810 |
| Total images in area | 54,717 | 100,589 | 7,335,982 |
| Multibeam FOVs | 897 | 1649 | 120,262 |
| Pixels (megapixels) | 75,276 | 857,086 | 1.07 × 1010 |
| Size (Terabytes) | 0.08 | 0.87 | 10.98 |
Dataset metrics from three generations of mSEM maps from three different human hip samples obtained with IRB approval.
FOVs refers to Fields of View. See links for navigable, rendered maps comprising each dataset:
#https://www.mechbio.org/sites/mechbio/files/maps5/index.html (
+https://www.mechbio.org/sites/mechbio/files/maps7/index.html user: mechbio, password: #google-maps.
In our first pilot study (
Automation of Landmark Identification Using the You Only Look Once Machine Learning Algorithm
Increasing dataset sizes necessitated development of objective, automated methods for identification and quantification of cells, an ideal application for machine learning approaches. To this end, we implemented the so-called “You Only Look Once” (YOLO) machine learning algorithm (
FIGURE 2

Automated detection algorithm to classify osteocytes using manual methods and the You Only Look Once algorithm (YOLO;
FIGURE 3

Automated detection algorithm applied to the 2nd generation map (A), where detections with greater than 70% confidence (quantitative prediction of match to “ground truth” defined by training data) are depicted. (B) The entire map, depicting higher resolution details at increasing levels of zoom (C,D) in the Google Maps API. Note: the red circles indicate detected cells which are changed to green when the classifier (three or more processes) is met. Red circles above the dotted yellow line delineating the edge of the tissue surface (B, upper corner) are cells, vessels and artifacts outside of the femoral head and neck tissue. The map depicted here can be navigated and explored like Google Maps at http://www.mechbio.org/sites/mechbio/files/maps7/index.html to access the map, type in user: mechbio, password: #google-maps.
In summary, we acquired three datasets rendering osteocyte networks in tissues of the femoral head. The different datasets include data from different samples imaged using new generations of mSEM and associated increasing computational capacity. The first data set included 5.69 mm2 tissue and over 50,000 images (0.08 terabyte), with manual identification of 629 osteocytes taking several weeks’ time. To enable automated, rapid detection of osteocytes and in consideration of the increasing size and complexity of the datasets enabled through advances in the mSEM instrument and parallel computational advances (second and third generations) over the past decade (from an advanced prototype to a commercial system), we applied a machine learning algorithm to detect osteocytes based on the You Only Look Once (YOLO) convolutional neural network described originally by
We applied the machine learning approach to second and third generation maps, with the third data set comprising 1,810 mm2 tissue and over 7,000,000 images (10.98 TB), identifying a total of 206,180 osteocytes in 100 h on a graphics processing unit (GPU, GeForce GTX 1080 graphics card enabled) compared to the manual pinning method of our previously published work (
Osteocyte coordinates can be extracted from the YOLO classified image set, enabling high throughput analyses of massive datasets, which in the future could include other cellular inhabitants of tissues including blood cells, immune cells, chondrocytes, etc. While the method shows great promise for automated detection of cells, the greatest limitation of the method is the definition of appropriate and unbiased classifiers. The definition of osteocytes as pyknotic and viable based on the number of cell processes was shown to be flawed in a parallel study testing the assumption using biochemical based viability measures (
With these limitations in mind, the technological approach provides novel opportunities for a new field of cellular epidemiology, where emergent changes in cell health may in the future be used to predict disease outbreaks and prevent disease transmission, much like they are used at the length scale of human inhabitants of geographically defined environments (
Implications for Understanding Neuromechanics and the Control of Physical Behavior and Neuromuscular Training
In addition to its obvious application for development of next generation materials, devices and diagnostics, this disruptive biosystems engineering platform provides a novel tool for elucidating the relationship between neural and musculoskeletal connectomics, movement, navigation and memory (
Materials and Methods
Sample Preparation
Tissues were fixed in a combination of 4% formaldehyde and 2.5% paraformaldehyde in 0.2 M cacodylate buffer. Tissues were then embedded in poly(methyl methacrylate) under vacuum (
Identification of Relevant Landmarks, Creation of Training Datasets
Manual marking of landmarks was described in a previous paper (
Machine Learning Approach Using the You Only Look Once (YOLO) Algorithm
The “You Only Look Once” (YOLO) neural network automated object detection algorithm (
Initially, YOLO was trained for automated osteocyte detection, using 629 annotated cells, which were further augmented to 106 examples through variation by rotation, scale and contrast (Figure 2). Unseen images were then processed with YOLO and automatically detected objects were identified and marked by bounding boxes. The success of the YOLO algorithm has been proven for detecting osteocytes in the 2nd generation map within 100 h of testing. A straight-forward approach to improve the detector performance includes collection of more than 1,000 false- and missed-detections (Figure 4) to obtain a more representative training dataset.
FIGURE 4

Processing of images and application of the machine learning classification algorithm. (A) Module A preprocesses the mSEM output by stitching individual images into region-wide panoramas by the virtue of recorded image coordinates. In the interest of computational efficiency, the resulting image is down-scaled such that individual cells occupy circa 200 × 200 pixels. Here, the 11TB mSEM output from the 3rd generation map is stitched into 30 image regions amounting to 150 GB of data after downscaling. (B) Module B applies the pretrained object detector. (C) The output is a file listing that includes the location of each detected cell as a bounding box (X,Y,W,H), class (viable/pyknotic, annotated as “deceased”) and associated confidence p. Each of N detections results in a set of five predictions for each corresponding bounding box, including the x and y coordinates of the center of the bounding box (Xn,Yn), width and height of the bounding box (Wn,Hn), and the confidence of the detection (pn where a confidence or probability of 0 means no object was detected and 1.0 means a perfect match with “ground truth”). The object is then further classified as live or deceased (Cn). The object detector is pretrained using 600 living and 50 pyknotic examples (
Statements
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.
Ethics statement
The studies involving human participants were reviewed and approved by the Cleveland Clinic Institutional Review Board.
Author contributions
MK and DZ conceived the work. The specimens were collected and prepared for imaging by MK with assistance from the Cleveland Clinic and Zeiss teams. Image acquisition was carried out by the Zeiss Team, led by DZ and MK in the Demonstration Labs at Carl Zeiss Microscopy GmbH in Oberkochen. The machine learning algorithm was developed and tested by AS and CW. The manuscript was written by MK and was revised and approved of by all coauthors. All authors contributed to the article and approved the submitted version.
Funding
This work was supported in part through the National Health and Medical Research Council (Development Grant), Paul Trainor Foundation and the Alexander von Humboldt Foundation, and in kind support of Carl Zeiss Microscopy AG.
Acknowledgments
We express our appreciation to Ulf Knothe, M.D., D.Sc., for his invaluable role in acquiring specimens through the Cleveland Clinic (IRB), and Tomasz Garbowski for his mSEM expertise as well as Carl Zeiss Microscopy GmbH, for in kind use of their mSEM microscope and infrastructure. We acknowledge with appreciation colleagues and trainees for myriad contributions to this ongoing research project. T. Garbowski from the Zeiss Team provided invaluable assistance in acquiring images at the Zeiss Demonstration Labs.
Conflict of interest
Zeiss provided in kind support for this project, which is of a fundamental and translational nature. AS and CW were employed by Zeiss AG. DZ was employed by Carl Zeiss MultiSEM GmbH. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
AnastopolousS.Knothe TateM. L. (2021). Unleashing The Power Of Multimodal Imaging To Assess Cell Survival In Tissues Of The Osteoarthritic Hip.Orthop. Res. Soc. Meeting64:2406.
2
AndersonE. J.KreuzerS. M.SmallO.Knothe TateM. L. (2008). Pairing computational and scaled physical models to determine permeability as a measure of communication in micro- and nano-scale pericellular spaces.Microfluidics Nanofluid.4193–204.
3
BlakelyT. (2021). A Browsable Petascale Reconstruction of the Human Cortex. Available online at: https://ai.googleblog.com/2021/06/a.browstable-petascale-reconstruction-at.html(accessed June, 2021).
4
DongE.DuH.GardnerL. (2019). An interactive web-based dashboard to track COVID-19 in real time.Lancet Infect. Dis.20533–534. 10.1016/S1473-3099(20)30120-1
5
EberleA.-L.MikulaS.SchalekR.LichtmanJ.Knothe TateM. L.ZeidlerD. (2015). High-resolution, high-throughput imaging with a multiple beam scanning electron microscope.J. Microscopy259114–120. 10.1111/jmi.12224
6
EpsteinR. A.PataiE. Q.JulianJ. B.SpiersH. J. (2017). The cognitive map in humans: Spatial navigation and beyond.Nat. Neurosci.201504–1513. 10.1038/nn.4656
7
GüntherA.DedekK.HaverkampS.IrsenS.BriggmanK. L.MouritsenH. (2021). Double cones and the diverse connectivity of photoreceptors and bipolar cells in an Avian Retina.J. Neurosci.41, 5015–5028. 10.1523/JNEUROSCI.2495-20.2021
8
HayworthK. J.PealeD.JanuszewskiM.KnottG. W.LuZ.XuC. S.et al (2020). Gas cluster ion beam SEM for imaging of large tissue samples with 10 nm isotropic resolution.Nat. Methods17, 68–7110.1038/s41592-019-0641-2
9
HayworthK. J.XuC. S.LuZ.KnottG. W.FetterR. D.TapiaJ. C.et al (2015). Ultrastructurally smooth thick partitioning and volume stitching for large-scale connectomics.Nat. Methods12319–322. 10.1038/nmeth.3292
10
HuangS.-W.WangW.-T.ChouL.-C.LiaoC.-D.LiouT.-H.LinH.-W. (2015). Osteoarthritis increases the risk of dementia: a nationwide cohort study in Taiwan.Sci. Rep.5:10145. 10.1038/srep10145
11
JbabdiS.BehrensT. E. (2013). Long-range connectomics.Ann. N Y. Acad. Sci.130583–93. 10.1111/nyas.12271
12
KnappH. F.ReillyG. C.StemmerA.NiedererP.Knothe TateM. L. (2002). Development of preparation methods for and insights obtained from Atomic Force Microscopy of fluid spaces in cortical bone.Scanning2425–33. 10.1002/sca.4950240104
13
Knothe TateM. L. (2017). “Navigation of bee brains to human hips - microscopy and the modern Magellans,” in A new age in scanning electron microscopy: Applications in the life sciences (Washington, DC: AAAS), 19–23.
14
Knothe TateM. L. (2020). Advanced design and manufacture of mechanoactive materials - inspired by skin, bones and skin-on-bones.Front. Bioeng. Biotechnol.8:845. 10.3389/fbioe.2020.00845
15
Knothe TateM. L.FathT. (2016). The only constant is change: next generation medical devices for physical and mental health.Adv. Healthc. Mat.51840–1843.
16
Knothe TateM. L.DetamoreM.CapadonaJ.WooleyA.KnotheU. (2016c). Engineering and Commercialization of Human-Device Interfaces, from Bone to Brain.Biomaterials9535–46. 10.1016/j.biomaterials.2016.03.038
17
Knothe TateM. L.FallsT.MishraS.AtitR. (2010). Engineering an ecosystem: taking cues from nature’s paradigm to build tissue in the lab and the body.Fields Instit. Mathemat. Biol. Monogr. Ser. N. Perspect. Mathemat. Biol.57113–134.
18
Knothe TateM. L.GunningP. W.SansaloneV. (2016a). Emergence of form from function - mechanical engineering approaches to probe the role of stem cell mechanoadaptation in sealing cell fate.Bioarchitecture6:1229729. 10.1080/19490992.2016.1229729
19
Knothe TateM. L.TamiA. E. G.BauerT. W.KnotheU. (2002). Micropathoanatomy of Osteoporosis - Indications for a Cellular Basis of Bone Disease.Adv. Osteopor. Fracture Manage.29–14.
20
Knothe TateM. L.ZeidlerD.NathansonA.NgoL.GarbowskiT.SrikanthaA.et al (2019). Diagnostics of the future for musculoskeletal connectomics – from nano to mesoscale and prototype to pipeline.Trans. Arth. Res. Soc.62:2046.
21
Knothe TateM. L.ZeidlerD.PereiraA. F.HagemanD.GarbowskiT.MishraS.et al (2016b). Organ-to-Cell-Scale Health Assessment Using Geographical Information System Approaches with Multibeam Scanning Electron Microscopy.Adv. Healthc. Mat.51581–1587. 10.1002/adhm.201600026
22
LichtmanJ. W.PfisterH.ShavitN. (2014). The big data challenges of connectomics.Nat. Neurosci.171448–1454.
23
MikulaS.DenkW. (2015). High-resolution whole-brain staining for electron microscopic circuit reconstruction.Nat. Methods12541–546.
24
MikulaS.BindingJ.DenkW. (2012). Staining and embedding the whole mouse brain for electron microscopy.Nat. Methods91198–1201. 10.1038/nmeth.2213
25
NgJ.KilbreathS.KerschM.Knothe TateM. L. (2017a). Establishing the Basis for Mechanobiology-Based Physical Therapy Protocols to Potentiate Cellular Healing and Tissue Regeneration.Front. Physiol. Exerc. Physiol.2017:00303. 10.3389/fphys.2017.00303
26
NgJ.KnotheL.WhanR.KnotheU.Knothe TateM. L. (2017b). Scale-up of Nature’s Tissue Weaving Algorithms to Engineer Advanced Functional Materials.Sci. Rep.7:40396. 10.1038/srep40396
27
NgoL.NathansonA.GarbowskiT.KnotheU.ZeidlerD.Knothe TateM. L. (2019). Sample preparation protocol enabling nano-to-mesoscopic mapping of cellular connectomes and their habitats in human tissues and organs.Bio-protocol9:e3298. 10.1101/533448
28
PereiraA. F.HagemanD. J.GarbowskiT.RiedeselC.KnotheU.ZeidlerD.et al (2016). ‘Creating High-Resolution Multiscale Maps of Human Tissue Using Multi-beam SEM’.PLoS Comp. Biol.12:e1005217. 10.1371/journal.pcbi.1005217
29
PutraV. D. L.SongM. J.McBride-GagyiS.ChangH.PooleK.WhanR.et al (2019). Mechanomics approaches to understand cell behavior in context of tissue neogenesis, during prenatal development and postnatal healing.Front. Cell Dev. Biol.7:354. 10.3389/fcell.2019.00354
30
RedmonJ.DivvalaS.GirshickR.FarhadiA. (2015). You Only Look Once: Unified, Real-Time Object Detection.New Jersey, NJ: IEEE. 10.1109/CVPR.2016.91
31
ReillyG.KnappH.StemmerA.NiedererP.Knothe TateM. L. (2001). Investigation of the morphology of the lacunocanalicular system of cortical bone using atomic force microscopy.Ann. Biomed. Eng.291074–1081. 10.1114/1.1424910
32
RossiniP. M.MiragliaF.AlúF.CotelliM.FerreriF.De IorioR.et al (2020). Neurophysiological hallmarks of neurodegenerative cognitive decline: the study of brain connectivity as a biomarker of early dementia.J. Pers. Med.10:34. 10.3390/jpm10020034
33
SeungS. (2012). Connectome - How the Brain’s Wiring Makes us Who We Are.Boston: Mariner Books.
34
ShivaprasadP. (2019). A Comprehensive Guide To Object Detection Using YOLO Framework — Part 1.Toronto: towardsdatascience.
35
SwansonL. W.LichtmanJ. W. (2016). From Cajal to connectome and beyond.Annu. Rev. Neurosci.39197–216. 10.1146/annurev-neuro-071714-0333954
Summary
Keywords
connectomics, imaging, machine learning, cell, cell memory, cellular epidemiology
Citation
Knothe Tate ML, Srikantha A, Wojek C and Zeidler D (2021) Connectomics of Bone to Brain—Probing Physical Renderings of Cellular Experience. Front. Physiol. 12:647603. doi: 10.3389/fphys.2021.647603
Received
30 December 2020
Accepted
10 June 2021
Published
12 July 2021
Volume
12 - 2021
Edited by
Taiar Redha, Université de Reims Champagne-Ardenne, France
Reviewed by
Michael B. Morris, The University of Sydney, Australia; Navid Rabiee, Sharif University of Technology, Iran
Updates

Check for updates
Copyright
© 2021 Knothe Tate, Srikantha, Wojek and Zeidler.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Melissa L. Knothe Tate, knothetate@gmail.com; proftate.bmwi3@gmail.com
This article was submitted to Integrative Physiology, a section of the journal Frontiers in Physiology
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.